Virtual image control method and device and related equipment

Abstract

The invention provides a virtual image control method, and the method comprises the steps: obtaining live broadcast interaction data of a live broadcast room, the live broadcast interaction data comprises live broadcast bullet screen data and anchor voice data, and the live broadcast room comprises a virtual image and an anchor; according to live broadcast bullet screen data, determining a target question to be replied, the live broadcast bullet screen data being sent by audiences in a live broadcast room; determining target interaction content according to the target to-be-replied question, wherein the target interaction content comprises reply content of the target to-be-replied question; target interaction data are determined according to the target interaction content, the target interaction data are used for displaying the target interaction content to audiences in the live broadcast room through a virtual image, and the target interaction data comprise target audio data and target video data; and adding the target interaction data in the live broadcast data of the live broadcast room. Therefore, interaction with the audience is realized through the virtual image, so that the interaction between the audience and the anchor can be responded in time, and the watching experience of live broadcast is improved. The invention further provides a corresponding device and related equipment.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on August 3, 2023, with application number 202310973189.1 and invention name “A virtual image control method, computing cluster and related equipment”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of media technology, and in particular to a virtual image control method, device and related equipment. Background Art

[0003] With the continuous evolution of computer technology, the application of live broadcast is becoming more and more extensive. In the live broadcast scene, the anchor can upload the live broadcast data stream through the terminal device. The live broadcast data stream can be forwarded to the client corresponding to the live broadcast audience through the live broadcast server (or other devices). The live broadcast data stream may include video data stream and / or audio data stream.

[0004] In live broadcast scenarios, the anchor needs to show the live content to users. In addition, in some scenarios, the anchor is also responsible for managing the live broadcast room, interacting with other anchors and interacting with the audience. When the live broadcast work is busy, it may be difficult for the anchor to take care of all aspects. Summary of the invention

[0005] In view of this, the embodiment of the present application provides a method for controlling a virtual image, aiming to provide a systematic solution to the problem that the anchor is difficult to take into account all aspects of the live broadcast scene. The present application also provides a corresponding device, a computing device cluster, a computer-readable storage medium, and a computer program product.

[0006] In a first aspect, the present application provides a virtual image control method. The virtual image control method can be implemented by a virtual image control device in a live broadcast system, and the virtual image can be used to assist the broadcast. The virtual image control device can be deployed in at least one data center. Specifically, during the live broadcast in the live broadcast room, the virtual image control device can obtain live broadcast interaction data. The live broadcast interaction data includes live broadcast barrage data, that is, barrage data sent by the audience to the anchor, and also includes anchor voice data, that is, sound data emitted by the anchor in the live broadcast room during the live broadcast. Then, the virtual image control device can determine the target question to be answered based on the live broadcast barrage data. And determine the target interactive content based on the question to be answered and the live broadcast interaction data. The target interactive content includes the answer content of the target question to be answered. Then, the virtual image control method can determine the target interactive data based on the target interactive content, and add the target interactive data to the live broadcast data in the live broadcast room. The target interactive data is used to display the target interactive content to the audience in the live broadcast room through the virtual image. When viewing the live broadcast data with the target interactive data added, the audience can see both the live broadcast data corresponding to the anchor and the process of the virtual image displaying the target interactive content. During the live broadcast process, the virtual image control device can actively obtain the live broadcast interaction data of the live broadcast room, and determine the corresponding target interaction content based on the questions raised by the barrage, and then display the target interaction content to the audience in the live broadcast room through the virtual image. In this way, the live broadcast interaction data that needs to be interacted with can be automatically determined based on the live broadcast interaction data, and the target interaction content can also be displayed to the audience through the virtual image. Automatic interaction with the audience is achieved through the virtual image, so that the interaction between the audience and the anchor can be responded to in a timely manner, thereby improving the viewing experience of the live broadcast. In addition, by using the virtual image to assist the broadcast, the anchor does not need to focus on the interaction with the audience, but can focus more on the live broadcast content itself, thereby improving the efficiency of the anchor's live broadcast.

[0007] In some possible implementations, the target question to be answered can be determined based on the host's voice data. Specifically, when determining the target question to be answered, at least one question to be answered can be determined based on the live broadcast barrage data, and then, based on the host's voice data, the question to be answered that has not been answered by the host is selected from the at least one question to be answered as the target question to be answered. In other words, the host's voice data does not include audio data for answering the target question to be answered. The target question to be answered is a question raised by the audience and not answered by the host. In this way, the virtual image can be used to supplement the host's unanswered questions to improve the viewing effect of the live broadcast room.

[0008] In some possible implementations, when the host stops answering questions, the virtual image can also answer the target unanswered questions. Specifically, it is possible to detect whether the host's voice data pauses, and then add the target interaction data to the live broadcast data after the host's voice data pauses. In this way, it is possible to avoid the host and the virtual image making sounds at the same time to affect the live broadcast effect.

[0009] In some possible implementations, the target interactive content can be determined based on a pre-set virtual image knowledge base. Specifically, when determining the target question to be answered, the live broadcast barrage data can be clustered first to determine the intent information set. Then, the target question to be answered can be determined based on the barrage intent information set. In this way, by clustering the barrage data, meaningless information in the barrage can be filtered out, so that questions raised by the audience can be answered in a targeted manner. When determining the target interactive content based on the target question to be answered, the answer content of the target question to be answered can be determined from the virtual image knowledge base based on the target question to be answered, thereby determining the target interactive content based on the answer content of the target question to be answered. In this way, questions raised by the audience in the live broadcast room can be answered.

[0010] In some possible implementations, the virtual image knowledge base can also be updated. Specifically, if the host actively answers the target question to be answered, the virtual image knowledge base can be updated based on the target question to be answered and the audio data sent by the target host to answer the question to be answered. In other words, if the host actively answers a question raised by the audience, the virtual image knowledge base can also be updated based on the host's answer and the question raised by the audience. In this way, the virtual image knowledge base can be updated, so as to more accurately generate answers to questions raised by the audience later that are similar in style and content to the host's answer.

[0011] In some possible implementations, when generating target interactive data, audio data may be generated first and then video data. Specifically, corresponding target audio data may be generated first according to the target interactive content. Then, target video data may be generated according to relevant data of the virtual image and the target audio data. Among them, relevant data of the virtual image may include data such as model data of the virtual image. In this way, the target video data obtained may be the video data of the virtual image when it emits the audio corresponding to the target interactive content. From the perspective of the audience watching the live broadcast, it is equivalent to the virtual image emitting sound while synchronously performing corresponding actions. In this way, synchronizing the audio and video of the virtual image improves the fluency of the virtual image and enhances the live broadcast effect.

[0012] In some possible implementations, the target audio data can be matched with the state of the target anchor. Specifically, the anchor voice data can be analyzed to determine the anchor emotion category. The anchor emotion category represents the emotional state of the anchor in the live broadcast room. Then, the target audio data matching the anchor emotion category can be generated according to the target interactive content. In this way, the virtual emotions of the virtual image can change with the emotions of the anchor in the live broadcast room, which improves the synchronization rate between the virtual image and the real anchor, and can improve the live broadcast effect.

[0013] In some possible implementations, the host can also instruct the virtual image to complete the corresponding action through instructions. Specifically, the host's voice data can be parsed to determine the host's instruction information. Specifically, the instruction interaction data can be determined based on the host's instruction information, and the instruction interaction data can be added to the live broadcast data of the live broadcast room. The instruction interaction data is used to show the audience in the live broadcast room through the virtual image that the virtual image completes the action indicated by the host's instruction information. In other words, when the host instructs the virtual image to complete an action, multimedia data of the virtual image completing the action can be generated, and the multimedia data can be added to the live broadcast data of the live broadcast room. In this way, the audience watching the live broadcast can hear and / or see the virtual image completing the action indicated by the host. In this way, the host can instruct the virtual image to complete the action through instructions, which improves the flexibility of the virtual image and helps to improve the live broadcast effect.

[0014] In some possible implementations, there may be communication barriers between the host and the audience, for example, the language used by the host and the language used by the audience may be different. To this end, subtitles can also be added to the live broadcast room. Specifically, after the host's voice data is obtained, the host's voice data can be translated to obtain target subtitle data and add the target subtitle data to the live broadcast data of the live broadcast room. In this way, the audience in the live broadcast room can see the subtitles in the video screen of the live broadcast room, so as to understand what the host said. In this way, barrier-free communication between the host and the audience is achieved.

[0015] In some possible implementations, the live broadcast room can also be adjusted according to the host's instructions. Specifically, the target host can issue live broadcast room operation instructions through voice, and the live broadcast room operation instructions are used to adjust the live broadcast parameters of the live broadcast room. After obtaining the host's voice data, the host's voice data can be parsed to determine the live broadcast room operation instructions issued by the target host. Then, the virtual image control device can adjust the live broadcast data of the target live broadcast room according to the live broadcast room operation instructions. In this way, the target host can adjust the live broadcast data of the target live broadcast room through voice, without the need for the target host to adjust manually, thereby improving the live broadcast efficiency.

[0016] In a second aspect, the present application provides a virtual image control device, which is located in at least one data center, and the device includes: an acquisition module, used to acquire live interaction data of a live broadcast room, the live interaction data includes live barrage data and anchor voice data, and the live broadcast room includes a virtual image and an anchor; a question determination module, used to determine a target question to be answered based on the live barrage data, and the live barrage data is sent by the audience in the live broadcast room; an interactive content determination module, used to determine a target interactive content based on the target question to be answered and the live interaction data, and the target interactive content includes an answer content to the target question to be answered; an interactive data determination module, used to determine a target interactive data based on the target interactive content, and the target interactive data is used to display the target interactive content to the audience in the live broadcast room through a virtual image, and the target interactive data includes a target audio data and a target video data; a live data adjustment module, used to add the target interactive data to the live data of the live broadcast room.

[0017] In some possible implementations, the question determination module is specifically used to determine at least one question to be answered based on the live broadcast barrage data; and to determine a target question to be answered from at least one question to be answered based on the host's voice data, and the host's voice data does not include audio data corresponding to the answer content of the target question to be answered.

[0018] In some possible implementations, the problem determination module is also used to determine whether a pause occurs in the host's voice data.

[0019] In some possible implementations, the virtual image control device includes a virtual image knowledge base and a question determination module, which is used to cluster live broadcast barrage data and determine a barrage intention information set; determine a target question to be answered based on the barrage intention information set; an interactive content determination module, which is used to determine the answer content of the target question to be answered from the virtual image knowledge base based on the target question to be answered; and determine the target interactive content based on the answer content.

[0020] In some possible implementations, the question determination module is further used to update the virtual image knowledge base based on the target question to be answered and the audio data used to answer the target question to be answered.

[0021] In some possible implementations, the interactive data determination module is specifically used to generate corresponding target audio data based on the target interactive content; generate target video data based on the virtual image and the target audio data, and the target video data is the video data when the virtual image emits the audio corresponding to the target interactive content.

[0022] In some possible implementations, the interactive data determination module is specifically used to analyze the host's voice data to determine the host's emotion category; and generate target audio data that matches the host's emotion category based on the target interactive content.

[0023] In some possible implementations, the interactive data determination module is also used to determine the host's command information based on the host's voice data; determine the command interaction data based on the host's command information, and the command interaction data is used to show the audience in the live broadcast room through the virtual image that the virtual image completes the action indicated by the host's command information; the live broadcast data adjustment module is also used to add the command interaction data to the live broadcast data in the live broadcast room.

[0024] In some possible implementations, the live broadcast data adjustment module is further used to determine target subtitle data based on the host voice data; and add the target subtitle data to the live broadcast data in the live broadcast room.

[0025] In some possible implementations, the live broadcast data adjustment module is also used to parse the host's voice data and determine the live broadcast room operation instructions; and adjust the live broadcast data of the live broadcast room according to the live broadcast room operation instructions.

[0026] In the third aspect, the present application provides a computing device cluster, the computing device includes at least one computing device, the at least one computing device includes at least one processor and at least one memory; the at least one memory is used to store instructions, and the at least one processor executes the instructions stored in the at least one memory, so that the computing device cluster executes the multi-role multi-terminal real-time synchronous code review method in the above-mentioned second aspect or any possible implementation of the second aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The at least one computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0027] In a fourth aspect, the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on at least one computing device, the at least one computing device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0028] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0029] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0031] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0032] Figure 2 A schematic diagram of a flow chart of a virtual image control method provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a structure of a virtual image control provided in an embodiment of the present application;

[0034] Figure 4 A schematic diagram of a structure of a computing device provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of a structure of a computing device cluster provided in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of an implementation method of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The scheme in the embodiment provided in this application will be described below in conjunction with the drawings in this application.

[0038] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is just a way of distinguishing objects with the same attributes when describing the embodiments of this application.

[0039] In live broadcasting scenarios, there is interaction between the host and the audience. The audience can use the bullet screen to express their opinions or questions, and the host can view the bullet screen and give feedback, which may include video and audio. For another example, the audience can ask questions to the host by sending bullet screens, and the host can view the bullet screens and answer the questions in the bullet screens. For another example, the host can express his opinions during the live broadcast, and the audience can evaluate the host's opinions through the bullet screens.

[0040] However, in some scenarios, the host has a lot of work to do, and it is difficult to balance the live broadcast content and the interaction with the audience. For example, for game live broadcasts, the host may not be able to read and respond to the barrage in time while playing the game. For another example, for live broadcasts with goods, the host may not be clear about some information about the goods sold in the live broadcast room. If the host cannot interact with the audience in time, it may affect the audience's experience of watching the live broadcast.

[0041] For this reason, in some live broadcast rooms, there may be a role of a live broadcast assistant, which is used to assist the anchor to interact with the audience. In the embodiment of the present application, assisting the anchor to interact with the audience can be referred to as assisting the broadcast. In the embodiment provided in the present application, the assistant anchor has a virtual image.

[0042] Optionally, some anchors can hire staff as live broadcast assistants, so that the live broadcast assistants can complete part or all of the work of interacting with the audience. Specifically, the staff as live broadcast assistants can concentrate on watching the bullet screen information in the live broadcast room, and respond to the bullet screen information in a timely manner to complete the interaction with the audience. For example, the live broadcast assistant can be responsible for answering some questions in the bullet screen, thanking for gifts, and adjusting relevant parameters of the live broadcast room. In addition, the live broadcast assistant can also adjust the relevant settings of the live broadcast room, for example, it can be responsible for adjusting the brightness, filter, live broadcast bit rate and other related parameters of the live broadcast room.

[0043] However, this method requires hiring additional staff, which is costly. In addition, the staff who serve as live broadcast assistants may also make mistakes and miss interactive content.

[0044] Alternatively, some anchors can also assist in the interaction with the audience through a barrage reader. Among them, the barrage reader can be software installed on the terminal device used by the anchor. The barrage reader can obtain the barrage data of the live broadcast room, and thereby obtain the barrage data sent by the audience, convert part or all of the barrage into voice and play it to the anchor, so that the anchor can obtain the barrage sent by the user in the form of sound. Then, the anchor can respond according to the voice. In this way, there is no need for the anchor to check the barrage in person, which improves the efficiency of the interaction between the anchor and the audience. However, this method actually still relies on the anchor to complete the interaction with the barrage. Therefore, when the anchor is busy, he may not be able to interact with the audience in time.

[0045] Based on this, an embodiment of the present application provides a virtual image control method. Among them, the virtual image control method can be implemented by a virtual image control device in a live broadcast system, and the virtual image can be used to assist the broadcast. The virtual image control device can be deployed in at least one data center. Specifically, during the live broadcast of the target live broadcast room, the virtual image control device can obtain live broadcast interaction data. The live broadcast interaction data includes live broadcast barrage data, that is, barrage data sent by the audience to the anchor, and also includes anchor voice data, that is, sound data emitted by the anchor of the target live broadcast room during the live broadcast. Then, the virtual image control device can determine the target to-be-answered question based on the live broadcast barrage data. And determine the target interactive content based on the to-be-answered question and the live broadcast interaction data. The target interactive content includes the answer content of the target to-be-answered question. Then, the virtual image control method can determine the target interactive data based on the target interactive content, and add the target interactive data to the live broadcast data of the target live broadcast room. The target interactive data is used to display the target interactive content to the audience of the target live broadcast room through the virtual image. When viewing the live broadcast data with the target interactive data added, the audience can see both the live broadcast data corresponding to the anchor and the process of the virtual image displaying the target interactive content. During the live broadcast process, the virtual image control device can actively obtain the live broadcast interaction data of the live broadcast room, and determine the corresponding target interactive content based on the questions raised by the barrage, and then display the target interactive content to the audience in the target live broadcast room through the virtual image. In this way, the target interactive content can be automatically determined based on the live broadcast interaction data, and the target interactive content can also be displayed to the audience through the virtual image. The virtual image realizes automatic interaction with the audience, so that the interaction between the audience and the anchor can get a timely response, improving the viewing experience of the live broadcast. In addition, by using the virtual image to assist the broadcast, the anchor does not need to focus on the interaction with the audience, but can focus more on the live broadcast content itself.

[0046] The live broadcast system includes an anchor terminal, a live broadcast server and an audience client. Among them, the anchor terminal is used to provide services for assistant broadcasters, and the audience client is used to provide services for audiences. For example, the anchor terminal may include an anchor client and an anchor server. The audience client may include an audience client. The anchor client may be a live broadcast software installed on a terminal device used by the anchor, which is used to collect relevant data during the anchor's live broadcast. The anchor server may be software installed on a server, which is used to process relevant data during the live broadcast. Optionally, the anchor server may be installed in at least one data center. The audience client refers to software installed on a terminal device used by a user.

[0047] The anchor client can obtain the live data stream and send the live data stream to the anchor server. The anchor server can process the live data stream and forward it to the live server. The live server can send the live data stream to the audience client. The audience client can receive the live data stream and play it to the user in the form of video (or audio). As an example, the above-mentioned virtual image control device can be deployed on the anchor end of the live broadcast system. As another example, the above-mentioned virtual image control device can be deployed on the live broadcast server of the live broadcast system. The following description takes the virtual image control device deployed on the anchor end as an example.

[0048] For example, in Figure 1 In the application scenario shown, anchor A can use anchor terminal 10 for live broadcast, audience B can use audience client 20 to watch anchor A's live broadcast, audience C can use audience client 30 to watch anchor A's live broadcast, and live broadcast server 40 is used to realize data processing and forwarding during the live broadcast.

[0049] The host terminal 10 includes a live broadcast interactive device 110 and a virtual image control device 120. The live broadcast interactive device 110 is used to collect data generated by the host A during the live broadcast to obtain a live broadcast data stream, and send the live broadcast data stream to the live broadcast server 40. The data collected by the live broadcast interactive device 110 may include at least one of the video data collected by the camera, the audio data collected by the microphone, the video data and / or audio data collected by the screen recording, etc. The live broadcast interactive device 110 can also receive the bullet screen data sent by the live broadcast server 40, and display the bullet screen data to the host A.

[0050] The virtual image control device 120 can collect the live broadcast interaction data during the live broadcast of anchor A, determine the target interactive content based on the live broadcast interaction data, and then determine the target interactive data during the process of the virtual image displaying the target interactive content based on the virtual image, and add the target interactive data to the live broadcast data stream. In this way, the live broadcast data stream received by the live broadcast server 40 includes both the data collected by the live broadcast interaction device 110 and the target interactive data inserted by the virtual image control device 120. Then the video data seen by the audience client 20 and the audience client 30 includes both the video data broadcast by anchor A and the video data of the target interactive content displayed by the virtual image. In this way, the virtual image can be used to assist the anchor to interact with the audience.

[0051] like Figure 1As shown, the virtual image control device 120 may include an acquisition module 121, a question determination module 122, an interactive content determination module 123, an interactive data determination module 124, and a live broadcast data adjustment module 125. Among them, the acquisition module 121 can be used to acquire the live broadcast interaction data of the target live broadcast room. The question determination module 122 can be used to determine the target to-be-answered question based on the live broadcast barrage data. The interactive content determination module 123 can be used to determine the target interactive content based on the target to-be-answered question. The interactive data determination module 124 can be used to determine the target interactive data based on the target interactive content and the virtual image. The live broadcast data adjustment module 125 can be used to add the target interactive data to the live broadcast data of the target live broadcast room.

[0052] exist Figure 1 In the implementation shown, the virtual image control device 120 runs on the anchor terminal 10. Optionally, the anchor terminal 10 may include an anchor client and an anchor server. The anchor client may be installed on the terminal device used by anchor A, and the anchor server may be installed in a data center. Optionally, the virtual image control device 120 may be a module in the anchor client or a module in the anchor server. In other words, the virtual image control device 120 may be located in the terminal device or in the data center.

[0053] Next, various non-limiting specific implementations of the virtual image control process are described in detail.

[0054] See also Figure 2 , is a flow chart of a virtual image control method in an embodiment of the present application. The method can be applied to the above Figure 1 The application scenario shown in FIG. 1 may also be applied to other applicable application scenarios. Figure 1 The application scenario shown is used as an example for explanation. It should be noted that Figure 1 In the application scenario shown, the virtual image control device 120 may include not only an acquisition module 121, an interactive content determination module 123, an interactive data determination module 124, and a live data adjustment module 125, but may also include a virtual image knowledge base 126 and an emotion perception module 127. For the functions of each module, please refer to the relevant description of the following embodiment.

[0055] Figure 2 The method shown may specifically include:

[0056] S201: The acquisition module 121 acquires the live broadcast interaction data of the live broadcast room.

[0057] In order to assist the anchor to interact with the audience in the target anchor room, the virtual image control device 120 can obtain the live interaction data of the target live broadcast room, thereby determining the content to be displayed during the interaction process according to the live interaction data. Specifically, the virtual image control device 120 can obtain the live interaction data of the target live broadcast room through the acquisition module 121.

[0058] In the embodiment of the present application, the live broadcast room refers to the live broadcast room corresponding to the virtual image control device 120, which can also be called the target live broadcast room. Figure 1 In the implementation shown, the live broadcast room corresponding to anchor A can be regarded as the target live broadcast room. For the sake of ease of introduction, the anchor corresponding to the target live broadcast room can be referred to as the target anchor in this application.

[0059] Live interactive data refers to interactive data generated by the target live broadcast room during the live broadcast process, which is used to transmit information to the other party. Interactive data refers to data sent by an object to transmit information to other objects. Live interactive data can include live barrage data and anchor voice data. Among them, live barrage data is data sent by users during the live broadcast of the target live broadcast room, which is used to transmit information to the anchor. Anchor voice data is data corresponding to the voice emitted by the target anchor during the live broadcast process, which is used to interact with other objects.

[0060] Optionally, the host voice data may include audio data emitted during the communication between the target host and the audience, or may include audio data emitted during the communication between the target host and other objects. For example, if the target host is connected to another host, the host voice data may also include audio data emitted by the target host during the connection process.

[0061] In an embodiment of the present application, live broadcast barrage data can be obtained from the live broadcast server 40. Specifically, the audience of the target live broadcast room can send barrage to the live broadcast server 40 through the audience client. The live broadcast server 40 can receive barrages sent by multiple viewers who are watching the target live broadcast room, and send these barrages to the anchor terminal 10, so that the anchor client can show the barrage sent by the audience to the anchor. Correspondingly, the acquisition module 121 can obtain the live broadcast barrage data sent by the live broadcast server 40. Optionally, the live broadcast interaction device 110 can receive the barrage data sent by the live broadcast server. The acquisition module 121 can obtain the live broadcast barrage data sent by the live broadcast server 40 from the live broadcast interaction device 110 through a preset interface.

[0062] exist Figure 1In the implementation shown, the virtual image control device 120 is deployed on the anchor terminal 10. Accordingly, the acquisition module 121 can acquire the anchor voice data through the relevant modules of the anchor client, for example, the anchor voice data can be acquired through the microphone of the terminal device used by the anchor. According to the above introduction, the voice of the target live broadcast during the live broadcast can be sent to the live broadcast server 40 through the anchor terminal 10, and then sent to the audience client of the audience watching the target live broadcast room by the live broadcast server 40. In this way, the acquisition module 121 can acquire the anchor voice data from the anchor terminal 10.

[0063] It is understandable that if the virtual image control device is deployed at other locations, the host voice data can be obtained by parsing the live data stream of the target live broadcast room. For example, assuming that the virtual image control device is deployed on the live broadcast server. Then, when obtaining the live data stream sent by the host, the virtual image control device can parse the live data stream to obtain the host voice data therein.

[0064] In the live broadcast scenario, data is mostly transmitted in the form of data streams. Accordingly, the live broadcast interaction data acquired by the acquisition module 121 can be data in the data stream format. In other words, the acquisition module 121 can acquire the live broadcast barrage data stream and the anchor voice data stream. Accordingly, the subsequent processing based on the live broadcast interaction data can also be processing based on the data stream.

[0065] After acquiring the live interaction data of the target live broadcast room, the acquisition module 121 may send the live interaction data to the interactive content determination module 123 so that the interactive content determination module 123 may determine the target interactive content according to the target question to be answered.

[0066] S202: The question determination module 122 determines the target question to be answered based on the live broadcast barrage data.

[0067] After acquiring the live broadcast interaction data, the question determination module 122 can determine the target questions to be answered based on the live broadcast barrage data, so as to control the virtual image to answer the questions to be answered. Among them, the target questions to be answered are questions raised by the audience watching the target live broadcast room through the barrage. In the embodiment of the present application, the virtual image control device 120 can control the virtual image to answer some or all questions raised by the audience, so as to improve the audience's experience of watching the live broadcast and realize the function of a live broadcast assistant.

[0068] In order to control the virtual image to answer questions raised by the audience. First, the questions that need to be answered can be determined. In an embodiment of the present application, the questions to be answered, which are raised by the audience through the barrage and determined by the question determination module 122, can be referred to as target questions to be answered. Optionally, the target question to be answered can be one question or multiple questions. Specifically, the question determination module 122 can parse the live broadcast barrage data, determine the questions asked by the audience in the target live broadcast room, and then determine the questions to be answered from the questions asked by the audience in the live broadcast room.

[0069] In actual scenarios, the number of viewers in the target live broadcast room may be large, and each viewer watching the target live broadcast room can send barrages. The barrages sent by the audience can be used to ask questions to the target anchor, and can also have other purposes. Each viewer may ask questions to the target anchor, and the questions asked by different viewers may be the same or different. In addition, the same viewer may ask multiple different questions to the target anchor through multiple barrages, or may ask multiple identical or similar questions to the target anchor through multiple barrages. This results in the possibility that there may be multiple barrages for asking questions in the live broadcast barrage data, and the questions raised by different barrages may be the same or different. In addition, in addition to asking questions to the target anchor, the audience in the target live broadcast room can also achieve other purposes by sending barrages. This results in the existence of both barrages for asking questions and barrages that are not used for asking questions in the live broadcast barrage data.

[0070] To this end, the question determination module 122 can first pre-process the bullet screens in the live bullet screen data, filter out non-repetitive bullet screens used to ask questions to the target anchor from the live bullet screen data, and then determine the questions to be answered based on these bullet screens. Optionally, the live bullet screen data can be clustered first to determine the bullet screen intention information set, and then the target questions to be answered are determined based on the bullet screen intention information set. The bullet screen intention information set includes multiple bullet screen intention information, each bullet screen intention information corresponds to the intention information of one (or more) bullet screens, indicating the intention of the audience who sent this (or more) bullet screens.

[0071] First, the problem determination module 122 can first perform repeated filtering on the live broadcast bullet screen data. Among them, repeated filtering refers to deleting bullet screens with consistent content. In this way, bullet screens with consistent content can be deleted from the live broadcast bullet screen data, thereby reducing the number of bullet screens that need to be processed in subsequent steps.

[0072] Next, the question determination module 122 can cluster the filtered live broadcast barrage data to obtain a barrage intention information set. For example, clustering can be performed based on the similarity of the barrages, or it can be clustered based on the semantics of the barrages. The embodiment of the present application does not limit the clustering method. After determining the barrage intention information set, the question determination module 122 can determine multiple questions to be answered from the barrage intention information set. Each question to be answered corresponds to a barrage intention information, which represents a question raised by one (or more) viewers through the barrage.

[0073] After clustering the barrage, the barrage of the same type is clustered into one category. That is, if there are multiple barrages with similar expressions in the live barrage data, these barrages will be clustered into one category. Different types of barrages are used to express different meanings, and the same type of barrages are used to express the same meaning. In this way, in the subsequent processing process, a barrage in a category of barrages can be used to replace the barrage of that category, thereby ignoring the remaining barrages in that category of barrages. For example, the barrage at the center of the cluster can be used as a representative of the barrage of that category for subsequent processing. In this way, by replacing a category of barrages with one barrage, the number of barrages to be processed is further reduced, and the processing efficiency is improved. In addition, by clustering the barrage, the barrages that are not used to ask questions to the target anchor can also be filtered out. For example, if the live barrage data includes barrages used to ask questions to the target anchor, and also includes barrages used to communicate with other viewers, then after clustering the live barrage data, barrages with different purposes will be clustered into multiple categories respectively. The barrage used to ask questions to the target anchor will not be clustered into the same category as other barrages. Then, when determining the questions to be answered, only the bullet screens used to ask questions to the target anchor can be focused on, and bullet screens for other purposes can be ignored, thereby reducing the number of bullet screens to be processed. The specific implementation method of clustering is not limited in the present application embodiment.

[0074] After determining a plurality of questions to be answered, the question determination module 122 may determine a target question to be answered from the plurality of questions to be answered. Optionally, the question determination module 122 may evaluate the question according to its importance. For example, the question determination module 122 may evaluate the importance of each question to be answered respectively to obtain an importance evaluation result of the question to be answered. The importance evaluation result of the question to be answered indicates the importance of the question to be answered.

[0075] In some implementations, the question determination module 122 can perform an importance assessment on the question to be answered in combination with at least one assessment influencing factor, thereby obtaining an importance assessment result. It should be noted that the at least one assessment influencing factor used to perform an importance assessment on the target question to be answered may be an assessment influencing factor configured by an administrator; it may also be an assessment influencing factor adjusted from the initially set assessment influencing factor according to actual business needs; it may also be an assessment influencing factor newly configured by the administrator before receiving the target question to be answered. The embodiment of the present application does not limit the number and content of the configured assessment influencing factors.

[0076] In some embodiments, the configuration process of the evaluation rule is: receiving an evaluation rule configuration request, the evaluation rule configuration request is used to request configuration of one or more of the type, priority and weight of at least one evaluation influencing factor. Based on this, according to the information carried in the evaluation rule configuration request, at least one evaluation influencing factor and the priority and weight of each evaluation influencing factor are stored.

[0077] Optionally, when there are multiple configured evaluation influencing factors, the question determination module 122 may also respond to the administrator's configuration and select a preset number of evaluation influencing factors with higher priorities to perform importance assessment on the questions to be answered. The embodiment of the present application does not limit the number of evaluation influencing factors used when performing importance assessment on the questions to be answered.

[0078] After determining the importance evaluation result of each question to be answered, the target question to be answered can be determined according to the importance evaluation result of the question to be answered. Optionally, the N questions to be answered with the highest importance can be determined as the target questions to be answered. N is a positive integer. Optionally, N can be a pre-configured number of questions that the avatar can answer.

[0079] In some implementations, the questions raised by the audience through the barrage may be relatively abstract, and it is not easy to determine the content of the answer. Accordingly, after selecting the questions to be answered, before determining the target questions to be answered, the questions to be answered can be clustered to determine the root problem of the questions to be answered, and the root problem can be used as the target question to be answered. Optionally, the question determination module 122 can determine the root problem corresponding to the question to be answered from the corpus by means of semantic clustering. Among them, a plurality of root problem corpora are pre-stored in the corpus, and each root problem corpus corresponds to a root problem. Optionally, the corpus can be a virtual image knowledge base 126.

[0080] In a possible implementation, the multiple fundamental question corpora pre-stored in the corpus can be divided into multiple levels, each level corresponds to at least one fundamental question corpus, and the granularity of the fundamental question and answer corpora corresponding to different levels can be different. Optionally, the corpus includes multiple corpus scenarios, such as live broadcast, conference chat, webinar, online class, etc.; each corpus scenario corresponds to a category tree, which includes multiple levels, each level has different granularity, each level includes at least one category node, and each category node corresponds to at least one fundamental question. Granularity indicates the degree of detail of the description of the fundamental question and answer corpus in this level. The granularity of the high level is large, and the problem description of the fundamental problem corresponding to its category node is relatively macroscopic; the granularity of the low level is small, and the problem description of the fundamental problem corresponding to its category node is more detailed.

[0081] When classifying the questions to be answered, the classification can be performed according to the target level. Specifically, the multiple fundamental questions included in the target level can be determined first, and then the fundamental questions corresponding to the questions to be answered can be selected from the multiple fundamental questions as the target questions to be answered. Among them, the target level for classification can be a pre-set level. In this way, multiple fundamental questions can be obtained from the fundamental question and answer corpus corresponding to the target level, so that the selection of the target questions to be answered is more accurate. At the same time, the amount of calculation for semantic clustering based on the corpus is reduced, and the efficiency of semantic clustering is greatly improved.

[0082] In an actual scenario, the audience of the target live broadcast room may ask multiple questions to the target anchor, and the target anchor answers some of the multiple questions but does not answer the remaining questions. Then, when controlling the virtual image to interact with the audience, the questions raised by the audience that have not been answered by the target anchor can be answered, while the questions that have been answered by the target anchor are not answered. That is, when determining the target questions to be answered, the question determination module 122 can select the questions to be answered that have not been answered by the target anchor as the target questions to be answered.

[0083] Specifically, the question determination module 122 can determine at least one question to be answered based on the live broadcast barrage data, and then determine the target question to be answered from the questions to be answered based on the host voice data. Specifically, the question determination module 122 can analyze the host voice data, so as to determine which questions to be answered that the target host has answered based on the host voice data. Then, the question determination module 122 can exclude the questions to be answered that have been answered by the target host from the barrage question set. Then the remaining questions to be answered in the barrage question set are the questions to be answered that have not been answered by the target host. The question determination module 122 can determine the target question to be answered therefrom.

[0084] In some possible implementations, the question determination module 122 can convert the speech emitted by the target anchor into text, so as to determine the unanswered questions that the target anchor has answered through the text. Specifically, the question determination module 122 can call the speech-to-text function of the anchor terminal 10 to convert the anchor's voice data into text data. Then, the question determination module 122 can analyze the converted text data to determine the information expressed by the target anchor during the live broadcast process. Then, the bullet screen question information set can be used to match the text data to determine the part of the text data used to answer the unanswered questions raised by the audience.

[0085] Optionally, the virtual image control device 120 can assist the target anchor to interact when the target anchor pauses. The pause of the target anchor may include a pause in the speech of the target anchor. Specifically, if the virtual image control device 120 also controls the virtual image to interact with the audience while the target anchor interacts with the audience, it may result in two objects interacting with the audience at the same time in the live broadcast, affecting the audience's viewing experience. To this end, in some possible implementations, the virtual image can be controlled to interact with the audience when the anchor pauses.

[0086] That is to say, the problem determination module 122 can determine whether the target anchor has a pause based on the anchor's voice data. Specifically, the problem determination module 122 can determine whether a pause occurs in the anchor's voice data stream. For example, it can be monitored whether there is a part including the anchor's voice in the anchor's voice data stream. If a pause occurs, for example, the anchor's voice does not appear within a preset time length, it means that the target anchor has a pause during the live broadcast. In this way, the problem determination module 122 can analyze the anchor's voice data to determine the target question to be answered. In this way, it can avoid the target anchor and the virtual image from interacting with the audience at the same time, thereby improving the audience's experience of watching the live broadcast.

[0087] Moreover, when the anchor pauses, there is a high possibility that the questions raised by the audience cannot be answered. For example, the anchor may leave the terminal device used for live broadcast for some reason, resulting in the anchor being unable to see the questions raised by the audience, so the anchor cannot answer the questions raised by the audience. For another example, the anchor may turn off the microphone for some reason, and the anchor end does not collect the voice emitted by the anchor, so the audience cannot obtain the anchor's answer to the question. In the above scenario, the anchor's pause is also accompanied by the problem that the questions raised by the audience are not answered in time. Therefore, after determining that the target anchor has paused, the virtual image control device 120 is used to interact with the audience, so that the problem of the audience's questions not being answered in time can be overcome in time.

[0088] Optionally, the live broadcast interaction data may be determined after determining that the target live broadcast is in the live broadcast process. For example, the bullet screen data within a preset time before a pause may be used as the live bullet screen data in the live broadcast interaction data, and / or the audio data emitted by the anchor within a preset time before a pause may be used as the anchor voice data in the live broadcast interaction data.

[0089] S203: The interactive content determination module 123 determines the target interactive content according to the target question to be answered.

[0090] After determining the target question to be answered, the interactive content determination module 123 can determine the target interactive content according to the target answer question. The target interactive content may include the answer content of the target question to be answered. In other words, in order to assist the target anchor to interact with the audience of the target live broadcast room, the virtual image control device 120 can imitate the target anchor and determine the target interactive content for responding to the barrage. That is, the target interactive content can be used to respond to the barrage on behalf of the target anchor.

[0091] In the embodiment of the present application, the target anchor's response to the barrage includes the target anchor's answer to the question raised by the audience. Accordingly, the virtual image control device 120 can control the virtual image to answer the question raised by the audience instead of the target anchor. The target interactive content may include the answer to the question raised by the audience. That is, the interactive content determination module 123 can determine the content that needs to be answered to the audience in the target live broadcast room based on the live broadcast interaction data.

[0092] After determining the target question to be answered, the interactive content determination module 123 can determine the answer to the target question to be answered, and the answer to the target question to be answered is the aforementioned target interactive content. Optionally, if the interactive content determination module 123 determines multiple target questions to be answered, the target interactive content can include the answer to each target question to be answered. In addition, the order of the answers to the target questions to be answered in the target interactive content can match the order in which the target questions to be answered are raised.

[0093] Optionally, the answer content of the target question to be answered can be determined based on the virtual image knowledge base 126 of the anchor terminal 10, and then the target interactive content is generated according to the answer content. The answer content of the target question to be answered can be the answer to the target question to be answered. The virtual image knowledge base 126 is a pre-set database for answering questions raised by the audience. The virtual image knowledge base 126 can include multiple preset questions and answers corresponding to each preset question. After determining the target question to be answered, a preset question corresponding to the target question to be answered can be selected from multiple preset questions according to the target question to be answered, and the answer to the preset question is determined as the target interactive content.

[0094] Optionally, the avatar control device 120 may also update the avatar knowledge base 126. For example, if the target host actively answers a question raised by the audience, the avatar knowledge base 126 may be updated according to the target host's answer and the question raised by the audience. Specifically, if the target host actively answers the target question to be answered, the avatar knowledge base may be updated according to the target question to be answered and the audio data sent by the target host to answer the question to be answered.

[0095] In the implementation described above, the target interactive content determined by the interactive content determination module 123 can be used to replace the host in answering the audience's questions. In some other possible implementations, the interactive content determination module 123 can also determine the target interactive content in other interactive scenarios.

[0096] In some implementations, the target interactive content can also be used to control the avatar to complete certain actions. The interactive content determination module 123 can determine the action that the avatar needs to complete based on the host voice data and / or the live broadcast barrage data. For example, the target host can instruct the avatar to complete certain actions through voice, and the audience can also instruct the avatar to complete certain actions by sending barrages. For the sake of ease of introduction, the action that the avatar is instructed to complete is referred to as the target action below.

[0097] Take the example of a target host instructing a virtual image to complete a target action through voice. The interactive content determination module 123 may first determine the instruction information based on the host's voice data, and then determine the target interactive content based on the instruction information. The target interactive content corresponds to the target action, and may include, for example, model data of the virtual image in the process of completing the target action. By rendering the model data, video data of the virtual object completing the target action can be obtained.

[0098] Optionally, a plurality of preset instruction information may be pre-set in the virtual image control device 120, and a corresponding relationship between the preset instruction information and the interactive content may be established. In this way, the target anchor and / or the audience of the target live broadcast room may instruct the virtual image to complete the corresponding interactive content through the preset instruction information. Specifically, after the anchor voice data or the live broadcast barrage data is parsed to obtain the instruction information, if the instruction information matches a certain preset instruction information, the interactive content corresponding to the preset instruction information may be determined as the target interactive content.

[0099] After the interactive content determination module 123 determines the target interactive content, the interactive content determination module 123 may send the target interactive content to the interactive data determination module 124 so that the interactive data determination module 124 may determine the target interactive data.

[0100] S204: The interaction data determination module 124 determines target interaction data according to the target interaction content.

[0101] After determining the target interactive content according to the live interactive data, the interactive data determination module 124 may determine the target interactive data according to the target interactive content. The target interactive data user displays the target interactive content to the audience of the target live broadcast room through a virtual image, including target video data and target audio data. The virtual image may be a pre-set animated image or a physical image, which is used to assist the target anchor to interact with the bullet screen.

[0102] The target interaction data includes media data, which is used to display the target interaction content to the audience of the target live broadcast room through a virtual image. By inserting the target interaction data into the live broadcast data of the target live broadcast room, the audience of the target live broadcast room can see and / or hear the target interaction content displayed by the virtual image. Specifically, the target interaction data may include video data, which is data for the virtual image to display the target interaction content, for example, it may include animation data of the virtual image completing the target action. The target interaction data may also include audio data, which corresponds to the target interaction content. In order to distinguish it from the audio data and video data in the live broadcast data stream, the audio data in the target interaction data is referred to as the target audio data, and the video data in the target interaction data is referred to as the target video data.

[0103] As described above, the target interactive content may include answers to questions raised by the audience, or model data of the process in which the avatar completes the target action. The following introduces these two implementation methods respectively.

[0104] In a first implementation, the target interactive content includes answers to questions raised by the audience. The target interactive data may include target audio data and target video data. When determining the target interactive data, the interactive data determination module 124 may first determine the target audio data based on the target interactive content, and then determine the target video data based on the target audio data and the virtual image.

[0105] Specifically, if the target interactive content is data in text format, the interactive data determination module 124 may first convert the target interactive content into target audio data. For example, the target audio data corresponding to the target interactive content may be generated according to a preset audio database. The target audio data is used to simulate the virtual image reading out the text corresponding to the target interactive content. In this way, the target audio data is inserted into the live broadcast data, and users watching the target live broadcast room can listen to the target audio data and know the answer to the question.

[0106] Optionally, the interactive data determination module 124 may also generate target audio data with reference to the situation of the target host, so that the generated target audio data is combined with the situation of the target host, so that the virtual image matches the actual situation of the target host, thereby improving the broadcasting effect.

[0107] Specifically, the host voice data can be analyzed by the emotion perception module 127 to determine the emotional information of the target host during the live broadcast. Then, the interactive data determination module 124 can generate target audio data corresponding to the emotional information according to the target interactive content. Optionally, multiple audio databases can be pre-set in the virtual image control device 120, and different audio databases correspond to different emotional information. For example, the audio of the audio database A in the multiple audio databases can be recorded under an angry mood, and the audio of the audio database B in the multiple audio databases can be recorded under a happy mood. After determining the emotional information, the audio database that matches the emotional information can be selected according to the emotional information, and the target audio data can be generated using the selected audio database. In this way, the emotions of the generated target audio data match the actual emotions of the target host, and the emotions of the target host felt by the audience in the target live broadcast room match the emotions of the virtual image, achieving the effect of the virtual image simulating the emotions of the target host.

[0108] After the target audio data is generated according to the target interactive content, the corresponding target video data can be generated according to the target audio data and the virtual image. Among them, the target video data can be a video of the virtual image emitting the target audio data. That is, if the virtual image includes a face, then the target video data can show the facial movements of the virtual image during the process of emitting the target audio data. For example, if the virtual image is an animated image corresponding to the target anchor, then the video data can include an animation of the animated image emitting the sound corresponding to the target audio data. The target video data can be synchronized with the target audio data. That is, the virtual image displayed by the target video data at a certain point in time is consistent with the virtual image that emits the target audio data at that point in time.

[0109] In this way, after the target video data and the target audio data are inserted into the live broadcast data of the target live broadcast room, the audience watching the target live broadcast room can hear the audio corresponding to the target interactive content and see the image of the virtual image emitting the audio corresponding to the target interactive content. In this way, the audience watching the target live broadcast room is equivalent to watching the virtual image read out the audio corresponding to the target interactive content, achieving the purpose of answering the questions raised by the audience in the target live broadcast room through the virtual image, and improving the viewing experience of the audience in the target live broadcast room.

[0110] In a second implementation, the target interactive content includes model data of the virtual image in the process of completing the target action.

[0111] The target interaction data may include target video data. When determining the target interaction data, the interaction data determination module 124 may render the above-mentioned model data, and render the model data into visual two-dimensional image data to obtain the target video data. Optionally, if the avatar is an animated avatar, the rendered target video data may be animation data of the avatar in the process of completing the target action.

[0112] Optionally, the target interaction data may also include target audio data. The target audio data may correspond to the target video data. For example, the target audio data may be determined based on the target action. Specifically, assuming that multiple actions that the avatar can perform are preconfigured, audio data may be set for one or more of the multiple actions. When determining the target interaction data, the interaction data determination module 124 may select the audio data corresponding to the target action as the target audio data.

[0113] After acquiring the live broadcast interaction data, the interaction data determination module 124 may send the target interaction data to the live broadcast data adjustment module 125 so as to insert the target interaction data into the live broadcast data of the target live broadcast room.

[0114] S205: The live broadcast data adjustment module 125 adds the target interactive data to the live broadcast data of the target live broadcast room.

[0115] After determining the target interaction data corresponding to the live broadcast interaction data, the live broadcast data adjustment module 125 may add the target interaction data to the live broadcast data of the target live broadcast room.

[0116] In an actual scenario, the live broadcast data of the target live broadcast room can be a data stream. The live broadcast interactive device 110 can collect the video data stream and the audio data stream, and upload the two as the live broadcast data stream of the target live broadcast room to the live broadcast server 40. In an embodiment of the present application, the live broadcast data adjustment module 125 can add the target interactive data to the live broadcast data stream of the target live broadcast room in the form of a video stream.

[0117] Specifically, if the target interactive data includes target video data and target audio data, the live data adjustment module 125 can first determine the target video data stream and the target audio data stream, and synchronize the target video data stream and the target audio data stream with the live data stream of the target live room.

[0118] For the target video data stream, the live data adjustment module 125 can add it to the video data stream of the live data stream to obtain a video data stream with the target interactive data added. For example, the live data adjustment module 125 can add the picture corresponding to the target video data stream to a specific position in the live picture of the target live broadcast room. In this way, when watching the target live broadcast, the audience in the target live broadcast room can see the target live broadcast content at the above-mentioned specific position while the virtual image displays the target interactive content. At the above-mentioned specific position, the audience can see the live broadcast content of the target anchor at a position other than the above-mentioned specific position.

[0119] For the target audio data stream, the live data adjustment module 125 can add it to the audio data stream of the live data stream to obtain an audio data stream with the target interactive data added. For example, the live data adjustment module 125 can superimpose the target audio data stream with the audio data stream of the live data stream. In this way, when watching the target live broadcast, the audience in the target live broadcast room can hear the sound emitted by the virtual image during the display of the target interactive content, and can also hear the voice of the target anchor during the live broadcast.

[0120] In this way, viewers watching the target live broadcast room can not only see the target anchor in the live broadcast, but also see the target interactive content displayed through the virtual image. This is equivalent to using the virtual image to assist the anchor to interact with the audience in the live broadcast room, so that the interaction between the audience and the anchor can be responded to in a timely manner, improving the viewing experience of the live broadcast.

[0121] In the above implementations, the virtual image control device can control the virtual image to assist the anchor to interact with the audience in the live broadcast room. In some possible implementations, the virtual image control device can also have the function of assisting the anchor in live broadcasting. The following introduces two implementations of the virtual image control device assisting the anchor in live broadcasting. It is understandable that the virtual image control device can also assist the anchor in live broadcasting in other ways. I will not go into details here.

[0122] In a first implementation, the virtual image control device may have language translation and subtitle display functions to assist the target anchor and the audience watching the target live broadcast room to understand each other's language.

[0123] In actual scenarios, there may be situations where the host and the audience do not speak the same language. For example, the target host and the audience in the target live broadcast room may come from different countries or regions. In this way, the audience in the target live broadcast room may not understand the words spoken by the target host during the live broadcast, and the target host may not understand the meaning of the barrage sent by the audience. For this reason, the virtual image control device can have a language translation function to assist the host and the audience in the target live broadcast room to communicate.

[0124] On the one hand, after acquiring the live broadcast barrage data, the virtual image control device can translate the live broadcast barrage data from the second target language into text in the first target language. Then, the virtual image control device can display the translated text in the first target language on the anchor end. The target anchor can understand the content of the barrage sent by the audience in the target live broadcast room by viewing the content displayed by the virtual image control device. Among them, the first target language is the language used by the target anchor, and the second voice is the language used by the audience in the target live broadcast room. Optionally, the first target voice and the second target language can be set on the anchor end.

[0125] On the other hand, after acquiring the host voice data, the virtual image control device can first convert the host voice data into text data, and then convert the text in the first language into the text in the second target language to obtain the target subtitle data. Then, the virtual image control device can add the target subtitle data to the live broadcast data of the target live broadcast room. For example, the virtual image control device can generate a video data stream based on the target subtitle data, and add the video data stream generated based on the target subtitle data to the live broadcast data stream of the target live broadcast room. In this way, the audience watching the target live broadcast room can see the target subtitle data in the live broadcast screen of the target live broadcast room, so as to understand the words spoken by the target host during the live broadcast.

[0126] In the second implementation, the virtual image control device can assist the target anchor to control the target live broadcast room.

[0127] In actual scenarios, the anchor can control various live broadcast parameters of the live broadcast room, such as adjusting the brightness of the live broadcast room, the filter of the live broadcast room, the shooting angle of the camera, etc. At present, the control of the live broadcast parameters of the live broadcast room needs to be done manually by the anchor, which is inefficient. In the embodiment of the present application, the virtual image control device can help the target anchor adjust the live broadcast parameters of the target live broadcast room, thereby improving the live broadcast efficiency of the target anchor.

[0128] Specifically, the target anchor can issue a live broadcast room operation instruction through voice, and the live broadcast room operation instruction is used to adjust the live broadcast parameters of the live broadcast room. After obtaining the anchor voice data, the virtual image control device can parse the anchor voice data to determine the live broadcast room operation instruction issued by the target anchor. Then, the virtual image control device can adjust the live broadcast parameters of the target live broadcast room according to the live broadcast room operation instruction. In this way, the target anchor can adjust the live broadcast parameters of the target live broadcast room through voice, without the need for the target anchor to adjust manually, thereby improving the live broadcast efficiency.

[0129] The present application also provides a virtual image control device, wherein the virtual image control device can be applied to Figure 1 The anchor terminal 10 in the embodiment shown in the embodiment can be operated on the anchor client or the anchor server, for example, to implement Figure 2 The functions of the virtual image control device in the implementation shown are as follows. Figure 3 As shown, the virtual image control device 300 includes:

[0130] An acquisition module 310 is used to acquire live broadcast interaction data of a live broadcast room, wherein the live broadcast interaction data includes live broadcast barrage data and the host voice data, and the live broadcast room includes the virtual image and the host;

[0131] A question determination module 320, configured to determine a target question to be answered based on the live broadcast barrage data, wherein the live broadcast barrage data is sent by viewers in the live broadcast room;

[0132] An interactive content determination module 330, configured to determine target interactive content according to the target question to be answered and the live interactive data, wherein the target interactive content includes an answer content to the target question to be answered;

[0133] An interactive data determination module 340 is used to determine target interactive data according to the target interactive content, wherein the target interactive data is used to display the target interactive content to the audience of the live broadcast room through the virtual image, and the target interactive data includes target audio data and target video data;

[0134] The live broadcast data adjustment module 350 is used to add the target interactive data to the live broadcast data of the live broadcast room.

[0135] Among them, the acquisition module 310, the problem determination module 320, the interactive content determination module 330, the interactive data determination module 340 and the live data adjustment module 350 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the problem determination module 320 is introduced below by taking the problem determination module 320 as an example. Similarly, the implementation of the acquisition module 310, the interactive content determination module 330, the interactive data determination module 340 and the live data adjustment module 350 can be the implementation of the problem determination module 320.

[0136] As an example of a software functional unit, the problem determination module 320 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the problem determination module 320 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0137] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0138] As an example of a hardware functional unit, the problem determination module 320 may include at least one computing device, such as a server, etc. Alternatively, the problem determination module 320 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0139] The multiple computing devices included in the problem determination module 320 can be distributed in the same region or in different regions. The multiple computing devices included in the problem determination module 320 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the problem determination module 320 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0140] It should be noted that, in other embodiments, the acquisition module 310 can be used to execute any step in the virtual image control method, the problem determination module 320 can be used to execute any step in the virtual image control method, the interactive content determination module 330 can be used to execute any step in the virtual image control method, the interactive data determination module 340 can be used to execute any step in the virtual image control method, and the live data adjustment module 350 can be used to execute any step in the virtual image control method. The steps that the acquisition module 310, the problem determination module 320, the interactive content determination module 330, the interactive data determination module 340 and the live data adjustment module 350 are responsible for implementing can be specified as needed. By implementing different steps in the virtual image control method, the acquisition module 310, the problem determination module 320, the interactive content determination module 330, the interactive data determination module 340 and the live data adjustment module 350 can realize all the functions of the virtual image control device.

[0141] The present application also provides a computing device 100. Figure 4 As shown, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102. The computing device 100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0142] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus 102 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0143] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0144] The memory 106 may include a volatile memory, such as a random access memory (RAM). The processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0145] The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned acquisition module 310, the problem determination module 320, the interactive content determination module 330, the interactive data determination module 340, and the live broadcast data adjustment module 350, thereby implementing the virtual image control method. That is, the memory 106 stores instructions for executing the storage method.

[0146] Alternatively, the memory 106 stores executable codes, and the processor 104 executes the executable codes to implement the functions of the aforementioned virtual image control device, thereby implementing the virtual image control method. That is, the memory 106 stores instructions for executing the virtual image control method.

[0147] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0148] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0149] like Figure 5As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the virtual image control method.

[0150] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the virtual image control method. In other words, the combination of one or more computing devices 100 may jointly execute the instructions for executing the virtual image control method.

[0151] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the virtual image control device. That is, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more modules among the acquisition module 310, the problem determination module 320, the interactive content determination module 330, the interactive data determination module 340 and the live broadcast data adjustment module 350.

[0152] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 6 A possible implementation is shown. Figure 6 As shown, two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the acquisition module 310 and the live data adjustment module 350. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the problem determination module 320, the interactive content determination module 330, and the interactive data determination module 340.

[0153] Figure 6 The connection method between the computing device clusters shown can be based on the consideration that the virtual image control method provided in the present application can be divided into two steps: determining interactive data and adjusting live broadcast data. Therefore, it is considered that the function of determining the target interactive data is executed by the computing device 100B, and the function of adjusting the live broadcast data is executed by the computing device 100B.

[0154] It should be understood that Figure 6 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0155] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Figure 5 and Figure 6 The connection mode of the computing device cluster is different in that the memory 106 in one or more computing devices 100A in the computing device cluster may store the same instructions for executing the virtual image control method.

[0156] In some possible implementations, the memory of one or more computing devices 100B in the computing device cluster may also store partial instructions for executing the virtual image control method. In other words, the combination of one or more computing devices may jointly execute the instructions for executing the virtual image control method.

[0157] It should be noted that the memory 106 in different computing devices 100A in the computing device cluster can store different instructions for executing part of the functions of task processing. That is, the instructions stored in the memory 106 in different computing devices 100A can implement the functions of one or more devices in the client, server and model.

[0158] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the virtual image control method.

[0159] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the virtual image control method, or instruct the computing device to execute the virtual image control method.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual image control method, characterized in that: The method is applied to a virtual image control device, the virtual image control device is located in at least one data center, and the method includes: Acquire live broadcast interaction data of a live broadcast room, wherein the live broadcast interaction data includes live broadcast barrage data and the host voice data, and the live broadcast room includes the virtual image and the host; Determine a target question to be answered based on the live broadcast barrage data, wherein the live broadcast barrage data is sent by a viewer of the live broadcast room; Determine target interactive content according to the target question to be answered, the target interactive content including the answer content of the target question to be answered; Determine target interaction data according to the target interaction content, the target interaction data is used to display the target interaction content to the audience of the live broadcast room through the virtual image, and the target interaction data includes target audio data and target video data; The target interaction data is added to the live broadcast data of the live broadcast room.

2. The method according to claim 1, characterized in that Determining the target question to be answered according to the live broadcast barrage data includes: Determine at least one question to be answered according to the live broadcast barrage data; The target question to be answered is determined from the at least one question to be answered according to the host voice data, and the host voice data does not include audio data corresponding to the answer content of the target question to be answered.

3. The method according to claim 1 or 2, characterized in that: Before adding the target interactive data to the live broadcast data of the target live broadcast, the method further includes: It is determined that a pause occurs in the host voice in the host voice data.

4. The method according to any one of claims 1 to 3, characterized in that: The virtual image control device includes a virtual image knowledge base, and the step of determining a target question to be answered based on the live broadcast barrage data includes: Clustering the live broadcast barrage data to determine a barrage intention information set; Determining the target question to be answered according to the barrage intention information set; Determining the target interactive content according to the target question to be answered includes: According to the target question to be answered, determining the answer content of the target question to be answered from the virtual image knowledge base; The target interactive content is determined according to the reply content.

5. The method according to claim 4, characterized in that The method further comprises: The virtual image knowledge base is updated according to the target question to be answered and the audio data used to answer the target question to be answered.

6. The method according to any one of claims 1 to 5, characterized in that: Determining target interaction data according to the target interaction content includes: Generate the corresponding target audio data according to the target interactive content; The target video data is generated according to the virtual image and the target audio data, and the target video data is the video data when the virtual image emits the audio corresponding to the target interactive content.

7. The method according to claim 6, characterized in that Generating the corresponding target audio data according to the target interactive content includes: Analyze the host's voice data to determine the host's emotion category; The target audio data matching the host's emotion category is generated according to the target interactive content.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Determine anchor instruction information according to the anchor voice data; Determining instruction interaction data according to the anchor instruction information, wherein the instruction interaction data is used to show the audience of the live broadcast room through the virtual image that the virtual image completes the action indicated by the anchor instruction information; The instruction interaction data is added to the live broadcast data of the live broadcast room.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Determine target subtitle data according to the host voice data; The target subtitle data is added to the live broadcast data of the live broadcast room.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: Analyze the anchor's voice data to determine the live broadcast room operation instructions; Adjust the live broadcast data of the live broadcast room according to the live broadcast room operation instruction.

11. A virtual image control device, characterized in that: The virtual image control device is located in at least one data center, and the device includes: An acquisition module, used to acquire live broadcast interaction data of a live broadcast room, wherein the live broadcast interaction data includes live broadcast barrage data and the host voice data, and the live broadcast room includes the virtual image and the host; A question determination module, used for determining a target question to be answered based on the live broadcast barrage data, wherein the live broadcast barrage data is sent by the audience of the live broadcast room; An interactive content determination module, configured to determine target interactive content according to the target question to be answered and the live interactive data, wherein the target interactive content includes an answer to the target question to be answered; An interactive data determination module, used to determine target interactive data according to the target interactive content, wherein the target interactive data is used to display the target interactive content to the audience of the live broadcast room through the virtual image, and the target interactive data includes target audio data and target video data; The live broadcast data adjustment module is used to add the target interactive data to the live broadcast data of the live broadcast room.

12. The device according to claim 11, characterized in that The question determination module is specifically used to determine at least one question to be answered based on the live broadcast barrage data; and to determine the target question to be answered from the at least one question to be answered based on the host voice data, wherein the host voice data does not include audio data corresponding to the answer content of the target question to be answered.

13. The device according to claim 11 or 12, characterized in that The problem determination module is also used to determine whether a pause occurs in the host's voice data.

14. The device according to any one of claims 11 to 13, characterized in that The virtual image control device includes a virtual image knowledge base, The question determination module is used to cluster the live broadcast barrage data to determine a barrage intention information set; and determine the target question to be answered based on the barrage intention information set; The interactive content determination module is used to determine the answer content of the target question to be answered from the virtual image knowledge base according to the target question to be answered; and determine the target interactive content according to the answer content.

15. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory: The memory is used to store instructions; The processor is configured to cause the computing device cluster to execute the method according to any one of claims 1 to 10 according to the instructions.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computing device, enable the computing device to perform the method according to any one of claims 1 to 10.

17. A computer program product comprising instructions which, when executed on a computing device, cause the computing device to perform the method according to any one of claims 1 to 10.

Images