Live broadcast method, apparatus and device, and storage medium

Abstract

The invention discloses a live broadcast method and device, equipment and a storage medium, and belongs to the field of human-computer interaction. The method comprises the following steps: receiving an uploading operation of a material video, wherein the material video is obtained by pre-recording; receiving a configuration operation of the live broadcast commodity, wherein the configuration operation is used for configuring commodity information of the live broadcast commodity; and playing a live broadcast video of the first live broadcast room, the live broadcast video comprising a video picture of the material video and an explanation audio, the explanation audio being a human voice audio synthesized based on an explanation text, and the explanation text being generated according to the commodity information and / or the material video. The method can improve the digital human live broadcast efficiency.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction, and in particular to a live broadcast method, device, equipment and storage medium. Background Art

[0002] Digital human live broadcast refers to a form of live broadcast that uses virtual characters (i.e. digital humans) generated based on technologies such as artificial intelligence and computer graphics as hosts, and projects digital humans into virtual or real scenes through real-time rendering and animation technology for real-time performances and interactions.

[0003] In related technologies, digital human live broadcast requires the creation of a detailed digital human model first, and then, based on the characteristics of the products to be promoted in the live broadcast, natural language processing (NLP) and intelligent content generation technology are used to automatically generate spoken text, which contains a detailed introduction to the products. Subsequently, the digital human model is controlled to perform corresponding actions and expressions according to the spoken text, and the digital model is rendered in real time to obtain the video stream of the digital human live broadcast.

[0004] The methods in the related art require the use of complex algorithms to control the digital human to perform actions and render the video stream, which has high requirements on the performance of the live broadcast equipment and low efficiency in generating live broadcast videos. Summary of the invention

[0005] The present application provides a live broadcast method, device, equipment and storage medium, which can improve the efficiency of digital human live broadcast. The technical solution is as follows:

[0006] According to one aspect of the present application, a live broadcast method is provided, the method being executed by a host client, the method comprising:

[0007] receiving an upload operation of a material video, where the material video is pre-recorded;

[0008] Receive a configuration operation of a live broadcast commodity, where the configuration operation is used to configure commodity information of the live broadcast commodity;

[0009] The live video of the first live broadcast room is played, wherein the live video includes the video screen of the material video and the commentary audio, wherein the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated according to the product information and / or the material video.

[0010] According to another aspect of the present application, a live broadcast method is provided, the method being executed by a server, the method comprising:

[0011] Receiving a material video uploaded by a host client, where the material video is pre-recorded;

[0012] Receiving configuration information of the live broadcast product sent by the anchor client, wherein the configuration information includes product information of the live broadcast product;

[0013] A live video is sent to the first live broadcast room, wherein the live video includes a video screen of the material video and a commentary audio, wherein the commentary audio is a human voice audio synthesized based on a commentary text, and the commentary text is generated based on the product information and / or the material video.

[0014] According to another aspect of the present application, a live broadcast method is provided, the method being executed by a viewer client, the method comprising:

[0015] Display the live broadcast room selection interface;

[0016] Receiving a selection operation for a first live broadcast room in the live broadcast room selection interface;

[0017] Play the live video of the first live broadcast room; the live video includes the video screen and commentary audio of the material video; the material video is pre-recorded; the commentary audio is a human voice audio synthesized based on the commentary text of the live broadcast product; the commentary text is generated based on the product information of the live broadcast product and / or the material video.

[0018] According to another aspect of the present application, a live broadcast device is provided, the device comprising:

[0019] A first interaction module is used to receive an upload operation of a material video, where the material video is pre-recorded;

[0020] The first interaction module is used to receive a configuration operation of a live broadcast commodity, wherein the configuration operation is used to configure commodity information of the live broadcast commodity;

[0021] The first playback module is used to play the live video of the first live broadcast room, the live video includes the video screen of the material video and the commentary audio, the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated according to the product information and / or the material video.

[0022] According to another aspect of the present application, a live broadcast device is provided, the device comprising:

[0023] A receiving module, used to receive a material video uploaded by a host client, where the material video is pre-recorded;

[0024] The receiving module is used to receive the configuration information of the live broadcast commodity sent by the anchor client, wherein the configuration information includes the commodity information of the live broadcast commodity;

[0025] A sending module is used to send a live video of the first live broadcast room, wherein the live video includes a video screen and a commentary audio of the material video, wherein the commentary audio is a human voice audio synthesized based on a commentary text, and the commentary text is generated based on the product information and / or the material video.

[0026] According to another aspect of the present application, a live broadcast device is provided, the device comprising:

[0027] The second display module is used to display the live broadcast room selection interface;

[0028] A second interaction module is used to receive a selection operation for the first live broadcast room in the live broadcast room selection interface;

[0029] The second playback module is used to play the live video of the first live broadcast room; the live video includes the video screen and commentary audio of the material video; the material video is pre-recorded; the commentary audio is a human voice audio synthesized based on the commentary text of the live broadcast product; the commentary text is generated based on the product information of the live broadcast product and / or the material video.

[0030] According to another aspect of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the live broadcast method as described above.

[0031] According to another aspect of the present application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the computer-readable storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the live broadcast method as described above.

[0032] According to another aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the live broadcast method provided in various optional implementations of the above aspects.

[0033] The beneficial effects of the technical solution provided by this application include at least:

[0034] Use pre-recorded material video as the video screen of the live video; generate commentary text based on the product information of the live product, and synthesize human voice audio based on the commentary text to obtain the commentary audio of the live video. Based on the live video synthesized by commentary audio and material video, digital human live broadcast can be simply realized without complex model control and rendering, simplifying the implementation method of digital human live broadcast and improving the generation efficiency of digital human live broadcast. This method can automatically generate live video based on the material video and product information uploaded by the merchant, without the need for a real anchor, saving the manpower cost of live broadcast, and this method does not require digital human model rendering, which greatly reduces the real-time computing amount of digital human live broadcast, reduces the technical cost and hardware requirements of digital human live broadcast, and facilitates the promotion and application of digital human live broadcast. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 is a schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present application;

[0037] Figure 2 is a flow chart of a live broadcast method provided by an exemplary embodiment of the present application;

[0038] Figure 3 is a schematic diagram of a live broadcast method provided by an exemplary embodiment of the present application;

[0039] Figure 4 is a schematic diagram of a live broadcast method provided by an exemplary embodiment of the present application;

[0040] Figure 5 is a flow chart of a live broadcast method provided by an exemplary embodiment of the present application;

[0041] Figure 6 is a flow chart of a live broadcast method provided by an exemplary embodiment of the present application;

[0042] Figure 7 is a flow chart of a live broadcast method provided by an exemplary embodiment of the present application;

[0043] Figure 8 is a schematic diagram of a live broadcast method provided by an exemplary embodiment of the present application;

[0044] Fig. 9 is a flow chart of a live broadcast method provided by an exemplary embodiment of the present application;

[0045] Fig.10 is a structural schematic diagram of a live broadcast device provided by an exemplary embodiment of the present application;

[0046] Fig.11 is a structural schematic diagram of a live broadcast device provided by an exemplary embodiment of the present application;

[0047] Fig.12 is a structural schematic diagram of a live broadcast device provided by an exemplary embodiment of the present application;

[0048] Fig.13 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application.

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0051] Figure 1 A schematic diagram of a computer system provided by an exemplary embodiment of the present application is shown, and the computer system may include a terminal device 101 and a server 103 .

[0052] Exemplarily, the live broadcast method shown in the embodiment of the present application can be applied in a terminal device or a server, and the terminal device 101 is installed with a live broadcast client. The terminal device may include a mobile phone, a tablet computer, a notebook computer, a laptop computer, a desktop computer, an all-in-one computer, an Internet of Things device, an intelligent robot workstation, a television, a set-top box, smart glasses, a smart watch, a digital camera, an MP4 player, an MP5 player, a learning machine, a point-reading machine, an electronic paper book, an electronic dictionary, a vehicle-mounted device, a virtual reality (VR) player or an augmented reality (AR) player, etc.

[0053] Exemplarily, the live broadcast method provided by the present application can be executed by a client (anchor client or audience client) in a terminal device. The client can be a client of an application 102 that supports the live broadcast function. When the live broadcast method provided by an embodiment of the present application needs to be used, the anchor client receives the configuration operation of the material video and live broadcast product uploaded by the anchor, and reports the material video and the product information of the live broadcast product to the server. The server automatically generates a live broadcast video based on the product information of the material video and the live broadcast product, and sends it to the anchor client and the audience client for playback.

[0054] The terminal device 101 includes a first memory and a first processor. A live broadcast program is stored in the first memory; the live broadcast program is called and executed by the first processor to implement the live broadcast method provided in the present application. The first memory may include but is not limited to the following: random access memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, EPROM), and electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM).

[0055] The first processor may be composed of one or more integrated circuit chips. Optionally, the first processor may be a general-purpose processor, such as a central processing unit (CPU) or a network processor (NP). Optionally, the first processor may implement the live broadcast method provided by the present application by running a program or code.

[0056] In an optional embodiment, the terminal device 101 and the server 103 may be connected to each other via a wired or wireless network.

[0057] The server 103 is used to provide background services for the live broadcast client of the terminal device 101. Optionally, the server 103 undertakes the main computing work, and the terminal device 101 undertakes the secondary computing work; or, the server 103 undertakes the secondary computing work, and the terminal device 101 undertakes the main computing work; or, the server 103 and the terminal device 101 adopt a distributed computing architecture for collaborative computing.

[0058] The server 103 may be a single server, or a server cluster consisting of multiple servers, or a cloud computing service center.

[0059] Optionally, the server 103 includes a second memory and a second processor. The second memory stores a live broadcast program; the live broadcast program is called by the second processor to implement the live broadcast method provided in the present application. Optionally, the second memory may include but is not limited to the following: RAM, ROM, PROM, EPROM, EEPROM. Optionally, the second processor may be a general-purpose processor, such as a CPU or NP.

[0060] Figure 2is a flowchart of a live broadcast method provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The terminal device or the anchor client in the terminal device shown. The method includes the following steps.

[0061] Step 210: receiving an upload operation of a material video, where the material video is pre-recorded.

[0062] The method can be executed by an anchor terminal, on which an anchor client of a live broadcast application is installed, and an anchor account is logged in to the anchor client. The anchor account has management authority for the first live broadcast room.

[0063] Exemplarily, the anchor client can display the digital human live broadcast management interface of the first live broadcast room, in which a video upload control is displayed, and in response to receiving an operation that triggers the video upload control, receives the uploaded material video.

[0064] The host can upload material videos during the live broadcast of the first live broadcast room, and update the live broadcast video of the first live broadcast room to the live broadcast video generated based on the material video in real time; the host can also upload the material video in advance before the first live broadcast room starts broadcasting, and after the first live broadcast room starts broadcasting, the live broadcast application can automatically play the live broadcast room video generated based on the material video in the first live broadcast room.

[0065] The material video is a pre-recorded video. The anchor can pre-record the material video based on the live broadcast theme of the first live broadcast room and upload it. The video content of the material video is related to the live broadcast theme of the first live broadcast room, or the video content of the material video is related to the live broadcast products sold in the first live broadcast room, or the video content of the material video is related to the promotional content of the first live broadcast room.

[0066] For example, when the live broadcast product in the first live broadcast room is food, the material video can show the production process of the food. When the first live broadcast room is used to select a marathon event, the material video can be a marathon competition video, or the material video can be a marathon action explanation video. When the first live broadcast room is used to sell coupons for the first store, the material video can be an in-store display video of the first store.

[0067] In an optional embodiment, when the anchor uploads multiple videos, the anchor client can splice the multiple videos to obtain the material video. The splicing order of the multiple videos can be specified by the anchor; it can also be determined by the anchor client based on the live broadcast products corresponding to each video and the explanation order of the live broadcast products.

[0068] Step 220: Receive a configuration operation of the live broadcast product, where the configuration operation is used to configure product information of the live broadcast product.

[0069] Exemplarily, the anchor client can display the digital human live broadcast management interface of the first live broadcast room, in which a product configuration control is displayed, and in response to receiving an operation that triggers the product configuration control, the configured live broadcast product information is received. Exemplarily, the product information may include at least one of the following: live broadcast product name, live broadcast product price, live broadcast product link, live broadcast product explanation order, live broadcast product text introduction, live broadcast product picture, live broadcast product video, live broadcast product remaining quantity, live broadcast product promotion activities.

[0070] Live broadcast products are products sold in the first live broadcast room. The first live broadcast room provides a purchase link for live broadcast products. Users can trigger the purchase link in the first live broadcast room to jump to the purchase page of the live broadcast product to place an order to purchase the product. Alternatively, live broadcast products are products promoted by the first live broadcast room. The first live broadcast room is used to introduce and promote live broadcast products and guide users to purchase live broadcast products through designated channels.

[0071] Optionally, the new order of step 220 and step 210 can be arbitrary. The anchor can upload the material video first and then configure the live broadcast product, or can configure the live broadcast product first and then upload the material video.

[0072] In an optional embodiment, when the host first executes step 220 to configure the live broadcast product, when executing step 210, the host client can display recommended videos, which include videos in other live broadcast rooms corresponding to the live broadcast product. The host selects the recommended video as the material video.

[0073] It should be noted that the number of live broadcast products is at least one. When the number of live broadcast products is at least two, the anchor can specify the order of explanation of at least two live broadcast products; or, if the anchor does not specify the order of explanation of at least two live broadcast products, the anchor client can determine the order of explanation of at least two live broadcast products according to preset rules.

[0074] Step 230: Play the live video of the first live broadcast room, the live video includes the video screen of the material video and the commentary audio, the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated according to the product information and / or the material video.

[0075] Live video is a video synthesized from the source video and the commentary audio. For example, the audio of the live video is the commentary audio, and the screen of the live video is the screen of the source video. Alternatively, the audio of the live video is the audio obtained by merging the commentary audio and the audio of the source video. The screen of the live video includes the screen of the source video. Among them, the live video can be a video synthesized by optimizing the source video and the commentary audio. Alternatively, the live video can be obtained by optimizing the video synthesized from the source video and the received audio.

[0076] Exemplarily, the anchor client uploads the material video and product information to the server, and the server synthesizes the live video based on the material video and product information. Alternatively, the anchor client synthesizes the live video based on the material video and product information and sends the synthesized live video to the server.

[0077] For example, the first live broadcast room is a food live broadcast room, and the material videos are demonstration videos of various foods. Figure 3 As shown, the live broadcast interface of the first live broadcast room is displayed, and a video screen 201 of a material video is displayed in the live broadcast interface, and the video screen includes various food products sold in the first live broadcast room. Subtitles 202 of audio commentary can also be displayed in the live broadcast video. When introducing hamburgers, the audio commentary can be "We select the best ingredients and cook them with care, so every bite can taste fresh and delicious", with corresponding subtitles 202.

[0078] The commentary audio can be automatically generated by the digital human driven by artificial intelligence according to the product information. That is, this method only needs to call some functions in the digital human system for generating the digital human voice audio, without the need to build a digital human model and render a video. Since the technical cost of building a digital human model and rendering a video is relatively high, abandoning this part of the demand can greatly reduce the technical cost of digital human live broadcast and improve the efficiency of digital human live broadcast.

[0079] For example, after the first live broadcast room starts broadcasting, the server sends live video to the anchor client and the audience client. During the live broadcast, the server can also generate reply audio based on the audience's comments in the first live broadcast room, and use the reply audio to update the live video in real time, so that the reply audio to the audience's comments is played in the live video, and the commentary audio continues to be played after the playback ends. This part of the function can be realized by calling the digital human driven by artificial intelligence. During the live broadcast, the audience's comments are sent to the digital human system in real time, so that the digital human system generates reply text and reply audio, and realizes the real-time interaction between the digital human and the audience during the live broadcast.

[0080] Exemplarily, the server generates prompt words based on the comments of viewers in the first live broadcast room, the product information of the live broadcast products in the first live broadcast room, the order of product explanations, the product links of each live broadcast product, etc., and inputs the prompt words into the large language model to obtain reply text to the comments.

[0081] For example, Figure 4As shown, when the viewer account posts a comment 203 "I want to buy a hamburger!" in the comment area of ​​the first live broadcast room, the server calls the digital human driven by artificial intelligence to generate a reply text "@ the audience can take a look at link No. 1" to the comment. And generate a reply audio for the reply text. In the comment area, the anchor account is controlled to send a reply text 204 "@ the audience can take a look at link No. 1", and the reply audio is played at the same time, and the subtitles of the reply audio 205 can be synchronously displayed in the live video. Exemplarily, the first live broadcast room can play the live video in a loop during the live broadcast, and use the digital human technology to achieve real-time interaction with the audience, and realize the digital human live broadcast with a lower technical cost.

[0082] The method provided in the embodiment of the present application can achieve an overall live broadcast content quality close to that of a digital human at an extremely low cost by synthesizing pre-recorded material videos and a digital human voice-over explaining a product, coupled with AI interactive responses, and can even exceed the content quality of a digital human live broadcast in some scenarios.

[0083] In the related art, the digital human live broadcast method uses AI to automatically generate oral scripts and oral voices based on the live broadcast products, drive the digital human's lips and body expression to be consistent with the oral content, and support the AI ​​image to interact with the user's public screen reply. However, the cost of digital human live broadcast is still relatively high. The current market price is about 800 yuan / month / channel, and the price is still relatively high for long-tail merchants. The core cost lies in the reasoning cost of driving the synthesis of digital human images. The method provided in the embodiment of the present application can eliminate the reasoning link, and the pre-recorded material video and the digital human voice can be synthesized into a live video, which will save more than 90% of the cost.

[0084] Figure 5 is a flowchart of a live broadcast method provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The method comprises the following steps.

[0085] Step 310: receiving the material video uploaded by the anchor client, where the material video is pre-recorded.

[0086] The method can be executed by a server, on which a server end of a live broadcast application is installed, and the server is used to provide background services for the anchor client and the audience client.

[0087] Step 320: receiving configuration information of the live broadcast product sent by the anchor client, where the configuration information includes product information of the live broadcast product.

[0088] Step 330: Send the live video of the first live broadcast room, the live video includes the video screen of the material video and the commentary audio, the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated according to the product information and / or the material video.

[0089] The server can generate a live video based on the material video and product information uploaded by the anchor client, and send the live video to the anchor client of the first live broadcast room and the viewer client entering the first live broadcast room, so that the live video of the first live broadcast room can be played on the anchor client and the viewer client.

[0090] Figure 6 is a flowchart of a live broadcast method provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The terminal device or the viewer client in the terminal device shown. The method comprises the following steps.

[0091] Step 410: Display the live broadcast room selection interface.

[0092] The method can be performed by a viewer terminal, on which a viewer client of a live broadcast application is installed, and a viewer account is logged in to the viewer client. The viewer account can enter the first live broadcast room as a viewer to watch the live broadcast.

[0093] For example, the live broadcast room selection interface may be a live broadcast room lobby interface of a live broadcast application. The audience client may display the live broadcast room lobby interface of the live broadcast application, in which at least two live broadcast room entry controls are displayed, and the user may trigger the entry control to enter the corresponding live broadcast room.

[0094] Step 420: Receive a selection operation for the first live broadcast room in the live broadcast room selection interface.

[0095] For example, in response to triggering the entry control corresponding to the first live broadcast room, the live broadcast interface of the first live broadcast room is displayed, and the live broadcast interface is used to play the live video of the first live broadcast room.

[0096] Step 430: Play the live video of the first live broadcast room; the live video includes the video screen and commentary audio of the material video; the material video is pre-recorded; the commentary audio is a human voice audio synthesized based on the commentary text of the live broadcast product; the commentary text is generated based on the product information of the live broadcast product and / or the material video.

[0097] In summary, the method provided in this embodiment uses a pre-recorded material video as the video screen of the live video; based on the product information of the live broadcast product, an explanation text is generated, and the human voice audio is synthesized according to the explanation text to obtain the explanation audio of the live video. Based on the live video synthesized by the explanation audio and the material video, the digital human live broadcast can be simply realized without complex model control and rendering, which simplifies the implementation method of the digital human live broadcast and improves the generation efficiency of the digital human live broadcast. The method can automatically generate a live video based on the material video and product information uploaded by the merchant, without the need for a real anchor, saving the manpower cost of the live broadcast, and the method does not require the rendering of the digital human model, which greatly reduces the real-time computing amount of the digital human live broadcast, reduces the technical cost and hardware requirements of the digital human live broadcast, and facilitates the promotion and application of the digital human live broadcast.

[0098] An exemplary embodiment of interaction between a host client, a server, and a viewer client is provided.

[0099] Figure 7 is a flowchart of a live broadcast method provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The terminal and server shown in the figure. The method comprises the following steps.

[0100] Step 501: The anchor client sends a material video and product information of the live product to the server. The server generates a live video based on the material video and product information.

[0101] In an optional embodiment, the first live broadcast room may be a live broadcast room of a first online store, and at least one product is available in the first online store. The host may select a live broadcast product from the at least one product available in the first online store in the live broadcast management interface. The host client or server may obtain detailed information of the product selected by the host. After the live broadcast product selection is completed, the host may also adjust the explanation order of at least two live broadcast products so that the digital human explains the at least two live broadcast products in the explanation order.

[0102] Alternatively, the anchor can list the product link of the live broadcast product in the live broadcast management interface and edit the detailed information of the live broadcast product in the product link.

[0103] The product information of the live broadcast product includes the detailed information of the live broadcast product and the explanation order. The server generates the explanation text of the live broadcast product based on the product information and / or the material video, generates the explanation audio based on the explanation text, and synthesizes the live broadcast video according to the explanation audio and the material video.

[0104] For example, the live video can be generated locally by the anchor client or by the server. Figure 7The embodiment shown is only described by taking the live video generated by the server as an example. When the host video is generated locally by the host client, the host client can directly report the generated live video to the server.

[0105] Exemplarily, the product information includes a preset explanation order for at least two live products; when the live video is generated locally by the anchor client, the anchor client generates an explanation text based on the product information, and the explanation text is used to explain at least two live products in sequence according to the preset explanation order; synthesizes commentary audio based on the explanation text; synthesizes the live video based on the commentary audio and the material video; and plays the live video.

[0106] When the live video is generated by the server, the server generates an explanatory text based on the product information, and the explanatory text is used to explain at least two live products in sequence according to a preset explanation order; synthesizes the explanatory audio based on the explanatory text; and synthesizes the live video based on the explanatory audio and the material video.

[0107] When the live video is generated locally by the anchor client, the AI-driven digital human can be deployed on the anchor terminal, and the anchor client can call the locally deployed AI-driven digital human to generate commentary text and commentary audio. Alternatively, the AI-driven digital human can be deployed on the digital human server, and the anchor client can remotely call the digital human to generate commentary text and commentary audio; the anchor client sends a call request for the AI-driven digital human to the server, and the call request carries the material video and the product information of the live product, and the call request is used to request the call of the AI-driven digital human to generate commentary text and commentary audio based on the material video and product information. For example, the anchor client can call the AI-driven digital human to generate product commentary text for each product based on the product information of at least two live products, and then coherently splice at least two product commentary texts in the order of the commentary of at least two products to obtain the final commentary text.

[0108] When the live video is generated by the server, the AI-driven digital human can be deployed on the server. For example, the server can call the AI-driven digital human to generate a product description text for each product based on the product information of at least two live products reported by the anchor client, and then coherently splice the at least two product description texts according to the description order of at least two products to obtain the final description text.

[0109] AI-driven digital humans can understand product information and generate product description texts, which are used to introduce live broadcast products to viewers and guide them to purchase live broadcast products. For example, product description texts can include product names, product promotion activities, product purchase link numbers, product application scenarios, product applicable person recommendations, and other content.

[0110] After obtaining the commentary text, the artificial intelligence-driven digital human can generate commentary audio for the commentary text based on the target human voice and tone.

[0111] Optionally, the host can also configure the voice tone used in the first live broadcast room. The host client receives the selection operation of the target voice tone; the host client generates the commentary audio of the commentary text according to the target voice tone. Alternatively, the server receives the selection instruction of the target voice tone sent by the host client; and generates the commentary audio of the commentary text according to the target voice tone.

[0112] In an optional embodiment, the host client can display at least one candidate voice tone, and the host can select one of the candidate voice tones as the target voice tone. Alternatively, the host client can receive an upload operation of a sample voice audio, and call the artificial intelligence-driven digital human to generate a commentary audio according to the voice tone characteristics in the sample voice audio. Then the target voice tone can refer to the voice tone in the sample voice audio.

[0113] Optionally, when the current commentary product is a first live product, the anchor client or server may call an artificial intelligence-driven digital human to generate a commentary text for the first live product based on the product information of the first live product and the audience's comment information, and synthesize a commentary audio based on the commentary text; wherein the commentary text for the first live product is generated based on the commentary requirements for the first live product in the audience's comment information.

[0114] That is, during the live broadcast of the first live broadcast room, the host client or server can obtain the audience comment information in the first live broadcast room in real time. The audience comment information may include the need for explanation of some products in the live broadcast product list. For example, some viewers will ask the host to explain the promotion of the first live broadcast product, or ask the host to explain the maximum size of the second product, etc. During the live broadcast, the host client or server can also call the artificial intelligence-driven digital human to generate real-time explanation text and real-time explanation audio according to the explanation needs expressed in the audience comment information in real time during the live broadcast process, and use the real-time explanation audio to replace the original explanation audio, and update to obtain a new live broadcast video. In this way, the explanation text in the live broadcast room can be close to the real-time needs of the audience, and the interaction between the audience and the digital human can be enhanced.

[0115] Step 502: The server sends the live video to the host client and the viewer client.

[0116] Exemplarily, the server may also adjust the top link of the first live broadcast room in real time according to the current interpretation product of the first live broadcast room. For example, when the current interpretation product is the third live broadcast product, the server sends a top link indication information to the viewer client and / or the anchor client, and the top link indication information is used to replace the top link of the first live broadcast room with the third live broadcast product.

[0117] When the currently explained product is the third live broadcast product, the viewer client and / or the host client displays a pinned link of the third live broadcast product in the first live broadcast room.

[0118] For example, Figure 8 As shown, when the currently explained product in the first live broadcast room is a hamburger 206, the server sends a pinned link indication message to the viewer client and / or the host client. After receiving the pinned link indication message, the viewer client and / or the host client pins the purchase link 207 of the hamburger to facilitate the viewer to quickly purchase the currently explained product.

[0119] Step 503: The server adjusts the order of explaining the live product according to the real-time video screen of the material video and updates the live video.

[0120] Optionally, the anchor client or server may also adjust the order of product explanations in real time according to the live products displayed in the real-time video screen of the live video.

[0121] For example, at least two live broadcast products include a second live broadcast product; the anchor client or server can play the real-time commentary audio of the second live broadcast product when the second live broadcast product exists in the real-time video screen of the live broadcast video, and the real-time commentary audio is a human voice audio synthesized based on the real-time commentary text, and the real-time commentary audio is generated according to the product information of the second live broadcast product and the real-time video screen; if the live broadcast product is not recognized in the real-time video screen, the commentary audio continues to be played.

[0122] For example, the anchor client or server can call the AI-driven digital human to generate real-time commentary text based on the real-time video screen and the product information of the second live broadcast product, and call the AI-driven digital human to generate real-time commentary audio corresponding to the real-time commentary text. The real-time commentary text includes text content that explains the second live broadcast product based on the screen content of the real-time video screen.

[0123] Step 504: The viewer client sends the comment information to the server. The server generates a reply text and a reply audio based on the comment information.

[0124] Exemplarily, the anchor client or server can also generate reply information to the comment information in real time based on the audience's comment information, and generate reply audio of the reply information, and respond to the audience's questions in real time in the form of text and audio.

[0125] The audience client receives the operation of sending comment information in the first live broadcast room; the reply information of the anchor account is displayed in the first live broadcast room, and the reply information is generated by the digital human driven by artificial intelligence according to the comment information and product information; the reply audio is played, and the reply audio is the human voice audio synthesized by the digital human driven by artificial intelligence according to the reply information.

[0126] For example, Figure 4 As shown, when the viewer account posts a comment 203 "I want to buy a hamburger!" in the comment area of ​​the first live broadcast room, the server calls the digital human driven by artificial intelligence to generate a reply text "@ The viewer can take a look at link No. 1" to the comment. And generates a reply audio of the reply text. In the comment area, the anchor account is controlled to send a reply text 204 "@ The viewer can take a look at link No. 1", and the reply audio is played at the same time, and the subtitles of the reply audio can be synchronously displayed in the live video 205.

[0127] The server receives comment information sent by the audience client in the first live broadcast room; the artificial intelligence-driven digital human generates reply information based on the comment information and product information; and the artificial intelligence-driven digital human synthesizes reply audio based on the reply information; and sends the reply information and reply audio to the anchor client and / or the audience client.

[0128] The anchor client displays the comment information sent by the audience account in the first live broadcast room; displays the reply information sent by the anchor account, which is generated by calling the artificial intelligence-driven digital human based on the comment information and product information; and plays the reply audio, which is the human voice audio synthesized by the artificial intelligence-driven digital human based on the reply information.

[0129] Step 505: The server sends reply text and reply audio to the host client and the viewer client.

[0130] Exemplarily, step 503 may be implemented as an embodiment alone, and step 504 and step 505 may also be implemented as an embodiment alone.

[0131] In summary, the method provided in this embodiment calls the digital human driven by artificial intelligence to generate the commentary text and audio of the live broadcast of the product; the live broadcast video is synthesized based on the audio commentary and the video material uploaded by the anchor. This method only requires lightweight calling of the content generation function of the digital human, without the need for digital human model rendering, which reduces the technical cost of the digital human live broadcast and improves the efficiency of the digital human live broadcast.

[0132] The method provided in this embodiment can adjust the order of product explanation according to the live products in the real-time video screen of the live video, so that the currently explained product fits the content displayed in the live video, ensures that the sound and picture match, and makes the audience's experience of watching the digital human live broadcast smoother.

[0133] The method provided in this embodiment can call on the digital human driven by artificial intelligence to generate reply text and reply audio to the audience's comments in real time, and use the reply audio to update the live video in real time, thereby realizing real-time interaction between the digital human and the audience.

[0134] Exemplarily, the server can be divided into a live broadcast service, a content generation service and a digital human background, and an embodiment of the live broadcast method provided in the embodiment of the present application is provided in which the three services of a server are jointly implemented.

[0135] Fig. 9 is a flowchart of a live broadcast method provided by an exemplary embodiment of the present application. The method can be used for Figure 1 The method comprises the following steps.

[0136] Step 601, the live broadcast service responds to the live broadcast creation operation of the anchor client, creates a live broadcast preview, and configures the live broadcast products and the start and end time of the live broadcast.

[0137] Step 602, the digital human backstage configures the live broadcast content, which includes the live broadcast products and the explanation order, the digital human voice color, the live broadcast room decoration, and the upload of pre-recorded material videos.

[0138] Step 603: The digital human backend submits a content generation request to the content generation service based on the live broadcast content. The content generation service generates the script, material, AI reply and commentary audio, and synthesizes the recorded material video and commentary audio into a live broadcast video. If the material video is relatively short, the material video will be played in a loop to obtain a live broadcast video with a longer duration.

[0139] Step 604: After the live broadcast starts, the digital human backend starts to push the stream to the live broadcast service and transmit the live broadcast data, which includes the live broadcast video. The live broadcast service receives the live broadcast data and sends it to the host client and the audience client.

[0140] Step 605: The content generation service returns the progress of the current live broadcast product according to the progress of the commentary in the live broadcast room, and the digital human backend ensures that the progress of the live broadcast product is consistent with the top product in the live broadcast room.

[0141] Step 606: The Digital Human backend receives the audience's public screen comments sent by the live broadcast service, and uses the reply text of the public screen comments as a question. The content generation service generates a reply text. On the one hand, the Digital Human backend writes to the live broadcast room interface for text reply. On the other hand, the content generation service generates a reply audio of the reply text for audio reply.

[0142] Step 607: When the live broadcast ends, the digital human background and live broadcast service stop streaming.

[0143] In summary, compared with the digital human live broadcast method in the related art, the method provided by the embodiment of the present application can eliminate the reasoning link, and synthesize the pre-recorded video and the digital human voice broadcast audio into a live video, which will save more than 90% of the cost. For some merchants, displaying recorded videos such as real store environments and food processing processes, and displaying digital human models are more conducive to attracting users to buy. The method can intelligently identify and understand the video content, and generate oral text for the corresponding content. For example, if the video content is the process of frying meatballs, the oral product can be adjusted to the corresponding product, and accompanied by the corresponding explanation text "The fried meatballs you see are link number x", etc., and the explanation audio and product explanation order are adjusted in real time according to the video screen, so that the digital human can explain the products displayed in the current live video in real time.

[0144] It should be noted that before collecting relevant user data (such as operation data and game data in this application) and during the process of collecting relevant user data, this application can display a prompt interface, pop-up window or output voice prompt information. The prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining user-related data after obtaining the user's confirmation operation on the prompt interface or pop-up window, otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining user-related data are terminated, that is, the user's relevant data is not obtained. In other words, all user data collected by this application are collected with the user's consent and authorization, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0145] It should be noted that the sequence of the method steps provided in the embodiments of the present application can be appropriately adjusted, and the steps can be increased or decreased accordingly according to the circumstances. Any technician familiar with the technical field can easily think of the changed methods within the technical scope disclosed in the present application, and they should be included in the protection scope of the present application, so they will not be repeated here.

[0146] Fig.10 1 is a schematic diagram of the structure of a live broadcast device provided by an exemplary embodiment of the present application. The device is used to implement a terminal with a host client installed, and the device includes:

[0147] The first interaction module 801 is used to receive an upload operation of a material video, where the material video is pre-recorded;

[0148] The first interaction module 801 is used to receive a configuration operation of a live broadcast commodity, where the configuration operation is used to configure commodity information of the live broadcast commodity;

[0149] The first playback module 802 is used to play the live video of the first live broadcast room, wherein the live video includes the video screen of the material video and the commentary audio, wherein the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated based on the product information and / or the material video.

[0150] In an optional embodiment, the product information includes at least two preset explanation orders of the live broadcast products; and the device further includes:

[0151] A first generating module 803 is used to generate the explanation text according to the product information, and the explanation text is used to explain the at least two live broadcast products in sequence according to the preset explanation order;

[0152] The first generating module 803 is used to synthesize the commentary audio according to the commentary text;

[0153] The first generating module 803 is used to synthesize the live video according to the commentary audio and the material video;

[0154] The first playing module 802 is used to play the live video.

[0155] In an optional embodiment, the first generating module 803 is used to call the artificial intelligence-driven digital human to generate the explanation text of the first live broadcast product according to the product information of the first live broadcast product and the audience comment information, and synthesize the explanation audio according to the explanation text when the current explanation product is the first live broadcast product.

[0156] Among them, the commentary text of the first live broadcast product is generated according to the commentary requirements for the first live broadcast product in the audience comment information.

[0157] In an optional embodiment, the at least two live broadcast commodities include a second live broadcast commodity;

[0158] The first playing module 802 is used to play the real-time commentary audio of the second live broadcast product when there is a second live broadcast product in the real-time video screen of the live broadcast video, wherein the real-time commentary audio is a human voice audio synthesized based on the real-time commentary text, and the real-time commentary audio is generated according to the product information of the second live broadcast product and the real-time video screen;

[0159] The first playing module 802 is used to continue playing the commentary audio when no live broadcast product is recognized in the real-time video image.

[0160] In an optional embodiment, the device further includes:

[0161] The first display module 804 is used to display the comment information sent by the viewer account in the first live broadcast room;

[0162] The first display module 804 is used to display the reply information sent by the anchor account, where the reply information is generated by calling the artificial intelligence-driven digital human according to the comment information and the product information;

[0163] The first playback module 802 is used to play the reply audio, which is a human voice audio synthesized by a digital human driven by artificial intelligence based on the reply information.

[0164] In an optional embodiment, the live broadcast commodity includes a third live broadcast commodity; and the device further includes:

[0165] The first display module 804 is used to display a top link of the third live broadcast product in the first live broadcast room when the currently explained product is the third live broadcast product.

[0166] In an optional embodiment, the first interaction module 801 is used to receive a selection operation of a target human voice tone;

[0167] The first generating module 803 is used to generate the commentary audio of the commentary text according to the target human voice tone.

[0168] Fig.11 1 is a schematic diagram of the structure of a live broadcast device provided by an exemplary embodiment of the present application. The device is used to implement a server, and the device includes:

[0169] The receiving module 805 is used to receive the material video uploaded by the anchor client, where the material video is pre-recorded;

[0170] The receiving module 805 is used to receive the configuration information of the live broadcast product sent by the anchor client, wherein the configuration information includes the product information of the live broadcast product;

[0171] The sending module 806 is used to send the live video of the first live broadcast room, wherein the live video includes the video screen and commentary audio of the material video, wherein the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated based on the product information and / or the material video.

[0172] In an optional embodiment, the product information includes at least two preset explanation orders of the live broadcast products; and the device further includes:

[0173] The second generating module 807 is used to generate the explanation text according to the product information, and the explanation text is used to explain the at least two live broadcast products in sequence according to the preset explanation order;

[0174] The second generating module 807 is used to synthesize the commentary audio according to the commentary text;

[0175] The second generating module 807 is used to synthesize the live video according to the commentary audio and the material video.

[0176] In an optional embodiment, the second generating module 807 is used to generate the explanation text of the first live broadcast product based on the product information of the first live broadcast product and the audience comment information based on the artificial intelligence-driven digital human, and synthesize the explanation audio according to the explanation text when the current explanation product is the first live broadcast product;

[0177] Among them, the commentary text of the first live broadcast product is generated according to the commentary requirements for the first live broadcast product in the audience comment information.

[0178] In an optional embodiment, the at least two live broadcast commodities include a second live broadcast commodity; and the device further includes:

[0179] The second generating module 807 is used to generate a real-time commentary audio of the second live broadcast product when there is a second live broadcast product in the real-time video screen of the live broadcast video, wherein the real-time commentary audio is a human voice audio synthesized based on the real-time commentary text, and the real-time commentary audio is generated according to the product information of the second live broadcast product and the real-time video screen;

[0180] The sending module 806 is used to send the real-time commentary audio of the second live broadcast product;

[0181] The sending module 806 is used to send the commentary audio when the live broadcast product is not recognized in the real-time video picture.

[0182] In an optional embodiment, the device further includes:

[0183] The receiving module 805 is used to receive comment information sent by the viewer client in the first live broadcast room;

[0184] The second generation module 807 is used for generating reply information based on the comment information and the product information by the digital human driven by artificial intelligence; and synthesizing reply audio based on the reply information by the digital human driven by artificial intelligence;

[0185] The sending module 806 is used to send the reply information and the reply audio.

[0186] In an optional embodiment, the live broadcast commodity includes a third live broadcast commodity;

[0187] The sending module 806 is used to send a top link indication information when the current commentary product is the third live broadcast product, and the top link indication information is used to replace the top link of the first live broadcast room with the third live broadcast product.

[0188] In an optional embodiment, the device further includes:

[0189] The receiving module 805 is used to receive a selection instruction of a target human voice tone sent by the anchor client;

[0190] The second generating module 807 is used to generate the commentary audio of the commentary text according to the target human voice tone.

[0191] Fig.12 1 is a schematic diagram of the structure of a live broadcast device provided by an exemplary embodiment of the present application. The device is used to implement a terminal with a viewer client installed, and the device includes:

[0192] The second display module 810 is used to display a live broadcast room selection interface;

[0193] The second interaction module 808 is used to receive a selection operation for the first live broadcast room in the live broadcast room selection interface;

[0194] The second playback module 809 is used to play the live video of the first live broadcast room; the live video includes the video screen and commentary audio of the material video; the material video is pre-recorded; the commentary audio is a human voice audio synthesized based on the commentary text of the live broadcast product; the commentary text is generated based on the product information of the live broadcast product and / or the material video.

[0195] In an optional embodiment, the product information includes a preset explanation order of at least two of the live broadcast products; and the commentary text is used to explain the at least two live broadcast products in sequence according to the preset explanation order.

[0196] In an optional embodiment, when the currently explained product is a first live broadcast product, the explanation text of the first live broadcast product is generated based on the product information of the first live broadcast product and the audience comment information;

[0197] Among them, the commentary text of the first live broadcast product is generated according to the commentary requirements for the first live broadcast product in the audience comment information.

[0198] In an optional embodiment, the at least two live broadcast commodities include a second live broadcast commodity;

[0199] The second playing module 809 is used to play the real-time commentary audio of the second live broadcast product when the real-time video screen of the live broadcast video includes the second live broadcast product, wherein the real-time commentary audio is a human voice audio synthesized based on the real-time commentary text, and the real-time commentary audio is generated according to the product information of the second live broadcast product and the real-time video screen;

[0200] The second playing module 809 is used to play the commentary audio when the real-time video screen does not include the live broadcast product.

[0201] In an optional embodiment, the second interaction module 808 is used to receive an operation of sending comment information in the first live broadcast room;

[0202] The second display module 810 is used to display the reply information of the anchor account in the first live broadcast room, where the reply information is generated by calling the artificial intelligence-driven digital human according to the comment information and the product information;

[0203] The second playing module 809 is used to play the reply audio, which is a human voice audio synthesized by a digital human driven by artificial intelligence according to the reply information.

[0204] In an optional embodiment, the live broadcast commodity includes a third live broadcast commodity;

[0205] The second display module 810 is used to display a top link of the third live broadcast product in the first live broadcast room when the currently explained product is the third live broadcast product.

[0206] In an optional embodiment, the commentary audio is vocal audio generated according to a target vocal tone selected by the host client.

[0207] It should be noted that the live broadcast device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the live broadcast device provided in the above embodiment and the live broadcast method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0208] The embodiment of the present application also provides a computer device, which includes: a processor and a memory, wherein at least one instruction, at least one program, code set or instruction set is stored in the memory, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the live broadcast method provided by the above-mentioned method embodiments. The computer device can be implemented as a terminal device or a server.

[0209] For example, Fig.13 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application.

[0210] Typically, the computer device 1700 includes a processor 1701 and a memory 1702 .

[0211] The processor 1701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0212] The memory 1702 may include one or more computer-readable storage media, which may be non-transitory. The memory 1702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1702 is used to store at least one instruction, which is used to be executed by the processor 1701 to implement the live broadcast method provided in the method embodiment of the present application.

[0213] In some embodiments, the computer device 1700 may also optionally include: a peripheral device interface 1703 and at least one peripheral device. The processor 1701, the memory 1702 and the peripheral device interface 1703 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1703 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1704, a display screen 1705, a camera assembly 1706, an audio circuit 1707 and a power supply 1708.

[0214] The peripheral device interface 1703 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1701 and the memory 1702. In some embodiments, the processor 1701, the memory 1702, and the peripheral device interface 1703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1701, the memory 1702, and the peripheral device interface 1703 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0215] The radio frequency circuit 1704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1704 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1704 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1704 can communicate with other computer devices through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1704 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0216] The display screen 1705 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1705 is a touch display screen, the display screen 1705 also has the ability to collect touch signals on the surface or above the surface of the display screen 1705. The touch signal can be input to the processor 1701 as a control signal for processing. At this time, the display screen 1705 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1705 can be one, and the front panel of the computer device 1700 is set; in other embodiments, the display screen 1705 can be at least two, which are respectively set on different surfaces of the computer device 1700 or are folded; in some other embodiments, the display screen 1705 can be a flexible display screen, which is set on the curved surface or folded surface of the computer device 1700. Even, the display screen 1705 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1705 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0217] The camera assembly 1706 is used to capture images or videos. Optionally, the camera assembly 1706 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the computer device 1700, and the rear camera is arranged on the back of the computer device. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1706 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0218] The audio circuit 1707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1701 for processing, or input them into the radio frequency circuit 1704 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the computer device 1700. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1701 or the radio frequency circuit 1704 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1707 may also include a headphone jack.

[0219] The power supply 1708 is used to power various components in the computer device 1700. The power supply 1708 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1708 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0220] In some embodiments, the computer device 1700 further includes one or more sensors 1709 , including but not limited to: an acceleration sensor 1710 , a gyroscope sensor 1711 , a pressure sensor 1712 , an optical sensor 1713 , and a proximity sensor 1714 .

[0221] The acceleration sensor 1710 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the computer device 1700. For example, the acceleration sensor 1710 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1701 can control the touch display screen 1705 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1710. The acceleration sensor 1710 can also be used to collect game or user motion data.

[0222] The gyro sensor 1711 can detect the body direction and rotation angle of the computer device 1700, and the gyro sensor 1711 can cooperate with the acceleration sensor 1710 to collect the user's 3D actions on the computer device 1700. The processor 1701 can implement the following functions based on the data collected by the gyro sensor 1711: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0223] The pressure sensor 1712 can be set on the side frame of the computer device 1700 and / or the lower layer of the touch display screen 1705. When the pressure sensor 1712 is set on the side frame of the computer device 1700, it can detect the user's grip signal of the computer device 1700, and the processor 1701 performs left and right hand recognition or shortcut operations according to the grip signal collected by the pressure sensor 1712. When the pressure sensor 1712 is set on the lower layer of the touch display screen 1705, the processor 1701 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 1705. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0224] The optical sensor 1713 is used to collect the ambient light intensity. In one embodiment, the processor 1701 can control the display brightness of the touch display screen 1705 according to the ambient light intensity collected by the optical sensor 1713. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 1705 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 1705 is reduced. In another embodiment, the processor 1701 can also dynamically adjust the shooting parameters of the camera assembly 1706 according to the ambient light intensity collected by the optical sensor 1713.

[0225] The proximity sensor 1714, also called a distance sensor, is usually arranged on the front panel of the computer device 1700. The proximity sensor 1714 is used to collect the distance between the user and the front of the computer device 1700. In one embodiment, when the proximity sensor 1714 detects that the distance between the user and the front of the computer device 1700 is gradually decreasing, the processor 1701 controls the touch display screen 1705 to switch from the screen-on state to the screen-off state; when the proximity sensor 1714 detects that the distance between the user and the front of the computer device 1700 is gradually increasing, the processor 1701 controls the touch display screen 1705 to switch from the screen-off state to the screen-on state.

[0226] Those skilled in the art will understand that Fig.13 The structure shown in the figure does not constitute a limitation on the computer device 1700, and the computer device 1700 may include more or less components than shown in the figure, or combine some components, or adopt a different arrangement of components.

[0227] A computer-readable storage medium is also provided in an embodiment of the present application, in which at least one instruction, at least one program, code set or instruction set is stored. When the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor of a computer device, the live broadcast method provided by the above-mentioned method embodiments is implemented.

[0228] The present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the live broadcast method provided by each of the above method embodiments.

[0229] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned readable storage medium may be a read-only memory, a disk or an optical disk, etc.

[0230] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent switching, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A live broadcast method, characterized in that: The method is executed by a host client, and the method includes: receiving an upload operation of a material video, where the material video is pre-recorded; Receive a configuration operation of a live broadcast commodity, where the configuration operation is used to configure commodity information of the live broadcast commodity; The live video of the first live broadcast room is played, wherein the live video includes the video screen of the material video and the commentary audio, wherein the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated according to the product information and / or the material video.

2. The method according to claim 1, characterized in that The product information includes a preset explanation order of at least two of the live broadcast products; Playing the live video in the first live broadcast room includes: Generate the explanation text according to the product information, the explanation text is used to explain the at least two live broadcast products in sequence according to the preset explanation order; synthesizing the commentary audio according to the commentary text; Synthesize the live video according to the commentary audio and the material video; Play the live video.

3. The method according to claim 2, characterized in that The step of generating the explanation text according to the product information and synthesizing the explanation audio according to the explanation text includes: In the case where the currently explained product is the first live broadcast product, calling the artificial intelligence-driven digital human to generate the explanation text of the first live broadcast product according to the product information of the first live broadcast product and the audience comment information, and synthesizing the explanation audio according to the explanation text; Among them, the commentary text of the first live broadcast product is generated according to the commentary requirements for the first live broadcast product in the audience comment information.

4. The method according to claim 2, characterized in that: The at least two live broadcast commodities include a second live broadcast commodity; The playing of the live video includes: When a second live broadcast product exists in the real-time video screen of the live broadcast video, a real-time commentary audio of the second live broadcast product is played, wherein the real-time commentary audio is a human voice audio synthesized based on the real-time commentary text, and the real-time commentary audio is generated according to the product information of the second live broadcast product and the real-time video screen; When the live broadcast product is not recognized in the real-time video picture, the commentary audio continues to be played.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Displaying comment information sent by the viewer account in the first live broadcast room; Displaying the reply information sent by the anchor account, wherein the reply information is generated by calling the artificial intelligence-driven digital human based on the comment information and the product information; Play reply audio, where the reply audio is a human voice audio synthesized by a digital human driven by artificial intelligence based on the reply information.

6. The method according to any one of claims 1 to 4, characterized in that: The live broadcast commodity includes a third live broadcast commodity; and the method further includes: When the currently explained product is the third live broadcast product, a pinned link of the third live broadcast product is displayed in the first live broadcast room.

7. The method according to claim 2, characterized in that: The method further comprises: Receiving a selection operation of a target human voice tone; The step of synthesizing the commentary audio according to the commentary text comprises: The commentary audio of the commentary text is generated according to the target human voice tone.

8. A live broadcast method, characterized in that: The method is executed by a server, and comprises: Receiving a material video uploaded by a host client, where the material video is pre-recorded; Receiving configuration information of the live broadcast product sent by the anchor client, wherein the configuration information includes product information of the live broadcast product; A live video is sent to the first live broadcast room, wherein the live video includes a video screen of the material video and a commentary audio, wherein the commentary audio is a human voice audio synthesized based on a commentary text, and the commentary text is generated based on the product information and / or the material video.

9. The method according to claim 8, characterized in that The product information includes at least two preset explanation orders of the live broadcast products; the method further includes: Generate the explanation text according to the product information, the explanation text is used to explain the at least two live broadcast products in sequence according to the preset explanation order; synthesizing the commentary audio according to the commentary text; The live video is synthesized according to the commentary audio and the material video.

10. The method according to claim 9, characterized in that generating the explanation text according to the product information; The step of synthesizing the commentary audio according to the commentary text includes: In the case where the currently explained product is the first live broadcast product, the digital human driven by artificial intelligence generates the explanation text of the first live broadcast product according to the product information of the first live broadcast product and the audience comment information, and synthesizes the explanation audio according to the explanation text; Among them, the commentary text of the first live broadcast product is generated according to the commentary requirements for the first live broadcast product in the audience comment information.

11. The method according to claim 9, characterized in that The at least two live broadcast commodities include a second live broadcast commodity; and the method further includes: When the second live broadcast product exists in the real-time video screen of the live broadcast video, generating and sending real-time commentary audio of the second live broadcast product, wherein the real-time commentary audio is human voice audio synthesized based on the real-time commentary text, and the real-time commentary audio is generated according to the product information of the second live broadcast product and the real-time video screen; When the live broadcast product is not recognized in the real-time video picture, the commentary audio is sent.

12. The method according to any one of claims 8 to 11, characterized in that: The method further comprises: Receiving comment information sent by the viewer client in the first live broadcast room; The artificial intelligence-driven digital human generates reply information according to the comment information and the product information; and the artificial intelligence-driven digital human synthesizes reply audio according to the reply information; The reply message and the reply audio are sent.

13. The method according to any one of claims 8 to 11, characterized in that: The live broadcast commodity includes a third live broadcast commodity; and the method further includes: In the case where the currently explained product is the third live product, a pinned link indication information is sent, and the pinned link indication information is used to replace the pinned link of the first live room with the third live product.

14. The method according to claim 9, characterized in that The method further comprises: Receive a selection instruction of a target voice tone sent by the anchor client; The step of synthesizing the commentary audio according to the commentary text comprises: The commentary audio of the commentary text is generated according to the target human voice tone.

15. A live broadcast method, characterized in that: The method is performed by a viewer client, and includes: Display the live broadcast room selection interface; Receiving a selection operation for a first live broadcast room in the live broadcast room selection interface; Play the live video of the first live broadcast room; the live video includes the video screen and commentary audio of the material video; the material video is pre-recorded; the commentary audio is a human voice audio synthesized based on the commentary text of the live broadcast product; the commentary text is generated based on the product information of the live broadcast product and / or the material video.

16. The method according to claim 15, characterized in that The product information includes a preset explanation order of at least two of the live broadcast products; the commentary text is used to explain the at least two live broadcast products in sequence according to the preset explanation order.

17. The method according to claim 16, characterized in that In the case where the currently explained product is a first live broadcast product, the explanation text of the first live broadcast product is generated according to the product information of the first live broadcast product and the audience comment information; Among them, the commentary text of the first live broadcast product is generated according to the commentary requirements for the first live broadcast product in the audience comment information.

18. The method according to claim 16, characterized in that The at least two live broadcast commodities include a second live broadcast commodity; and the method further includes: When the real-time video screen of the live video includes a second live broadcast product, playing a real-time commentary audio of the second live broadcast product, wherein the real-time commentary audio is a human voice audio synthesized based on a real-time commentary text, and the real-time commentary audio is generated according to the product information of the second live broadcast product and the real-time video screen; When the live broadcast product is not included in the real-time video screen, the commentary audio is played.

19. The method according to any one of claims 15 to 18, characterized in that: The method further comprises: Receiving an operation of sending comment information in the first live broadcast room; Displaying reply information of the anchor account in the first live broadcast room, where the reply information is generated by a digital person driven by artificial intelligence based on the comment information and the product information; Play the reply audio, which is a human voice audio synthesized by a digital human driven by artificial intelligence based on the reply information.

20. The method according to any one of claims 15 to 18, characterized in that: The live broadcast commodity includes a third live broadcast commodity; and the method further includes: When the currently explained product is the third live broadcast product, a pinned link of the third live broadcast product is displayed in the first live broadcast room.

21. The method according to claim 16, characterized in that The commentary audio is a human voice audio generated according to the target human voice tone selected by the anchor client.

22. A live broadcast device, characterized in that: The device comprises: A first interaction module is used to receive an upload operation of a material video, where the material video is pre-recorded; The first interaction module is used to receive a configuration operation of a live broadcast commodity, wherein the configuration operation is used to configure commodity information of the live broadcast commodity; The first playback module is used to play the live video of the first live broadcast room, the live video includes the video screen of the material video and the commentary audio, the commentary audio is a human voice audio synthesized based on the commentary text, and the commentary text is generated according to the product information and / or the material video.

23. A live broadcast device, characterized in that: The device comprises: A receiving module, used to receive a material video uploaded by a host client, where the material video is pre-recorded; The receiving module is used to receive the configuration information of the live broadcast commodity sent by the anchor client, wherein the configuration information includes the commodity information of the live broadcast commodity; A sending module is used to send a live video of the first live broadcast room, wherein the live video includes a video screen and a commentary audio of the material video, wherein the commentary audio is a human voice audio synthesized based on a commentary text, and the commentary text is generated based on the product information and / or the material video.

24. A live broadcast device, characterized in that: The device comprises: The second display module is used to display the live broadcast room selection interface; A second interaction module is used to receive a selection operation for the first live broadcast room in the live broadcast room selection interface; The second playback module is used to play the live video of the first live broadcast room; the live video includes the video screen and commentary audio of the material video; the material video is pre-recorded; the commentary audio is a human voice audio synthesized based on the commentary text of the live broadcast product; the commentary text is generated based on the product information of the live broadcast product and / or the material video.

25. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the live broadcast method as described in any one of claims 1 to 21.

26. A computer-readable storage medium, characterized in that: At least one program is stored in the readable storage medium, and the at least one program is loaded and executed by the processor to implement the live broadcast method as described in any one of claims 1 to 21.

27. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the live broadcast method as described in any one of claims 1 to 21.

Images