Digital human live broadcast method, apparatus and device, and storage medium
By obtaining interactive information and generating response audio, and combining preset digital human models for live broadcast of digital humans, the problem of poor application flexibility of digital humans is solved, and the effect of efficient response and reducing maintenance costs is achieved.
Patent Information
- Application Number
- CN202411115048.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-06
AI Technical Summary
The existing digital human application flexibility is poor, and it is difficult to flexibly promote and maintain in different scenarios, and the maintenance cost is high.
By obtaining the interactive information during the live broadcast of digital people, input it into the preset response model, generating response audio, and combining the preset digital people model to perform live broadcast of digital people, dynamically adjusting the performance of digital people to meet the needs of different scenarios.
It realizes the flexibility and efficient response of digital people's live broadcasts, reduces maintenance costs, and can respond and adjust in real time during the live broadcast to adapt to the needs of different scenarios.
Smart Images

Figure CN119094804A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a digital human live broadcast method, device, equipment and storage medium. Background Art
[0002] With the development of artificial intelligence technology, the maturity of 5G communication technology, and the continuous enrichment of human-computer interaction methods, virtual digital humans have emerged. Currently, the existing digital human technologies in the industry are mostly designed for certain scenarios, such as large-screen digital humans suitable for service halls, and performance-type digital humans suitable for image creation. Although digital humans in different scenarios have their own responsibilities and functions, and have produced a good experience effect to a certain extent, the more common scenarios currently use pre-set and offline synthesis of digital human related resources to ensure better results. Each new use scenario will increase the maintenance cost of the service, and it is difficult to promote the application of digital humans flexibly and conveniently. Summary of the invention
[0003] The main purpose of the present application is to provide a digital human live broadcast method, device, equipment and storage medium, aiming to solve the technical problem of poor flexibility in existing digital human applications.
[0004] To achieve the above purpose, the present application proposes a digital human live broadcast method, which includes:
[0005] Obtain interactive information during the digital human live broadcast;
[0006] Inputting the interaction information into a preset response model to obtain a response result output by the preset response model;
[0007] Generate response audio based on the response result combined with TTS;
[0008] The digital human live broadcast is performed according to the response audio and the preset digital human model.
[0009] Optionally, before the step of obtaining the interactive information during the live broadcast of the digital human, the method further includes:
[0010] When a live broadcast demand is received, determining live broadcast scene information according to the live broadcast demand;
[0011] Determine live broadcast related materials based on the live broadcast scene information;
[0012] Acquire target live broadcast material selected by the user based on the live broadcast related material;
[0013] Determine a preset digital human model based on the target live broadcast material and live broadcast scene information;
[0014] The digital human live broadcast is started according to the target live broadcast material, the live broadcast scene information and the preset digital human model.
[0015] Optionally, the target live broadcast material includes timbre;
[0016] After the step of obtaining the target live broadcast material selected by the user based on the live broadcast related material, the method further includes:
[0017] When receiving a timbre generation request, determining a sampled sound according to the timbre generation request;
[0018] determining a basic sound based on the timbre generation request;
[0019] Performing timbre editing according to the sampled sound and the basic sound to obtain an editing result;
[0020] The timbre is synthesized based on the editing result to obtain the target timbre.
[0021] Optionally, the step of starting the digital human live broadcast according to the target live broadcast material, the live broadcast scene information and the preset digital human model includes:
[0022] Determine the live broadcast background and the digital human position information according to the live broadcast scene information;
[0023] Determine the live broadcast screen based on the live broadcast background, the digital human position information and the preset digital human model;
[0024] Determine the live broadcast audio according to the target live broadcast material;
[0025] The digital human live broadcast is started according to the live broadcast audio and the live broadcast picture.
[0026] Optionally, before the step of starting the digital human live broadcast according to the target live broadcast material, the live broadcast scene information and the preset digital human model, the step further includes:
[0027] Determine the client information and the scene type according to the live scene information;
[0028] Determine an initial digital human image based on the client information and the scene type;
[0029] Obtain the image information of digital people required for live broadcast;
[0030] The initial digital human image is adjusted based on the digital human image information to obtain a preset digital human model.
[0031] Optionally, before the step of obtaining the interactive information during the live broadcast of the digital human, the method further includes:
[0032] When receiving a live broadcast demand, determining a live broadcast time and live broadcast resource information according to the live broadcast demand;
[0033] Querying live broadcast task information, and determining available live broadcast resources corresponding to the live broadcast time according to the live broadcast task information;
[0034] Determining whether a live broadcast condition is currently met according to the live broadcast resource information and the available live broadcast resources;
[0035] When the live broadcast condition is met, determine the business party information corresponding to the live broadcast requirement;
[0036] The target live broadcast resource is determined according to the available live broadcast resource, the live broadcast resource information and the business party information, and the digital human live broadcast is performed based on the target live broadcast resource.
[0037] Optionally, the step of inputting the interaction information into a preset response model to obtain a response result output by the preset response model includes:
[0038] Extracting features from the interaction information to obtain interaction features;
[0039] Acquire target live broadcast material, and determine basic response information according to the interaction feature and the target live broadcast material;
[0040] splicing the interaction information and the basic response information to obtain a splicing result;
[0041] The splicing result is input into a preset response model to obtain a response result output by the preset response model.
[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a digital human live broadcast device, which includes:
[0043] The acquisition module is used to obtain the interactive information of the digital human during the live broadcast;
[0044] A prediction module, used for inputting the interaction information into a preset response model to obtain a response result output by the preset response model;
[0045] An audio generation module, used to generate a response audio based on the response result in combination with TTS;
[0046] The digital human live broadcast module is used to perform digital human live broadcast according to the response audio and the preset digital human model.
[0047] In addition, to achieve the above-mentioned purpose, the present application also proposes a digital human live broadcast device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the digital human live broadcast method as described above.
[0048] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the digital human live broadcast method described above are implemented.
[0049] The present application obtains interactive information during the live broadcast of the digital human; inputs the interactive information into a preset response model to obtain a response result output by the preset response model; generates a response audio based on the response result combined with TTS; and performs live broadcast of the digital human according to the response audio and the preset digital human model. Since the present application determines the response audio according to the interactive information during the live broadcast of the digital human, the live broadcast of the digital human is performed according to the response audio and the preset digital human model. Compared with the existing method of playing pre-produced digital human videos, the above method of the present application can perform live broadcast of the digital human and perform efficient responses during the live broadcast. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0052] Figure 1 A flowchart of the first embodiment of the digital human live broadcast method of the present application;
[0053] Figure 2 A flow chart of the second embodiment of the digital human live broadcast method of the present application;
[0054] Figure 3 A schematic diagram of the interaction process provided for the second embodiment of the digital human live broadcast method of the present application;
[0055] Figure 4 This is a schematic diagram of the module structure of the digital human live broadcast device according to an embodiment of the present application;
[0056] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the digital human live broadcast method in the embodiment of the present application.
[0057] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0058] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0059] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0060] The main solution of the embodiment of the present application is: obtain the interactive information of the digital human during the live broadcast; input the interactive information into the preset response model to obtain the response result output by the preset response model; generate the response audio based on the response result combined with TTS; and perform the digital human live broadcast according to the response audio and the preset digital human model. Since the present application determines the response audio according to the interactive information during the live broadcast of the digital human, the digital human live broadcast is performed according to the response audio and the preset digital human model. Compared with the existing method of playing pre-produced digital human videos, the above method of the present application can perform digital human live broadcast and perform efficient responses during the live broadcast.
[0061] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a digital human live broadcast device, etc. The following takes the digital human live broadcast device as an example to illustrate this embodiment and the following embodiments.
[0062] Based on this, the embodiment of the present application provides a digital human live broadcast method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the digital human live broadcast method of the present application.
[0063] In this embodiment, the digital human live broadcast method includes steps S10 to S40:
[0064] Step S10, obtaining interactive information during the live broadcast of the digital human;
[0065] It should be noted that the interactive information may be interactive information sent by users watching the live broadcast of the digital human during the live broadcast of the digital human, for example, in the scenario of the live broadcast of the digital human selling goods, the user sends inquiry information about the product, about the live broadcast time and other chat information, etc. The live broadcast process of the digital human may be the process of live broadcasting using the digital human and materials designed in advance.
[0066] Step S20, inputting the interaction information into a preset response model to obtain a response result output by the preset response model;
[0067] It should be noted that the preset response model may be a pre-trained GPT model for response, which can parse the input interaction information and then obtain the response result according to the current live broadcast scene and material information.
[0068] Step S30, generating a response audio based on the response result in combination with TTS;
[0069] It should be noted that the TTS is from text to speech. The generation of response audio based on the response result combined with TTS can be to convert the response result into audio using TTS technology, that is, the response audio
[0070] Step S40: performing digital human live broadcast according to the response audio and the preset digital human model.
[0071] It should be noted that the live broadcast of the digital human according to the response audio and the preset digital human model may be to make the preset digital human model determined in advance according to the live broadcast scene broadcast the response audio to perform the live broadcast of the digital human.
[0072] Furthermore, in order to improve the utilization rate of live broadcast resources, before step S10, the method further includes: when receiving a live broadcast demand, determining a live broadcast time and live broadcast resource information according to the live broadcast demand;
[0073] Querying live broadcast task information, and determining available live broadcast resources corresponding to the live broadcast time according to the live broadcast task information;
[0074] Determining whether a live broadcast condition is currently met according to the live broadcast resource information and the available live broadcast resources;
[0075] When the live broadcast condition is met, determine the business party information corresponding to the live broadcast requirement;
[0076] The target live broadcast resource is determined according to the available live broadcast resource, the live broadcast resource information and the business party information, and the digital human live broadcast is performed based on the target live broadcast resource.
[0077] It should be noted that the live broadcast resource information may include resource information such as network bandwidth, GPU and TTS required for live broadcast, which may be specified by the user or obtained based on the live broadcast scene and live broadcast content evaluation. The live broadcast task information may be live broadcast resource information that has been reserved or occupied by the digital human live broadcast device. For example, 12:00-13:00 is the first live broadcast, and 12:00-14:00 is the first live broadcast. The live broadcast resource information also includes resources such as network bandwidth and GPU computing power to be occupied by each live broadcast. The determination of the available live broadcast resources corresponding to the live broadcast time according to the live broadcast task information may be to query the live broadcast task corresponding to the live broadcast time in the live broadcast task information, and determine the available live broadcast resources corresponding to the live broadcast time according to the occupied resources of the live broadcast task and the total resources of each indicator. The determination of whether the live broadcast condition is currently met according to the live broadcast resource information and the available live broadcast resources may be to compare the various indicators in the live broadcast resource information and the available live broadcast resources. When the various indicator values in the available live broadcast resources are all greater than the indicator values in the live broadcast resource information, it is determined that the live broadcast condition is met, otherwise, it is determined that the live broadcast condition is not met. When the live broadcast condition is met, in order to make the data processing speed faster during the live broadcast, more live broadcast resources can be allocated to the live broadcast according to the priority of the business party. Specifically, the business party information of the business party can be obtained, and the priority of the business party can be determined according to the business party information. When the priority of the business party is higher, more live broadcast resources can be allocated to the live broadcast according to the available live broadcast resources to obtain the target live broadcast resources.
[0078] In the specific implementation, before the live broadcast, the digital human live broadcast device needs to perform live broadcast configuration in advance for different viewing clients according to the live broadcast scene information, and pre-configures information such as resolution, streaming address, viewing address, pop-up window links during the live broadcast, etc. In order to enable the business party to better manage live broadcast resources and view live broadcast data statistics, in this embodiment, the business party can manage the live broadcast through the digital human live broadcast account.
[0079] This embodiment obtains interactive information during the digital human live broadcast; inputs the interactive information into a preset response model to obtain a response result output by the preset response model; generates a response audio based on the response result combined with TTS; and performs digital human live broadcast according to the response audio and the preset digital human model. Since this application determines the response audio based on the interactive information during the digital human live broadcast, the digital human live broadcast is performed according to the response audio and the preset digital human model. Compared with the existing method of playing pre-produced digital human videos, the above method of this embodiment can perform digital human live broadcast and perform efficient responses during the live broadcast.
[0080] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2, Figure 2 This is a flow chart of Embodiment 2 of the digital human live broadcast method of the present application. Before step S10, the following steps are also included:
[0081] Step S001: upon receiving a live broadcast demand, determining live broadcast scene information according to the live broadcast demand;
[0082] It should be noted that the live broadcast demand may be a request sent by the business party to broadcast live according to their demand information, including live broadcast scene information, which may include live broadcast start time, live broadcast duration, live broadcast scene, live broadcast resource information, digital human model image information, timbre, product information to be broadcast, etc.
[0083] Step S002: determining live broadcast related materials based on the live broadcast scene information;
[0084] It should be noted that the live broadcast related materials may include pictures that need to be displayed during the live broadcast, product introduction text, live broadcast background settings, screen editing and other information. The determination of live broadcast related materials based on the live broadcast scene information may be to determine material information related to the scene information according to the live broadcast scene information. For example, if the live broadcast scene is product introduction and the product is fruit, then the related materials may be pictures of the fruit, related introductions, live broadcast timbre, host position and live broadcast speech, etc.
[0085] Step S003: Acquire the target live broadcast material selected by the user based on the live broadcast related material;
[0086] It should be noted that the acquisition of the target live broadcast material selected by the user based on the live broadcast related material may be to display the live broadcast related material preliminarily determined based on the live broadcast scene information to the user, i.e., the business party, so that the user can select a satisfactory target live broadcast material based on the live broadcast related material for the digital human live broadcast. The target live broadcast material may also include the user's opinion information, such as suggestions on material adjustment, screen editing, etc.
[0087] Furthermore, in order to improve the user experience, the target live broadcast material includes timbre;
[0088] After the step of obtaining the target live broadcast material selected by the user based on the live broadcast related material, the method further includes:
[0089] When receiving a timbre generation request, determining a sampled sound according to the timbre generation request;
[0090] determining a basic sound based on the timbre generation request;
[0091] Performing timbre editing according to the sampled sound and the basic sound to obtain an editing result;
[0092] The timbre is synthesized based on the editing result to obtain the target timbre.
[0093] It should be noted that when the optional timbres provided do not meet the user's expectations, the timbre can be customized according to the sampled sound provided by the user. Specifically, a timbre generation request of the user is obtained, which contains a sampled sound corresponding to the timbre to be generated. It can be a user voice. The basic sound can be a sound determined according to the timbre style in the timbre generation request. Specifically, the basic sound is designed by software such as an electronic synthesizer or a digital audio workstation (DAW). This includes selecting appropriate waveforms, filters, effects and other parameters to create the desired timbre characteristics. The timbre editing according to the sampled sound and the basic sound can be further timbre editing and processing after the basic sound is obtained or designed to adjust and optimize the timbre characteristics. This includes: Equalization (EQ): changing the brightness and softness of the timbre by adjusting the gain of different frequency bands. Compression and limitation: controlling the dynamic range of the volume to make the timbre more stable and smooth. Reverberation and delay: adding a sense of space to simulate the sound propagation effect in different environments. Distortion and overload: creating specific timbre effects, such as guitar distortion. Filtering: Change the frequency response characteristics of the timbre through high-pass, low-pass, band-pass and other filters. The timbre synthesis based on the editing result can be performed after the timbre editing is completed, and the timbre synthesis and adjustment are performed to obtain the final timbre effect. This includes: multi-layer superposition: superimpose multiple basic sound samples or synthesized timbres to create richer timbre levels. Cross-fading: When processing timbre with large changes in intensity, use cross-fading technology to smoothly transition the timbre characteristics under different intensities. Fine-tuning parameters: Fine-tune the timbre parameters such as pitch, volume, timbre thickness, etc. according to actual needs to obtain a more natural timbre effect.
[0094] Step S004: determining a preset digital human model based on the target live broadcast material and live broadcast scene information;
[0095] It should be noted that determining the preset digital human model based on the target live broadcast material and the live broadcast scene information can be to train a digital human image that conforms to the live broadcast scene according to the live broadcast scene information, and then adjust the digital human image based on the opinion image in the target live broadcast material to obtain the preset digital human model.
[0096] Step S005: starting the digital human live broadcast according to the target live broadcast material, live broadcast scene information and the preset digital human model.
[0097] It should be noted that starting the digital human live broadcast according to the target live broadcast material, the live broadcast scene information and the preset digital human model can be to configure the live broadcast resources according to the live broadcast scene information, determine the content of the digital human live broadcast according to the target live broadcast material, and then use the preset digital human model to perform the digital human live broadcast.
[0098] Furthermore, in order to make the live broadcast screen more coordinated and beautiful and improve the user experience, the step S005 may include: determining the live broadcast background and the digital human position information according to the live broadcast scene information;
[0099] Determine the live broadcast screen based on the live broadcast background, the digital human position information and the preset digital human model;
[0100] Determine the live broadcast audio according to the target live broadcast material;
[0101] The digital human live broadcast is started according to the live broadcast audio and the live broadcast picture.
[0102] It should be noted that the live broadcast background may be the screen background during live broadcast, and the digital human position information may be the position of the digital human in the screen, whether it is centered or on the side or in the form of a picture-in-picture, etc. The determination of the live broadcast screen based on the live broadcast background, the digital human position information and the preset digital human model may be to use the live broadcast background as the live broadcast screen background, set the position of the preset digital human model according to the digital human position information, and obtain the live broadcast screen. The determination of the live broadcast audio according to the target live broadcast material may be to generate the live broadcast audio based on the target live broadcast material using TTS.
[0103] In order to improve the user experience, before step S005, the method may include: determining the client information and the scene type according to the live scene information;
[0104] Determine an initial digital human image based on the client information and the scene type;
[0105] Obtain the image information of digital people required for live broadcast;
[0106] The initial digital human image is adjusted based on the digital human image information to obtain a preset digital human model.
[0107] It should be noted that the scene types may include store live broadcast, production site live broadcast, live broadcast room live broadcast, and scenes with specific dress-up themes. The client information may be a PC or mobile phone, an outdoor large screen, etc. The determination of the initial digital human image based on the client information and the scene type may be to select a digital human image that conforms to the live broadcast scene and is adapted to the live broadcast page size of the client according to the client information and the scene type. The live broadcast requirements may also include the image information of the digital human required by the user, including style, facial features, clothing, and other information. The adjustment of the initial digital human image based on the digital human image information to obtain the preset digital human model may be to input the digital human image information and the initial digital human image into a preset digital human image generation model to obtain the preset digital human model output by the preset digital human image generation model. The initial digital human image may also be directly adjusted according to the style, facial features, clothing, and other information in the digital human image information to obtain the preset digital human model. The preset digital human image generation model may be a model that is pre-trained with sample data and can generate a preset digital human model according to the digital human image information and the initial digital human image.
[0108] In the specific implementation, please refer to Figure 3 , Figure 3 The interactive flow diagram provided for the second embodiment of the digital human live broadcast method of this application; wherein, the client identification can be used to determine the platform information of the digital human live broadcast, such as whether it is live broadcast on a mobile phone or on an outdoor large screen such as a computer, which determines the configuration information of the live broadcast page. The user's question input is to obtain the user's interactive information, and then the answer result is determined by the answering ability of the GPT model, and the response audio is generated in combination with TTS. The audio is played based on the digital human model to complete the real-time digital human live broadcast.
[0109] When receiving a live broadcast demand, this embodiment determines live broadcast scene information according to the live broadcast demand; determines live broadcast related materials based on the live broadcast scene information; obtains the target live broadcast material selected by the user based on the live broadcast related materials; determines a preset digital human model based on the target live broadcast material and live broadcast scene information; and starts the digital human live broadcast according to the target live broadcast material, live broadcast scene information and the preset digital human model. This embodiment can determine the related materials according to the live broadcast scene information, train the digital human model, and then perform digital human live broadcast according to the related materials, which can improve the efficiency of digital human live broadcast and improve the user experience.
[0110] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the digital human live broadcast method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.
[0111] This application also provides a digital human live broadcast device, please refer to Figure 4 , the digital human live broadcast device comprises:
[0112] The acquisition module 10 is used to acquire the interactive information of the digital human during the live broadcast;
[0113] The prediction module 20 is used to input the interaction information into a preset response model to obtain a response result output by the preset response model;
[0114] An audio generation module 30, configured to generate a response audio based on the response result in combination with TTS;
[0115] The digital human live broadcast module 40 is used to perform digital human live broadcast according to the response audio and the preset digital human model.
[0116] This embodiment obtains interactive information during the digital human live broadcast; inputs the interactive information into a preset response model to obtain a response result output by the preset response model; generates a response audio based on the response result combined with TTS; and performs digital human live broadcast according to the response audio and the preset digital human model. Since this application determines the response audio based on the interactive information during the digital human live broadcast, the digital human live broadcast is performed according to the response audio and the preset digital human model. Compared with the existing method of playing pre-produced digital human videos, the above method of this embodiment can perform digital human live broadcast and perform efficient responses during the live broadcast.
[0117] The digital human live broadcast device provided by the present application adopts the digital human live broadcast method in the above embodiment, which can solve the technical problem of poor flexibility of existing digital human applications. Compared with the prior art, the beneficial effects of the digital human live broadcast device provided by the present application are the same as the beneficial effects of the digital human live broadcast method provided by the above embodiment, and other technical features in the digital human live broadcast device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0118] The present application provides a digital human live broadcast device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the digital human live broadcast method in the above-mentioned embodiment one.
[0119] Reference below Figure 5, which shows a schematic diagram of the structure of a digital human live broadcast device suitable for implementing the embodiment of the present application. The digital human live broadcast device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The digital human live broadcast device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0120] like Figure 5 As shown, the digital human live broadcast device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the digital human live broadcast device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the digital human live broadcast device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a digital human live broadcast device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.
[0121] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0122] The digital human live broadcast device provided by the present application adopts the digital human live broadcast method in the above embodiment, which can solve the technical problem of poor flexibility of existing digital human applications. Compared with the prior art, the beneficial effects of the digital human live broadcast device provided by the present application are the same as the beneficial effects of the digital human live broadcast method provided by the above embodiment, and the other technical features of the digital human live broadcast device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0123] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0125] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the digital human live broadcast method in the above-mentioned embodiment.
[0126] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0127] The computer-readable storage medium may be included in the digital human live broadcast device; or it may exist independently without being assembled into the digital human live broadcast device.
[0128] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the digital human live broadcast device, the digital human live broadcast device executes the digital human live broadcast method.
[0129] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0131] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0132] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned digital human live broadcast method, and can solve the technical problem of poor flexibility in existing digital human applications. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the digital human live broadcast method provided in the above-mentioned embodiment, and will not be repeated here.
[0133] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A digital human live broadcast method, characterized in that: The digital human live broadcast method comprises the following steps: Obtain interactive information during the digital human live broadcast; Inputting the interaction information into a preset response model to obtain a response result output by the preset response model; Generate response audio based on the response result combined with TTS; The digital human live broadcast is performed according to the response audio and the preset digital human model.
2. The digital human live broadcast method according to claim 1, characterized in that: Before the step of obtaining the interactive information during the live broadcast of the digital human, the method further includes: When a live broadcast demand is received, determining live broadcast scene information according to the live broadcast demand; Determine live broadcast related materials based on the live broadcast scene information; Acquire target live broadcast material selected by the user based on the live broadcast related material; Determine a preset digital human model based on the target live broadcast material and live broadcast scene information; The digital human live broadcast is started according to the target live broadcast material, the live broadcast scene information and the preset digital human model.
3. The digital human live broadcast method according to claim 2, characterized in that: The target live broadcast material includes timbre; After the step of obtaining the target live broadcast material selected by the user based on the live broadcast related material, the method further includes: When receiving a timbre generation request, determining a sampled sound according to the timbre generation request; determining a basic sound based on the timbre generation request; Performing timbre editing according to the sampled sound and the basic sound to obtain an editing result; The timbre is synthesized based on the editing result to obtain the target timbre.
4. The digital human live broadcast method according to claim 2, characterized in that: The step of starting the digital human live broadcast according to the target live broadcast material, the live broadcast scene information and the preset digital human model includes: Determine the live broadcast background and the digital human position information according to the live broadcast scene information; Determine the live broadcast screen based on the live broadcast background, the digital human position information and the preset digital human model; Determine the live broadcast audio according to the target live broadcast material; The digital human live broadcast is started according to the live broadcast audio and the live broadcast picture.
5. The digital human live broadcast method according to claim 2, characterized in that: Before the step of starting the digital human live broadcast according to the target live broadcast material, the live broadcast scene information and the preset digital human model, the step further includes: Determine the client information and the scene type according to the live scene information; Determine an initial digital human image based on the client information and the scene type; Obtain the image information of digital people required for live broadcast; The initial digital human image is adjusted based on the digital human image information to obtain a preset digital human model.
6. The digital human live broadcast method according to any one of claims 1 to 5, characterized in that: Before the step of obtaining the interactive information during the live broadcast of the digital human, the method further includes: When receiving a live broadcast demand, determining a live broadcast time and live broadcast resource information according to the live broadcast demand; Querying live broadcast task information, and determining available live broadcast resources corresponding to the live broadcast time according to the live broadcast task information; Determining whether a live broadcast condition is currently met according to the live broadcast resource information and the available live broadcast resources; When the live broadcast condition is met, determine the business party information corresponding to the live broadcast requirement; The target live broadcast resource is determined according to the available live broadcast resource, the live broadcast resource information and the business party information, and the digital human live broadcast is performed based on the target live broadcast resource.
7. The digital human live broadcast method according to any one of claims 1 to 5, characterized in that: The step of inputting the interaction information into a preset response model to obtain a response result output by the preset response model includes: Extracting features from the interaction information to obtain interaction features; Acquire target live broadcast material, and determine basic response information according to the interaction feature and the target live broadcast material; splicing the interaction information and the basic response information to obtain a splicing result; The splicing result is input into a preset response model to obtain a response result output by the preset response model.
8. A digital human live broadcast device, characterized in that: The digital human live broadcast device comprises: The acquisition module is used to obtain the interactive information of the digital human during the live broadcast; A prediction module, used for inputting the interaction information into a preset response model to obtain a response result output by the preset response model; An audio generation module, used to generate a response audio based on the response result in combination with TTS; The digital human live broadcast module is used to perform digital human live broadcast according to the response audio and the preset digital human model.
9. A digital human live broadcast device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the digital human live broadcast method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the digital human live broadcast method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Digital character audio and video generation method and digital character live broadcast interaction method
CN119562140A
A digital role audio and video generation method and a digital role live interaction method
CN119562140B
Real-time interaction method and system for digital human live broadcast, electronic equipment and program product
CN120812308A
