Digital human live broadcast control method, device, system, equipment and program product

Abstract

This application relates to a control method, apparatus, system, device, and program product for live streaming of digital humans. The method includes: acquiring cookie information, wherein the cookie information is extracted from the browser by a browser plugin after detecting that a user has logged into a live streaming control panel page through the browser; processing audience interaction information during the live streaming of digital humans based on the cookie information; and / or controlling the display of product pop-ups during the live streaming of digital humans. The solution provided by this application can automatically capture and upload cookie information when a user logs into the live streaming control panel page using a browser plugin, without requiring additional user operation. Simultaneously, it can use cookie information to replace the live streaming control panel page to process audience interaction information during the live streaming of digital humans, and / or control the display of product pop-ups during the live streaming of digital humans, without requiring the opening of a specific browser version, thus improving the stability of live streaming of digital humans.

Description

Technical Field

[0001] This application relates to the field of digital human live streaming technology, and in particular to control methods, devices, systems, equipment and program products for digital human live streaming. Background Technology

[0002] The digital economy industry refers to a series of economic activities that use digital knowledge and information as core production factors, modern information networks as key carriers, and information and communication technologies to drive efficiency improvements and economic structure optimization. These include fields such as intelligent transportation, internet finance, mobile payment, and artificial intelligence. Digital human live streaming is an important branch of artificial intelligence.

[0003] With the rapid evolution of the digital economy, digital human technology is developing rapidly, and more and more live streaming scenarios are beginning to use digital humans for broadcasting. Currently, digital human live streaming widely uses web crawlers to parse the browser's central control page to achieve live streaming operation and control functions. However, this method requires continuously keeping a specific version of the browser open, and because browser pages are prone to interference or crashes, it leads to poor stability of digital human live streaming. Summary of the Invention

[0004] To address or partially address the problems existing in related technologies, this application provides a control method, apparatus, system, device, and program product for live streaming of digital humans, which can improve the stability of live streaming of digital humans.

[0005] The first aspect of this application provides a method for controlling live streaming of a digital human, including:

[0006] Obtain Cookie information; wherein, the Cookie information is extracted from the browser by the browser plugin after detecting that the user logs into the live broadcast control panel page through the browser;

[0007] Based on the cookie information, process the audience's interaction information during the digital human live stream, and / or control the display of product pop-ups during the digital human live stream.

[0008] Furthermore, in the method described above, processing the audience's interactive information during the digital human live stream includes:

[0009] In response to processing instructions from the live streaming device, the system captures interactive information from viewers during the live streaming of the digital human and sends the interactive information to the live streaming device via a first interface, so that the live streaming device generates and outputs a first response text in response to the interactive information.

[0010] Furthermore, in the method described above, processing the audience's interactive information during the digital human live stream includes:

[0011] In response to processing instructions from the live streaming device, the system captures interactive information from viewers during the live streaming of the digital human and sends the interactive information to the live streaming device via a first interface, so that the live streaming device generates a second response text in response to the interactive information.

[0012] The second response text is obtained, a digital human response audio and video is generated based on the second response text, and the digital human response audio and video is sent to the live streaming device so that the live streaming device outputs the digital human response audio and video.

[0013] Furthermore, in the method described above, controlling the display of product pop-ups during the digital human live stream includes:

[0014] In response to a pop-up command for a target product from the live streaming device, the product pop-up is retrieved from the product sales platform based on the identification identifier of the target product; the target product includes any product that needs to be displayed during the digital human live streaming process, and the pop-up command for the target product is generated by the live streaming device when displaying the target product;

[0015] The product pop-up window of the target product is sent to the live streaming device via the second interface, so that the live streaming device can output the product pop-up window of the target product at a set frequency when displaying the target product.

[0016] Furthermore, in the method described above, before the display of the product pop-up window during the live stream of the digital human, the following steps are included:

[0017] Based on the product identification marks, retrieve all product pop-ups from the product sales platform; the products include those to be displayed during the digital human live stream.

[0018] Correspondingly, the display of product pop-ups during the live stream of the digital human includes:

[0019] When the live streaming device starts displaying the target product, the product pop-up window of the target product is sent to the live streaming device through the second interface, so that the live streaming device can output the product pop-up window of the target product at a set frequency when displaying the target product; the target product includes any product that needs to be displayed during the live streaming of the digital human.

[0020] Furthermore, in the method described above, the pre-generated digital human live-streaming audio and video are played in a loop during the digital human live-streaming process; the step of sending the product pop-up window of the target product to the live-streaming device through the second interface when the live-streaming device is detected to start displaying the target product includes:

[0021] Based on the playback duration of the digital human live audio and video and the display time of each product in the digital human live audio and video, determine whether the live streaming device should start displaying the target product;

[0022] If the live streaming device starts displaying the target product, the product pop-up window of the target product is sent to the live streaming device through the second interface.

[0023] A second aspect of this application provides a control device for live streaming of digital humans, comprising:

[0024] The acquisition module is used to acquire Cookie information; wherein, the Cookie information is extracted from the browser by the browser plugin after detecting that the user logs into the live broadcast control console page through the browser;

[0025] The control module is used to process audience interaction information during the digital human live stream based on the cookie information, and / or control the display of product pop-ups during the digital human live stream.

[0026] A third aspect of this application provides a processing apparatus, comprising:

[0027] Processor; and

[0028] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0029] The fourth aspect of this application provides a control system for live streaming of digital humans, including: live streaming equipment, browser plugin, and processing equipment;

[0030] The processing device is communicatively connected to the browser plugin and the live streaming device, respectively.

[0031] The browser plugin is used to extract cookie information from the browser and send the cookie information to the processing device after detecting that a user has logged into the live broadcast control console page through a browser.

[0032] The live streaming device is used to conduct digital human live streaming by playing digital human live streaming audio and video;

[0033] The processing device is used to process audience interaction information during the digital human live stream based on the cookie information, and / or control the display of product pop-ups during the digital human live stream.

[0034] The fifth aspect of this application provides a computer program product comprising computer instructions that, when executed by a processor, implement the method described above.

[0035] The technical solution provided in this application may include the following beneficial results:

[0036] The technical solution of this application obtains Cookie information, which is extracted from the browser by the browser plugin after detecting that a user logs into the live streaming control panel page through the browser. Based on the Cookie information, it processes audience interaction information during the live streaming of the digital human and / or controls the display of product pop-ups during the live streaming. This setting allows the browser plugin to automatically capture and upload Cookie information when the user logs into the live streaming control panel page, without requiring additional user operation. Simultaneously, it can use Cookie information to replace the live streaming control panel page to process audience interaction information and / or control the display of product pop-ups during the live streaming of the digital human, without requiring a specific browser version, thus improving the stability of the live streaming of the digital human.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0038] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0039] Figure 1 This is a flowchart illustrating the control method for live streaming of digital humans as shown in an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of the structure of the control device for live streaming of digital humans, as shown in the embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the processing device shown in the embodiments of this application;

[0042] Figure 4 This is a schematic diagram of the control system for live streaming of digital humans, as shown in an embodiment of this application. Detailed Implementation

[0043] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0044] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0045] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0046] The digital economy industry refers to a series of economic activities that use digital knowledge and information as core production factors, modern information networks as key carriers, and information and communication technologies to drive efficiency improvement and economic structure optimization. These include fields such as intelligent transportation, internet finance, mobile payment, and artificial intelligence. Digital human live streaming is an important branch of artificial intelligence. With the rapid development of digital human technology, its penetration in the live streaming field is constantly expanding. From product guidance on e-commerce platforms and new product launches by brands, to course presentations by educational institutions and introductions to tourist attractions in the cultural and tourism industry, digital humans are increasingly becoming "new members" of the live streaming lineup due to their characteristics of not needing rest, having a uniform appearance, and being customizable.

[0047] Currently, digital human live streaming widely employs web crawlers to parse the browser's central control page to achieve live streaming operation and control functions. However, this method requires continuously keeping a specific version of the browser open, and because browser pages are prone to interference or crashes, digital human live streaming suffers from poor stability. Frequent technical glitches in digital human live streaming significantly diminish the overall effect of the live stream, making it difficult to achieve the expected dissemination or conversion goals, and greatly impacting the development of the digital economy industry.

[0048] To address the aforementioned issues, this application provides a control method, apparatus, system, device, and program product for digital human live streaming, which can improve the stability of digital human live streaming. Furthermore, improved stability of digital human live streaming can prevent frequent malfunctions during the live streaming process, thereby enhancing the live streaming effect, achieving the expected dissemination or conversion goals, and avoiding impact on the development of the digital economy industry.

[0049] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0050] This application proposes a control method for live streaming of digital humans. This method can be executed by a processing device, which can be any device with data and instruction processing capabilities. For example, it can be various types of user terminals such as laptops, tablets, desktop computers, and mobile devices, or any combination of two or more of these processing devices, or it can be a server. See also... Figure 1 As shown, the method includes:

[0051] S101. Obtain Cookie information.

[0052] In related technologies, when using digital humans for live streaming, users log in to the live streaming control panel page via a browser and use web crawlers to parse the page to implement operational and control functions. The live streaming control panel page refers to the web-based backend management interface for the digital human live stream.

[0053] The main approach involves keeping the live streaming control panel in the browser constantly open to ensure a continuous live data stream and prevent login failures or connection interruptions. It also involves periodically crawling and manipulating Document Object Model (DOM) elements to extract text content or attribute changes, thereby obtaining viewer behavior information such as comments, likes, entering and leaving the live stream, and adding items to shopping carts. However, this method requires continuously keeping a specific version of the browser open, and because browser pages are prone to interference or crashes, the stability of the digital human live stream is poor.

[0054] In the embodiments of this application, a browser plugin is provided. This browser plugin is used to extract cookie information from the browser after detecting that a user has logged into the live broadcast control panel page through a browser.

[0055] Specifically, the browser plugin can be a standard Chrome plugin that conforms to the Google Chrome extension development specification, and the browser plugin can crawl cookie information based on the corresponding domain.

[0056] Cookie information is small text data stored in a browser, used to identify a user or record their status when they visit the site. In the digital human live streaming scenario of this embodiment, the cookie information obtained by the browser plugin mainly includes user authentication information, such as interface signature verification, login status identifier, and device unique identifier, etc., which are not limited in this embodiment.

[0057] In one specific embodiment, a browser plugin can be installed first. Then, the user logs into the live broadcast control console page through the browser. After the browser plugin detects the user's login, it automatically extracts all cookie information of the current page, including interface signature verification, login status identifier, and device unique identification code, and then sends the cookie information to the processing device.

[0058] In this embodiment, the processing device obtains the cookie information extracted by the browser plugin.

[0059] S102. Based on cookie information, process the audience's interactive information during the digital human live stream, and / or control the display of product pop-ups during the digital human live stream.

[0060] Cookie information obtained through browser plugins is essentially a user identity credential. In the embodiments of this application, the processing device can use these user identity credentials to simulate a logged-in real user and send requests to the relevant platform. Thus, the technical solution of this embodiment can replace the browser to interact with the relevant platform, without needing to continuously open a specific version of the browser, avoiding the problem of browser pages being easily interfered with or crashing, which leads to poor stability of digital human live streaming.

[0061] Furthermore, based on cookie information, the processing device can simulate users to automatically process audience interaction information during the digital human live stream, and / or control the display of product pop-ups during the digital human live stream.

[0062] In the above embodiments, Cookie information is obtained. This Cookie information is extracted from the browser by the browser plugin after detecting that a user has logged into the live streaming control panel page through the browser. Based on the Cookie information, audience interaction information during the live streaming of the digital human is processed, and / or, the display of product pop-ups during the live streaming of the digital human is controlled. This setup allows the browser plugin to automatically capture and upload Cookie information when the user logs into the live streaming control panel page, without requiring additional user operation. Simultaneously, Cookie information can replace the live streaming control panel page in processing audience interaction information during the live streaming of the digital human, and / or controlling the display of product pop-ups during the live streaming of the digital human, without requiring the opening of a specific browser version, thus improving the stability of the live streaming of the digital human.

[0063] Furthermore, this setup relies solely on browser plugins to upload cookie information during login, with all subsequent interactions simulated on the processing device, completely eliminating dependence on the browser. It also avoids the problem of script failure due to webpage updates by no longer relying on the front-end page structure, resulting in stronger stability and compatibility. At the same time, it eliminates the need to run multiple browser instances, allowing a single machine to host multiple digital human live streams, significantly reducing maintenance costs.

[0064] As an optional implementation, the steps in the above embodiments for processing audience interaction information during digital human live streaming specifically include the following steps:

[0065] In response to processing instructions from the live streaming device, the system captures audience interaction information during the digital human live stream and sends the interaction information to the live streaming device via a first interface, so that the live streaming device generates and outputs a first response text to the interaction information.

[0066] The aforementioned live streaming device refers to a device that controls the playback of live audio and video of a digital human. This device can generate processing instructions. In some embodiments, the live streaming device can generate a first processing instruction every set interval, such as 2 seconds, and then send the first processing instruction to a processing device to trigger the processing device to process the interactive information. The set interval can be adjusted according to actual circumstances, and this embodiment does not impose a limitation.

[0067] More specifically, the processing device is equipped with a first interface. The live streaming device can poll the first interface once every set time interval, and send a processing command each time it is polled to trigger the processing device to process the interactive information.

[0068] The aforementioned interactive information refers to various participatory behaviors initiated by users during the live stream that are related to the live stream content, including entering the live stream room, commenting, liking, adding to cart, etc. This embodiment does not limit such behaviors.

[0069] In this embodiment, the processing device responds to the processing instructions from the live streaming device, captures the audience's interaction information during the digital human live stream, and then sends the interaction information to the live streaming device based on the first interface.

[0070] For example, after sending interactive information to the live streaming device according to the previous processing instruction and before receiving a new processing instruction, the generated new interactive information includes:

[0071] User A enters the live stream;

[0072] User B commented: Which size should I buy for this dress if I weigh 100 pounds?

[0073] User B liked the video.

[0074] In the embodiments of this application, in response to the processing instructions from the live streaming device, the aforementioned interactive information can be captured and sent to the live streaming device based on the first interface.

[0075] The live streaming equipment can generate and output the first response text based on the interactive information.

[0076] In some embodiments, the live streaming device has a pre-set response database containing response texts corresponding to various interactive information. After acquiring interactive information, the live streaming device can select the response text corresponding to the interactive information from the response database as the first response text and output it. In other embodiments, the live streaming device can call a large language model, input the interactive information into the large language model, obtain the response result output by the large language model as the first response text, and output it.

[0077] In the above embodiments, the processing device can simulate a user, automatically capture the audience's interactive information according to the processing instructions of the live streaming device, and send the interactive information to the live streaming device so that the live streaming device can automatically respond to the user's interactive information in the form of text, thereby improving the interactive effect of the live stream and enhancing the user's viewing experience.

[0078] In an optional embodiment, the steps of the above embodiments for processing audience interaction information during digital human live streaming may specifically include the following steps:

[0079] In response to processing instructions from the live streaming device, the system captures audience interaction information during the live streaming of the digital human and sends the interaction information to the live streaming device via a first interface, so that the live streaming device generates a second response text to respond to the interaction information; the system obtains the second response text, generates digital human response audio and video based on the second response text, and sends the digital human response audio and video to the live streaming device, so that the live streaming device outputs digital human response audio and video.

[0080] The aforementioned live streaming device refers to a device that controls the playback of live audio and video of a digital human. This device can generate processing instructions. In some embodiments, the live streaming device can generate a first processing instruction every set interval, and then send the first processing instruction to a processing device to trigger the processing device to process the interactive information. The set interval can be configured according to actual circumstances, and this embodiment does not impose a limitation.

[0081] More specifically, the processing device is equipped with a first interface. The live streaming device can poll the first interface once every set time interval, and send a processing command each time it is polled to trigger the processing device to process the interactive information.

[0082] The aforementioned interactive information refers to various participatory behaviors initiated by users during the live stream that are related to the live stream content, including entering the live stream room, commenting, liking, adding to cart, etc. This embodiment does not limit such behaviors.

[0083] In this embodiment, the processing device responds to the processing instructions from the live streaming device, captures the audience's interaction information during the digital human live stream, and then sends the interaction information to the live streaming device based on the first interface.

[0084] The live streaming device can generate a second response text based on the interactive information. In some embodiments, the live streaming device has a pre-set response database containing response texts corresponding to various interactive information. After obtaining the interactive information, the live streaming device can select the response text corresponding to the interactive information from the response database as the second response text. In other embodiments, the live streaming device can call a large language model, input the interactive information into the large language model, and obtain the response result output by the large language model as the second response text.

[0085] The live streaming device in this embodiment can generate response text, but it does not have the ability to synthesize audio and video. In the embodiments of this application, the live streaming device sends the second response text to the processing device, and the processing device generates digital human response audio and video based on the second response text and sends the digital human response audio and video to the live streaming device.

[0086] In some embodiments, the second response text can be first converted into audio, and then the converted audio can be used to drive the lip region of the digital human to generate a video of the digital human's lip region whose lip movements are consistent with the audio. For example, feature points and audio features of the digital human's lip region can be obtained during a live stream, where the audio features can use Mel-spectral coefficients.

[0087] The feature points and audio features of the digital lip region are input into a video generation network. This network can be Wav2Lip or Wav2Lip-GAN, etc., and this embodiment is not limited to any particular network. The video generation network modifies the shape of the static lip region based on the feature points and audio features of the digital lip region, making the movement of the digital lip region visually consistent with the audio. Then, it outputs a dynamic video of the lip region that is visually consistent with the audio.

[0088] Then, the lip area of ​​the digital human is masked, and the dynamic video of the lip area is pasted back onto the masked part to obtain the dynamic video of the digital human. The video and audio are combined together to obtain the digital human's response audio and video. The digital human's response audio and video is sent to the live streaming device, and the live streaming device can output the digital human's response audio and video.

[0089] This allows for the generation of digital human-generated audio and video responses to interact with the live stream audience, enhancing the interactivity of the live stream.

[0090] In an optional embodiment, the interactive information in the above embodiments includes first interactive information or second interactive information; the steps in the above embodiments process the interactive information of the audience during the digital human live broadcast, which may specifically include the following steps:

[0091] In response to processing instructions from the live streaming device, capture the audience's interaction information during the digital human live stream, and send the interaction information to the live streaming device based on the first interface;

[0092] The first interactive information is used to trigger the live streaming device to generate and output the first response text of the interactive response information.

[0093] The second interactive information is used to trigger the live streaming device to generate a second response text for the interactive information; the second response text is obtained, and a digital human response audio and video is generated based on the second response text and sent to the live streaming device so that the live streaming device can output the digital human response audio and video.

[0094] If all user interaction messages are responded to via text, the interactivity of the digital human live stream will be reduced. If all user interaction messages are responded to via digital human audio or video, the workload of the processing equipment will be increased, and the response delay will also reduce the interactivity of the digital human live stream.

[0095] Based on this, in the embodiments of this application, the interactive information is divided into first interactive information and second interactive information, wherein the first interactive information refers to information that can be replied to using text, and the second interactive information refers to information that can be replied to using digital human response audio and video.

[0096] For example, viewer interaction information includes entering the live stream, commenting, liking, and adding items to the cart. Entering the live stream, liking, and adding items to the cart can be set as the first interaction information, and commenting as the second interaction information. The interaction behaviors included in the first and second interaction information can also be set according to actual circumstances; this embodiment does not impose limitations.

[0097] Furthermore, in response to processing instructions from the live streaming device, the processing device can capture interactive information from the audience during the live streaming of the digital human and send the interactive information to the live streaming device based on the first interface.

[0098] If the interactive information includes first interactive information, the processing device can generate and output a first response text based on the first interactive information. If the interactive information includes second interactive information, the processing device can generate and send a second response text based on the second interactive information. After obtaining the second response text, the processing device generates digital human response audio and video based on the second response text and sends the digital human response audio and video to the live streaming device, so that the live streaming device outputs the digital human response audio and video.

[0099] In the above embodiments, text or audio / video replies can be selected based on the type of interactive information, effectively enhancing the interactivity of digital human live streaming and improving the live streaming effect.

[0100] As an optional embodiment, the steps of the above embodiment to control the display of product pop-ups during the digital human live stream specifically include the following steps:

[0101] In response to a pop-up command for a target product from the live streaming device, the device retrieves the product pop-up from the product sales platform based on the product's identification identifier. The target product includes any product that needs to be displayed during the digital human live stream. The pop-up command for the target product is generated by the live streaming device when displaying the target product. The product pop-up is sent to the live streaming device via a second interface so that the live streaming device can output the product pop-up at a set frequency when displaying the target product.

[0102] The aforementioned product pop-up refers to an interactive window that pops up in real time during the live stream to display core content such as product information, price, and purchase link, guiding viewers to click and place an order directly.

[0103] The target products mentioned above include any products that need to be displayed during the digital human live broadcast, and this embodiment does not limit them.

[0104] In the embodiments of this application, the live streaming device generates a pop-up command for the target product when displaying the target product, and sends the pop-up command to the processing device.

[0105] In response to the pop-up command for the target product from the live streaming device, the processing device retrieves the product pop-up from the product sales platform based on the product's identification identifier. The product's identification identifier refers to its Identity Document (ID) on the product sales platform; based on this identifier, the product pop-up can be retrieved from the platform.

[0106] It should be noted that the processing device obtains cookie information, and the processing device can communicate with the product sales platform based on the cookie information to obtain the product pop-up window for the target product.

[0107] The processing device is equipped with a second interface. The processing device sends the product pop-up window of the target product to the live streaming device via the second interface. When displaying the target product, the live streaming device outputs the product pop-up window of the target product at a set frequency. This set frequency can be set according to the actual situation, and is not limited in this embodiment.

[0108] This setup allows the system to retrieve the latest product pop-up from the product sales platform when displaying the target product. This ensures that even if the product information is modified, the latest product pop-up will still be available, preventing discrepancies between the product information displayed in the pop-up and the actual product, or situations where the pop-up is unavailable.

[0109] As an optional embodiment, the steps of the above embodiment, before displaying product pop-ups during the digital human live stream, include: obtaining product pop-ups for all products from the product sales platform based on product identification tags; the products include those to be displayed during the digital human live stream.

[0110] The steps of the above embodiments to control the display of product pop-ups during the live streaming of the digital human specifically include the following steps: when the live streaming device starts to display the target product, the product pop-up of the target product is sent to the live streaming device through the second interface, so that the live streaming device can output the product pop-up of the target product at a set frequency when displaying the target product; the target product includes any product that needs to be displayed during the live streaming of the digital human.

[0111] Specifically, in addition to obtaining the latest product pop-up from the product sales platform in real time when displaying the target product as described in the above embodiments, the embodiments of this application provide another way to output the product pop-up, as follows:

[0112] In the embodiments of this application, some preparatory work is required before displaying the product pop-up, including obtaining the product pop-ups of all products from the product sales platform based on the identification tags of all products to be displayed during the digital human live stream. As described in the above embodiments, the processing device obtains cookie information, and the processing device can communicate with the product sales platform based on the cookie information to obtain the product pop-up of the target product.

[0113] Then, when the live streaming device starts displaying the target product, the product pop-up window is sent to the live streaming device through the second interface. When the live streaming device displays the target product, it outputs the product pop-up window at a set frequency. The above-mentioned set frequency can be set according to the actual situation, and this embodiment does not limit it.

[0114] In the above embodiments, product pop-ups for all products can be obtained in advance, and the corresponding product pop-ups can be sent to the live streaming device when the target product starts to be displayed. With this setting, it is not necessary to request the product sales platform to send product pop-ups every time a product is displayed, which can reduce network usage and improve the smoothness of the live stream.

[0115] In an optional embodiment, the pre-generated digital human live-streaming audio and video are played in a loop during the digital human live-streaming process described in the above embodiments; the steps in the above embodiments, when the live-streaming device starts displaying the target product, include sending the product pop-up window of the target product to the live-streaming device through the second interface, specifically including the following steps:

[0116] Based on the playback duration of the digital human live audio and video and the display time of each product in the digital human live audio and video, determine whether the live streaming device should start displaying the target product; if the live streaming device starts displaying the target product, send the product pop-up window of the target product to the live streaming device through the second interface.

[0117] Specifically, the display time of each product in the digital human live audio and video can be marked before the live broadcast. For example, if the digital human live audio and video is 10 minutes long, product A can be displayed from the 2nd to the 5th minute, product B from the 5th to the 7th minute, and product C from the 7th to the 10th minute.

[0118] Since the audio and video of the digital human live stream are played in a loop, it is possible to infer whether the live streaming device has started displaying the target product based on the playback duration of the audio and video and the display time of each product in the digital human live stream audio and video.

[0119] It should be noted that the playback duration of the live audio and video in this embodiment refers to the actual playback duration of the live audio and video, and does not include the duration of the digital human response audio and video containing interactive information. Since the digital human response audio and video is generated by the processing device, the processing device can save the total duration of the digital human response audio and video. Taking the start time of the digital human live broadcast as the base time, the total duration of the inserted digital human response audio and video is subtracted from the duration between the current time and the base time to obtain the playback duration of the live audio and video.

[0120] Based on the playback duration of the live audio and video, and the display time of each product in the digital human live audio and video, it can be inferred whether the live streaming device has started displaying the target product. As described in the above embodiment, if the target product is product A, and the playback duration of the live audio and video is 12 minutes, it can be determined that the digital human live stream has started displaying product A.

[0121] When the live streaming device starts displaying the target product, the product pop-up window is sent to the live streaming device via a second interface. The live streaming device then outputs the target product pop-up window at a set frequency while displaying the target product. The set frequency can be adjusted according to actual conditions, and this embodiment does not impose any limitations.

[0122] Corresponding to the aforementioned application function implementation method embodiments, this application also provides a control device, processing equipment, control system for digital human live streaming, computer-readable storage medium, computer program product, and corresponding embodiments.

[0123] Figure 2 This is a schematic diagram of the structure of the control device for live streaming of digital humans, as shown in an embodiment of this application.

[0124] See Figure 2 The device includes:

[0125] The acquisition module 100 is used to acquire Cookie information; the Cookie information is extracted from the browser by the browser plugin after detecting that the user logs in to the live broadcast control panel page through the browser.

[0126] The control module 110 is used to process audience interaction information during the live streaming of the digital human based on cookie information, and / or control the display of product pop-ups during the live streaming of the digital human.

[0127] Furthermore, in the above embodiments, the control module 110, when processing audience interaction information during the digital human live stream, is specifically used for:

[0128] In response to processing instructions from the live streaming device, the system captures audience interaction information during the digital human live stream and sends the interaction information to the live streaming device via a first interface, so that the live streaming device generates and outputs a first response text to the interaction information.

[0129] Furthermore, in the above embodiments, the control module 110, when processing audience interaction information during the digital human live stream, is specifically used for:

[0130] In response to processing instructions from the live streaming device, the system captures audience interaction information during the live streaming of the digital human and sends the interaction information to the live streaming device via a first interface, so that the live streaming device generates a second response text to respond to the interaction information; the system obtains the second response text, generates digital human response audio and video based on the second response text, and sends the digital human response audio and video to the live streaming device, so that the live streaming device outputs digital human response audio and video.

[0131] Furthermore, in the above embodiment, when controlling the display of product pop-ups during the digital human live stream, the control module 110 is specifically used for:

[0132] In response to a pop-up command for a target product from the live streaming device, the device retrieves the product pop-up from the product sales platform based on the product's identification identifier. The target product includes any product that needs to be displayed during the digital human live stream. The pop-up command for the target product is generated by the live streaming device when displaying the target product. The product pop-up is sent to the live streaming device via a second interface so that the live streaming device can output the product pop-up at a set frequency when displaying the target product.

[0133] Furthermore, the apparatus of the above embodiments further includes:

[0134] The communication module is used to control the display of product pop-ups during the digital human live stream. Before the product pop-ups are displayed, it retrieves all product pop-ups from the product sales platform based on the product identification tags. The products include those that need to be displayed during the digital human live stream.

[0135] When controlling the display of product pop-ups during the live streaming of the digital human, the control module 110 in the above embodiments is specifically used for:

[0136] When the live streaming device starts displaying the target product, the product pop-up window of the target product is sent to the live streaming device through the second interface, so that the live streaming device can output the product pop-up window of the target product at a set frequency when displaying the target product; the target product includes any product that needs to be displayed during the digital human live streaming process.

[0137] In the above embodiments, the pre-generated digital human live-streaming audio and video are played in a loop during the digital human live-streaming process. When the control module 110 in the above embodiments detects that the live-streaming device has started displaying the target product, it sends the product pop-up window of the target product to the live-streaming device through the second interface. Specifically, this is used for:

[0138] Based on the playback duration of the digital human live audio and video and the display time of each product in the digital human live audio and video, determine whether the live streaming device should start displaying the target product; if the live streaming device starts displaying the target product, send the product pop-up window of the target product to the live streaming device through the second interface.

[0139] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0140] Figure 3 This is a schematic diagram of the processing device shown in the embodiments of this application.

[0141] See Figure 3 The processing device includes a memory 200 and a processor 210;

[0142] The memory 200 is connected to the processor 210 and is used to store programs;

[0143] Processor 210 is used to implement some or all of the methods described above by running programs stored in memory 200.

[0144] Specifically, the aforementioned processing device may further include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0145] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0146] A bus can include a pathway for transmitting information between various components of a computer system.

[0147] The processor 210 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0148] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0149] Memory 200 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 210 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices employ mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 200 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 200 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0150] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0151] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0152] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0153] The processor 210 executes the program stored in the memory 200 and calls other devices, and can be used to implement some or all of the methods described above.

[0154] Another embodiment of this application also proposes a control system for live streaming of digital humans. See [link to relevant documentation]. Figure 4 As shown, the control system for this digital human live stream includes:

[0155] Live streaming equipment 300, browser plugin 310, and processing equipment 320;

[0156] The processing device 320 is communicatively connected to the browser plugin 310 and the live streaming device 300, respectively.

[0157] Browser plugin 310 is used to extract cookie information from the browser and send the cookie information to the processing device 320 after detecting that a user has logged into the live broadcast control console page through a browser.

[0158] Live streaming device 300, used for live streaming of digital humans by playing live audio and video of digital humans;

[0159] Processing device 320 is used to process audience interaction information during the live stream of the digital human based on cookie information, and / or control the display of product pop-ups during the live stream of the digital human.

[0160] The control system for live streaming of digital humans provided in this embodiment belongs to the same application concept as the control method for live streaming of digital humans provided in the above embodiments of this application. It can execute the control method for live streaming of digital humans provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the control method for live streaming of digital humans. For technical details not described in detail in this embodiment, please refer to the specific processing content of the control method for live streaming of digital humans provided in the above embodiments of this application, which will not be repeated here.

[0161] Furthermore, the method according to this application can also be implemented as a computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above. Optionally, the computer program can be stored on a readable storage medium of a computer device or in the cloud; the processor of the computer device reads the computer program from the readable storage medium or the cloud.

[0162] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0163] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0164] Alternatively, this application may also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) that, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0165] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0166] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A control method for live streaming of a digital human, characterized in that, Applied to a processing device, the method includes: Obtain Cookie information; wherein, the Cookie information is extracted from the browser by the browser plugin after detecting that the user logs into the live broadcast control panel page through the browser; Based on the cookie information, process the audience's interaction information during the digital human live stream, and / or control the display of product pop-ups during the digital human live stream; The control of product pop-up display during the digital human live stream includes: responding to a pop-up command for a target product from the live stream device, obtaining the product pop-up of the target product from the product sales platform based on the identification identifier of the target product; the target product includes any product that needs to be displayed during the digital human live stream, and the pop-up command for the target product is generated by the live stream device when displaying the target product; sending the product pop-up of the target product to the live stream device via a second interface, so that the live stream device can output the product pop-up of the target product at a set frequency when displaying the target product; The interactive information includes first interactive information and second interactive information; the processing of audience interactive information during the digital human live stream includes: responding to processing instructions from the live streaming device, capturing audience interactive information during the digital human live stream, and sending the interactive information to the live streaming device based on a first interface; the first interactive information is used to trigger the live streaming device to generate and output a first response text in response to the interactive information; the second interactive information is used to trigger the live streaming device to generate and output a second response text in response to the interactive information, obtaining the second response text, generating digital human response audio and video based on the second response text, and sending the digital human response audio and video to the live streaming device so that the live streaming device outputs digital human response audio and video; the first interactive information includes entering the live stream room, liking, and adding to cart; the second interactive information includes commenting.

2. The method according to claim 1, characterized in that, During the digital human live stream, pre-generated digital human live stream audio and video are played in a loop; the step of sending a product pop-up window of the target product to the live stream device through a second interface when the live stream device is detected to start displaying the target product includes: Based on the playback duration of the digital human live audio and video and the display time of each product in the digital human live audio and video, determine whether the live streaming device should start displaying the target product; If the live streaming device starts displaying the target product, the product pop-up window of the target product is sent to the live streaming device through the second interface.

3. A control device for live streaming of digital humans, characterized in that, Applied to a processing device, the apparatus includes: The acquisition module is used to acquire Cookie information; wherein, the Cookie information is extracted from the browser by the browser plugin after detecting that the user logs into the live broadcast control console page through the browser; The control module is used to process the audience's interactive information during the digital human live stream based on the cookie information, and / or control the display of product pop-ups during the digital human live stream; The control of product pop-up display during the digital human live stream includes: responding to a pop-up command for a target product from the live stream device, obtaining the product pop-up of the target product from the product sales platform based on the identification identifier of the target product; the target product includes any product that needs to be displayed during the digital human live stream, and the pop-up command for the target product is generated by the live stream device when displaying the target product; sending the product pop-up of the target product to the live stream device via a second interface, so that the live stream device can output the product pop-up of the target product at a set frequency when displaying the target product; The interactive information includes first interactive information and second interactive information; the processing of audience interactive information during the digital human live stream includes: responding to processing instructions from the live streaming device, capturing audience interactive information during the digital human live stream, and sending the interactive information to the live streaming device based on a first interface; the first interactive information is used to trigger the live streaming device to generate and output a first response text in response to the interactive information; the second interactive information is used to trigger the live streaming device to generate and output a second response text in response to the interactive information, obtaining the second response text, generating digital human response audio and video based on the second response text, and sending the digital human response audio and video to the live streaming device so that the live streaming device outputs digital human response audio and video; the first interactive information includes entering the live stream room, liking, and adding to cart; the second interactive information includes commenting.

4. A processing device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-2.

5. A control system for live streaming of digital humans, characterized in that, include: Live streaming equipment, browser plugins, and processing equipment; The processing device is communicatively connected to the browser plugin and the live streaming device, respectively. The browser plugin is used to extract cookie information from the browser and send the cookie information to the processing device after detecting that a user has logged into the live broadcast control console page through a browser. The live streaming device is used to conduct digital human live streaming by playing digital human live streaming audio and video; The processing device is used to process audience interaction information during the digital human live stream based on the cookie information, and / or control the display of product pop-ups during the digital human live stream; The control of product pop-up display during the digital human live stream includes: responding to a pop-up command for a target product from the live stream device, obtaining the product pop-up of the target product from the product sales platform based on the identification identifier of the target product; the target product includes any product that needs to be displayed during the digital human live stream, and the pop-up command for the target product is generated by the live stream device when displaying the target product; sending the product pop-up of the target product to the live stream device via a second interface, so that the live stream device can output the product pop-up of the target product at a set frequency when displaying the target product; The interactive information includes first interactive information and second interactive information; the processing of audience interactive information during the digital human live stream includes: responding to processing instructions from the live streaming device, capturing audience interactive information during the digital human live stream, and sending the interactive information to the live streaming device based on a first interface; the first interactive information is used to trigger the live streaming device to generate and output a first response text in response to the interactive information; the second interactive information is used to trigger the live streaming device to generate and output a second response text in response to the interactive information, obtaining the second response text, generating digital human response audio and video based on the second response text, and sending the digital human response audio and video to the live streaming device so that the live streaming device outputs digital human response audio and video; the first interactive information includes entering the live stream room, liking, and adding to cart; the second interactive information includes commenting.

6. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, implement the method described in any one of claims 1-2.

Images