Display device and voice interaction method

Abstract

The invention discloses a display device and a voice interaction method, and relates to the technical field of artificial intelligence, the method comprises the steps that a display is controlled to display a media asset detail page associated with a currently played media asset, and a digital human interaction control is displayed in the media asset detail page; in response to a trigger operation for the digital human interaction control, controlling the display to display a digital human card of a candidate digital human associated with the currently played media asset; after a digital person card of a first digital person is selected, receiving an input user interaction voice, acquiring a first reply voice of the first digital person for the user interaction voice from a server according to the user interaction voice, and playing the first reply voice; wherein the timbre in the first reply voice is the timbre of the character role name corresponding to the first digital person, and personalization and interactivity of man-machine interaction are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a display device and a voice interaction method. Background Art

[0002] In the application scenario where users watch video media resources, in order to improve the user experience and interest in watching videos, usually during the video playback process, a human-computer interaction function is supported to enable human-computer interaction based on the played video media resources.

[0003] In the related art, usually users can only passively receive interactive recommendation content at the preset playback moment of the video, resulting in problems such as a single interaction method, lack of personalization and interactivity. Summary of the Invention

[0004] This application provides a display device and a voice interaction method, which can improve the personalization and interactivity of human-computer interaction during video playback.

[0005] In a first aspect, a voice interaction method is provided, which is applied to a display device and includes:

[0006] Controlling the display to display a media details page associated with the currently played media; wherein, a digital human interaction control is displayed on the media details page;

[0007] In response to a trigger operation on the digital human interaction control, controlling the display to display a digital human card of a candidate digital human associated with the currently played media; wherein, each digital human card of the candidate digital human includes a digital human image corresponding to the candidate digital human and / or the name of the character role in the currently played media represented by the candidate digital human;

[0008] After the digital human card of the first digital human is selected, receiving an input user interaction voice, wherein the first digital human is one of the candidate digital humans;

[0009] According to the user interaction voice, obtaining a first reply voice of the first digital human for the user interaction voice from a server and playing the first reply voice; wherein, the timbre in the first reply voice is the timbre of the name of the character role corresponding to the first digital human.

[0010] In the above embodiments, by displaying the media details page associated with the currently played media, and the digital human interaction control is displayed on the media details page, a bridge is provided for users to interact with the digital human. Furthermore, during the video playback process, instead of passively receiving interactive recommendation content, users can actively interact with the digital human through the digital human interaction control on the media details page, improving the initiative of users to interact with the digital human. Further, in response to a trigger operation on the digital human interaction control, by displaying the digital human card of the candidate digital human associated with the currently played media, since different candidate digital humans represent different character roles in the currently played media, subsequent voice interactions are made more relevant to the currently played media and a basis for subsequent personalized voice interactions is provided. In addition, after the digital human card of the first digital human is selected, the input user interaction voice can be received, and according to the user interaction voice, the first reply voice of the first digital human to the user interaction voice is obtained from the server, and the first reply voice with the voice color of the character name corresponding to the first digital human is played, improving the interest of the interaction between the digital human and the user.

[0011] In some embodiments, when the controller executes to control the display to display the media details page associated with the currently played media, it is configured to:

[0012] In the case where the user interaction voice does not contain digital human association information, according to the user interaction voice, obtain the first reply voice of the first digital human to the user interaction voice from the server, and play the first reply voice;

[0013] In the case where the user interaction voice contains digital human association information, if the second digital human represented by the digital human association information is different from the first digital human, focus the screen focus on the digital human card corresponding to the second digital human, and according to the user interaction voice, obtain the second reply voice of the second digital human to the user interaction voice from the server, and play the second reply voice; wherein, the voice color in the second reply voice is the voice color of the character name corresponding to the second digital human.

[0014] In the above embodiments, when inputting the user interaction voice, digital human association information can be carried to personalized select the digital human for voice reply, making the process of the user interacting with the digital human more personalized and improving the initiative of the user to interact with the digital human.

[0015] In some embodiments, the obtaining the first reply voice of the first digital human to the user interaction voice from the server according to the user interaction voice includes:

[0016] Convert the user interaction voice into user interaction text;

[0017] Send a first inquiry request to the server, carrying the user interaction text and the digital human identification of the first digital human;

[0018] Receive the first reply voice feedback by the server based on the first inquiry request; wherein, the first reply voice is determined by the server based on the digital human identification of the first digital human and calling the first digital human according to the user interaction text.

[0019] In the above embodiment, when obtaining the first reply voice, a first inquiry request carrying the user interaction text and the digital human identification of the first digital human is sent to the server, so that the server generates the first reply voice based on the user interaction text and the digital human identification of the first digital human. The timbre in the first reply voice is the timbre of the character name corresponding to the first digital human, making the first reply voice more in line with the digital human image, thereby improving the interest of interaction with the user.

[0020] In some embodiments, a voice service application and a desktop application for playing the currently played media asset are integrated in the display device;

[0021] Before receiving the input user interaction voice, it further includes:

[0022] The desktop application sends a user wake-up voice and current scene information to the voice service application; wherein, the user wake-up voice is used to wake up the voice service application;

[0023] The conversion of the user interaction voice into a user interaction text includes:

[0024] The desktop application sends the user interaction voice to the voice service application;

[0025] After the voice service application recognizes that the current scene information is a role interaction scene, it converts the user interaction voice into a user interaction text;

[0026] After the voice service application recognizes that the current scene information is a non-role interaction scene, it obtains a third reply voice corresponding to the user interaction voice and plays the third reply voice.

[0027] In the above embodiment, a voice service application and a desktop application for playing the currently played media asset are integrated in the display device. The desktop application can send a user wake-up voice and current scene information to the voice service application, so that the voice service application can determine whether it is a role interaction scene according to the current scene information, and further facilitate determining the way to play the reply voice corresponding to the user interaction voice. On the one hand, it broadens the functions of the voice service; on the other hand, it realizes the distinction between the role interaction scene and the non-role interaction scene, making the interaction method with the user more in line with the user's interaction needs, thereby improving the user interaction experience.

[0028] In some embodiments, the playing the first reply voice includes:

[0029] The desktop application calls a multimedia player to play the first reply voice.

[0030] In the above embodiment, a desktop application is used to call a multimedia player to play the first reply voice, which, on the one hand, improves the fun of interaction with the user, and on the other hand, reduces development costs; and multimedia players usually support multiple audio formats, reducing playback problems caused by format incompatibility and improving the stability and reliability of the first reply voice playback.

[0031] In some embodiments, after playing the first reply voice, the method further includes:

[0032] After the voice service application detects that there is no voice input for a period exceeding a preset time, the voice service application sends a return prompt message to the desktop application;

[0033] The desktop application obtains the prompt voice of the third digital human from the server based on the return prompt information, and calls a multimedia player to play the prompt voice.

[0034] In the above embodiment, after the voice service application detects that there is no voice input for more than a preset time, it sends a return prompt message to the desktop application, so that the desktop application can obtain the prompt voice of the third digital human from the server based on the return prompt message, and call the multimedia player to play the prompt voice to attract the user to continue watching the media or continue interacting with the digital human, thereby increasing the user's enthusiasm for watching the media and interacting with the digital human.

[0035] In some embodiments, the desktop application obtains the prompt voice of the third digital human from the server based on the return prompt information, and calls a multimedia player to play the prompt voice, including:

[0036] The desktop application sends a prompt acquisition request to the server based on the return prompt information; wherein the prompt acquisition request carries the media asset identifier of the currently playing media asset and the currently playing frame image;

[0037] The desktop application obtains the prompt voice of the third digital person fed back by the server; wherein the third digital person is selected from the candidate digital persons; the prompt voice is determined by the server based on the digital person identifier of the third digital person, calling the third digital person according to the media resource identifier and the current playing frame image;

[0038] The desktop application plays the prompt voice when the third digital human is the same as the candidate digital human corresponding to the digital human card where the screen focus is located;

[0039] When the third digital human is different from the candidate digital human corresponding to the digital human card where the screen focus is located, the desktop application focuses the screen focus on the digital human card corresponding to the third digital human and plays the prompt voice.

[0040] In the above embodiments, a prompt voice is generated according to the digital human identifier of the third digital human, the media identifier of the currently played media asset, and the currently played frame image. On the one hand, the generated prompt voice is more in line with the current plot and the subsequent plot of the currently played media asset, which can improve the user's enthusiasm for continuing to watch the media asset; on the other hand, playing the prompt voice with the timbre of the character name corresponding to the third digital human not only realizes switching digital humans according to the screen focus, but also makes the played voice more matching the image of the switched digital human.

[0041] In some embodiments, controlling the display to display the media details page associated with the currently played media asset includes:

[0042] Sending a request for obtaining the details page for the currently played media asset to the server;

[0043] Receiving the details page data feedback by the server based on the details page acquisition request;

[0044] When the details page data contains digital human information, generating a media details page including the digital human interaction control according to the details page data, and controlling the display to display the media details page.

[0045] In the above embodiments, when there is digital human information in the details page data of the currently played media asset, a media details page including digital human interaction controls can be generated, providing a bridge for the user to interact with the digital human, facilitating the user to actively interact with the digital human, and improving the initiative of the user to interact with the digital human.

[0046] In some embodiments, controlling the display to display the digital human cards of the candidate digital humans associated with the currently played media asset includes:

[0047] Controlling the display to display the digital human cards of the candidate digital humans associated with the currently played media asset in the digital human display order;

[0048] After controlling the display to display the digital human cards of the candidate digital humans associated with the currently played media asset, it further includes:

[0049] After the digital human card of the first digital human is selected, obtain the interaction guiding speech of the first digital human and play the interaction guiding speech of the first digital human.

[0050] In the above embodiment, on the one hand, according to the display order of digital humans, displaying the digital human cards of candidate digital humans associated with the currently played media can make the digital human displayed earlier more likely to meet the user's interaction needs; on the other hand, after the digital human card of the first digital human is selected, playing the interaction guiding speech of the first digital human can improve the enthusiasm of user interaction.

[0051] In a second aspect, a display device is provided, including:

[0052] A display and a controller;

[0053] The controller is configured to:

[0054] Control the display to display the media details page associated with the currently played media; wherein, a digital human interaction control is displayed on the media details page;

[0055] In response to a trigger operation on the digital human interaction control, control the display to display the digital human cards of candidate digital humans associated with the currently played media; wherein, each digital human card of the candidate digital humans includes the digital human image corresponding to the candidate digital human and / or the name of the character in the currently played media represented by the candidate digital human;

[0056] After the digital human card of the first digital human is selected, receive the input user interaction voice, where the first digital human is one of the candidate digital humans;

[0057] According to the user interaction voice, obtain the first reply voice of the first digital human for the user interaction voice from the server and play the first reply voice; wherein, the timbre in the first reply voice is the timbre of the name of the character corresponding to the first digital human.

[0058] In the above embodiment, by displaying the media resource details page associated with the currently playing media resource, the digital human interaction control is displayed in the media resource details page, which provides a bridge for the user to interact with the digital human. In the process of video playback, the user no longer passively accepts the interactive recommendation content, but can actively interact with the digital human through the digital human interaction control in the media resource details page, thereby improving the initiative of the user to interact with the digital human. Further, in response to the trigger operation on the digital human interaction control, by displaying the digital human card of the candidate digital human associated with the currently playing media resource, since different candidate digital human represents different characters in the currently playing media resource, the subsequent voice interaction is more suitable for the currently playing media resource, and a basis for subsequent personalized voice interaction is provided. In addition, after the digital human card of the first digital human is selected, the input user interaction voice can be received, and according to the user interaction voice, the first reply voice of the first digital human to the user interaction voice is obtained from the server, and the first reply voice with the timbre of the character name corresponding to the first digital human is played, thereby improving the fun of the interaction between the digital human and the user.

[0059] In a third aspect, a voice interaction device is provided, which is configured on a display device, including:

[0060] A first control unit is used to control the display to display a media resource detail page associated with the currently playing media resource; wherein the media resource detail page displays a digital human interactive control;

[0061] A second control unit is used to control the display to display digital human cards of candidate digital humans associated with the currently playing media in response to a trigger operation on the digital human interactive control; wherein each digital human card of the candidate digital human contains a digital human image corresponding to the candidate digital human and / or a character name in the currently playing media represented by the candidate digital human;

[0062] A receiving unit, configured to receive an input user interaction voice after a digital person card of a first digital person is selected, wherein the first digital person is one of the candidate digital persons;

[0063] The playing unit is used to obtain a first reply voice of the first digital human to the user interaction voice from the server according to the user interaction voice, and play the first reply voice; wherein the timbre in the first reply voice is the timbre of the character name corresponding to the first digital human.

[0064] In the above embodiments, by displaying the media details page associated with the currently played media, and the digital human interaction control is displayed on the media details page, which provides a bridge for users to interact with the digital human. Furthermore, during the video playback process, instead of passively receiving interactive recommendation content, users can actively interact with the digital human through the digital human interaction control on the media details page, improving the initiative of users to interact with the digital human. Further, in response to a trigger operation on the digital human interaction control, by displaying the digital human card of the candidate digital human associated with the currently played media, since different candidate digital humans represent different character roles in the currently played media, subsequent voice interactions are more in line with the currently played media and provide a basis for subsequent personalized voice interactions. In addition, after the digital human card of the first digital human is selected, the input user interaction voice can be received, and according to the user interaction voice, the first reply voice of the first digital human for the user interaction voice can be obtained from the server, and the first reply voice with the voice color of the character name corresponding to the first digital human is played, improving the interest of the interaction between the digital human and the user.

[0065] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is run by a processor, any one of the voice interaction methods in the first aspect is executed.

[0066] In a fifth aspect, a computer program product is provided. The computer program product includes: a computer program, and when the computer program is run by a processor, any one of the voice interaction methods in the first aspect is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0068] Figure 1 FIG. shows a schematic diagram of an operation scenario between a display device and a control device provided by some embodiments of the present application;

[0069] Figure 2 Exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment;

[0070] Figure 3 Exemplarily shows a configuration block diagram of the display device 200 according to an exemplary embodiment;

[0071] Figure 4 Exemplarily shows a system block diagram in the display device 200 according to an exemplary embodiment;

[0072] Figure 5 Shows a schematic flow chart of a voice interaction method provided by some embodiments of the present application;

[0073] Figure 6 Shows a schematic diagram of a digital human control provided by some embodiments of the present application;

[0074] Figure 7 Shows a schematic diagram of a digital human card provided by some embodiments of the present application;

[0075] Figure 8 Shows a schematic flow chart of a process for displaying media asset details provided by some embodiments of the present application;

[0076] Figure 9 Shows a schematic flow chart of a process for obtaining a first response voice provided by some embodiments of the present application;

[0077] Figure 10 Shows a schematic flow chart of a method for processing a user interaction voice by a voice service application provided by some embodiments of the present application;

[0078] Figure 11 Shows a schematic flow chart of a return prompt method provided by some embodiments of the present application;

[0079] Figure 12 Shows a schematic flow chart of a process for playing a prompt voice of a third digital human provided by some embodiments of the present application;

[0080] Figure 13 Shows a timing diagram of training a digital human model provided by some embodiments of the present application;

[0081] Figure 14 Shows a timing diagram of a voice interaction provided by some embodiments of the present application;

[0082] Figure 15 Shows a schematic diagram of a voice interaction device provided by some embodiments of the present application. Detailed implementation manners

[0083] Next, the technical solutions in the embodiments of the present application will be described in conjunction with the accompanying drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B; "and / or" herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality" means two or more than two.

[0084] Hereinafter, the terms "first", "second", and "third" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", and "third" may explicitly or implicitly include one or more of such features.

[0085] For illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from hindering the description of the present application.

[0086] The display device provided by the embodiments of the present application can have various implementation forms. For example, it can be a television, a smart TV, a laser projection device, a monitor, an electronic bulletin board, etc. Figure 1 and Figure 2 is a specific implementation manner of the display device of the present application.

[0087] Figure 1 is a schematic diagram of the operation scenario between the display device and the control device according to the embodiment. As Figure 1 shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0088] In some embodiments, the control device 100 can be a remote control. The communication between the remote control and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 wirelessly or wiredly. The user can input user instructions through the buttons on the remote control, voice input, control panel input, etc. to control the display device 200.

[0089] In some embodiments, the smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) can also be used to control the display device 200. For example, an application program running on the smart device is used to control the display device 200.

[0090] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, it can directly receive the user's voice instructions through a module for obtaining voice instructions configured inside the display device 200, or receive the user's voice instructions through a voice control device set outside the display device 200.

[0091] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various contents and interactions to the display device 200. The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0092] Figure 2 An exemplary configuration block diagram of the control device 100 according to an exemplary embodiment is shown. As Figure 2 shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive operation instructions input by a user and convert the operation instructions into instructions recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

[0093] As Figure 3 , the display device 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0094] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to n interfaces for input / output.

[0095] The display 260 includes a display screen component for presenting a picture and a driving component for driving image display, which is a component for receiving an image signal output from the controller and displaying video content, image content, and a menu control interface, as well as a user control UI interface.

[0096] The display 260 can be a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.

[0097] The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator can include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish the sending and receiving of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0098] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0099] The detector 230 is used to collect signals from the external environment or for external interactions. For example, the detector 230 includes an optical receiver, a sensor for collecting the ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0100] The external device interface 240 includes at least one High Definition Multimedia Interface (HDMI), and may also include, but is not limited to, the following: any one or more of an analog or digital component input interface (component), a composite video input interface (CVBS), an RGB port, etc., a DisplayPort (DP), a USB interface (such as USB Type-C). It can also be a composite input / output interface formed by the above multiple interfaces.

[0101] Among them, HDMI is a digital video / audio interface that can transmit audio and video signals simultaneously, without the need for additional audio connection cables, simplifies the installation difficulty of the lines, and ensures high-quality audio and video signal transmission. And with the development of HDMI technology, the bandwidth of HDMI has been continuously improved, with a long transmission bandwidth, which can support high-definition, ultra-high-definition, and even higher-resolution video transmissions, and can meet the growing demand for high-definition video.

[0102] It should be understood that various external devices such as game consoles, Blu-ray players, set-top boxes, etc. can be connected to the display device 200 through HDMI, enabling users to obtain a richer and more immersive audio-visual entertainment experience.

[0103] It should also be understood that in order to facilitate the distinction and management of HDMI interfaces, the display device and the system of the display device will label the HDMI interfaces, and different HDMI interfaces can be represented by numbers, letters, or specific symbols. For example, HDMI1, HDMI2, or HDMI A, HDMI B, etc.

[0104] In the system of the display device, each HDMI interface has its own independent interface code, which can be identified by a digital code, and there is a corresponding relationship with the label of the HDMI interface. For example, HDMI 1 corresponds to the number 1281, and HDMI 2 corresponds to the number 1282.

[0105] The tuner-demodulator 210 receives broadcast television signals through wired or wireless reception methods, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0106] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different split devices, that is, the tuner-demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0107] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0108] In some embodiments, the controller includes at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a Random Access Memory (RAM), a Read-Only Memory (ROM), a first interface to an nth interface for input / output, a communication Bus, etc.

[0109] The user can input a user command on the graphical user interface (GUI) that can be displayed on the display 260, and then the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and then the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0110] The "user interface" is a media interface for interaction and information exchange between an application program or an operating system and a user, and it realizes the conversion between the internal form of information and the form acceptable to the user. The common manifestation form of the user interface is the graphical user interface (Graphic User Interface, GUI), which refers to the user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, a window, a control, etc. displayed on the display screen of an electronic device, and the control can include visible interface elements such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

[0111] Such as Figure 4, in some embodiments, the system is divided into four layers, from top to bottom are the Applications layer (referred to as the "application layer" for short), the Application Framework layer (referred to as the "framework layer" for short), the Android runtime and the system libraries layer (referred to as the "system runtime layer" for short), and the kernel layer.

[0112] In some embodiments, at least one application runs in the application layer. These applications can be the window programs, system setting programs, or clock programs etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementation, the application packages in the application layer are not limited to the above examples.

[0113] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, which decides the actions of the applications in the application layer. Applications can access the resources in the system and obtain the services of the system through the API interfaces during execution.

[0114] As Figure 4 shown, in the embodiment of the present application, the application framework layer includes Managers, Content Provider, etc. Among them, the Managers include at least one of the following modules: ActivityManager is used to interact with all the activities running in the system; Location Manager is used to provide access to the system location service for system services or applications; Package Manager is used to retrieve various information related to the application packages currently installed on the device; NotificationManager is used to control the display and clearing of notification messages; Window Manager is used to manage the icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0115] In some embodiments, the ActivityManager is used to manage the life cycles of various applications and the usual navigation back functions, such as controlling the exit, opening, and backward of applications. The Window Manager is used to manage all window programs, such as obtaining the display screen size, judging whether there is a status bar, locking the screen, taking screenshots, and controlling the changes of the display window (such as shrinking the display window, jittering display, distorting display, etc.).

[0116] In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries included in the system runtime layer to implement the functions that the framework layer needs to achieve.

[0117] In some embodiments, the kernel layer is the layer between hardware and software. As Figure 4 shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0118] It should be noted that the above examples are only simple divisions of the functions of the operating system, and do not limit the specific form of the operating system of the display device 200 in the embodiments of the present application. According to factors such as the functions of the display device and the type of the operating system, the number of levels included in the operating system and the specific level types can be in other forms.

[0119] Exemplarily, with the development of artificial intelligence technology, a display device no longer only has functions such as displaying images and playing videos, but can support human-computer interaction functions during the process of displaying images or playing videos, so as to improve the user experience and interest in watching images or videos.

[0120] Taking the scenario where the display device plays a video as an example, in the related art, usually users can only passively receive interactive recommendation content at the preset playback moment of the video, and there are problems such as a single interaction method, lack of personalization and interactivity.

[0121] Based on this, the embodiments of the present application provide a display device and a voice interaction method. Among them, the voice interaction method is applied to the display device. In the voice interaction method, a controller in the display device controls the display to display a media detail page associated with the currently played media. A digital human interaction control is displayed on the media detail page, which provides a bridge for users to interact with the digital human. During the interaction with the user when playing a video, the user no longer passively receives interactive recommendation content, but can actively interact with the digital human through the digital human interaction control on the media detail page, improving the initiative of the user to interact with the digital human; furthermore, the controller responds to a trigger operation on the digital human interaction control, controls the display to display a digital human card of a candidate digital human associated with the currently played media; and after the digital human card of the first digital human is selected, receives the input user interaction voice, and according to the user interaction voice, obtains a first reply voice of the first digital human for the user interaction voice from the server, and plays the first reply voice with the voice color of the character name corresponding to the first digital human. The first reply voice can not only fit the user interaction voice, but also combine the media content of the currently played media, thereby improving the intelligence and interest of the interaction between the digital human and the user.

[0122] Optionally, refer to Figure 5 , Figure 5 which provides a schematic flowchart of a voice interaction method. Taking the application of this method to a display device as an example for illustration, it can be specifically applied to the controller in the display device. The voice interaction method includes the following steps:

[0123] S501, control the display to show the media details page associated with the currently playing media.

[0124] Exemplarily, the currently playing media can be video media, such as movies, TV dramas, short videos, etc. One or more media playback windows can be displayed in the navigation column of the display device. When the user selects a certain media and clicks on the playback window of the media, the selected media is the currently playing media. Among them, the user can focus on a certain media by operating the control device 100 to select a certain media; or can focus on a certain media by operating the smart device 300 to select a certain media; or can also use the voice assistant of the display device, by inputting voice, for example, voice input "play the first media", to select the first media and trigger the playback window of the media.

[0125] Optionally, the user can focus on a certain media by operating the control device 100 or the smart device 300, and operate the control device 100 or the smart device 300 to click on the playback window of the media. After clicking on the playback window of the media, the display device sends a request to the server to obtain the media details page associated with the currently playing media. The request can carry the media identifier of the currently playing media; the server can obtain the details page data of the currently playing media based on the media identifier of the currently playing media. For example, the details page data of the playing media can be obtained on a third-party platform, or the details page data of the playing media can be obtained by calling the port corresponding to the currently playing media.

[0126] Furthermore, the server can feedback the details page data to the display device. The details page data includes but is not limited to media content introduction, media cover, media-related data, etc. Taking the media as a TV drama as an example, the details page data can include plot summary, TV drama name, cover image, episodes, actor list, rating, digital human information, and related comments, etc. The digital human information can be information such as the character image corresponding to the characters in the TV drama.

[0127] Exemplarily, after the display device receives the detail page data associated with the currently playing media asset, the display device can perform page rendering based on the detail page data associated with the currently playing media asset; after the page rendering is completed, the controller in the display device controls the display to display the media asset detail page associated with the currently playing media asset. Among them, a digital human interaction control is displayed on the media asset detail page. The digital human interaction control is a control that can provide interaction between the user and the digital human associated with the currently playing media asset.

[0128] Exemplarily, a target image can be displayed in the digital human interaction control. The target image can be an independent digital human image or an aggregated digital human image. The independent digital human image can be an image of any character in the currently playing media asset; the aggregated digital human image can be an aggregated image of multiple characters in the currently playing media asset.

[0129] In the case where the target image is an independent digital human image, the independent digital human image can be the digital human image corresponding to the specified digital human among the candidate digital humans. Among them, the specified digital human can be the first candidate digital human in the digital human display order. It can also be the digital human image corresponding to the digital human randomly determined among the candidate digital humans. The candidate digital humans can be a set of all or part of the characters in the currently playing media asset.

[0130] Exemplarily, see Figure 6 , Figure 6 A schematic diagram of a digital human control is provided. Among them, the digital human control can be presented in the form of pictures, text, or a combination of pictures and text on the media asset detail page. For example, taking the currently playing media asset content as an animated film as an example, an image of the digital human can be displayed in the digital human interaction control, and there is a text description of "AI interaction". In this way, the digital human interaction control can be presented intuitively, and the function of the digital human interaction control can be displayed intuitively.

[0131] S502, in response to a trigger operation on the digital human interaction control, control the display to display the digital human card of the candidate digital human associated with the currently playing media asset.

[0132] Exemplarily, the user can trigger the digital human interaction control by manipulating the control device 100 or the smart device 300. The user can also trigger the digital human interaction control by means of far-field voice. For example, the user can voice input "Open the digital human interaction control" to trigger the digital human interaction control.

[0133] After triggering the digital human interaction control, the controller responds to the trigger operation for the digital human interaction control and controls the display to show the digital human cards of the candidate digital humans associated with the currently playing media asset. If there is only one digital human, the digital human card can be directly shown. If there are multiple digital humans, the controller can control the display to show the digital human cards of the candidate digital humans associated with the currently playing media asset in the display order of the digital humans, or can also show the digital human cards of the candidate digital humans associated with the currently playing media asset in a randomly displayed order.

[0134] Among them, the digital human card can be generated according to the digital human information; the digital human information can include the digital human image of the candidate digital human associated with the currently playing media asset and the name of the character in the currently playing media asset represented by it, etc. Each digital human card of a candidate digital human contains the digital human image corresponding to the candidate digital human or the name of the character in the currently playing media asset represented by the candidate digital human, or each digital human card of a candidate digital human contains the digital human image corresponding to the candidate digital human and the name of the character in the currently playing media asset represented by the candidate digital human. Taking the currently playing media asset as an animated cartoon as an example, assuming that the characters in the currently playing media asset include Qiqi, Miaomiao, and Tutu, then the digital human card of the digital human corresponding to Qiqi can include Qiqi's character image and Qiqi's character name, and can also include the introduction or greeting words corresponding to Qiqi, etc.

[0135] Exemplarily, refer to Figure 7 , Figure 7 A schematic diagram of a digital human card is provided. Taking the digital humans as Qiqi, Miaomiao, and Tutu as an example for illustration. Among them, each digital human card can include the digital human image corresponding to the candidate digital human and the name of the character in the currently playing media asset represented by the candidate digital human, and can also include the introduction words and display serial number of the digital human, etc.

[0136] Figure 7 It shows that the display order of the digital humans is Qiqi, Miaomiao, Tutu. The introduction words contained in the digital human card corresponding to Qiqi can be "I am Qiqi from the panda family and I am 7 years old this year"; the introduction words contained in the digital human card corresponding to Miaomiao can be "I am Miaomiao from the panda family and I am 6 years old this year"; the introduction words contained in the digital human card corresponding to Tutu can be "I am Tutu from the panda family and I am 5 years old this year". The introduction words contained in the above digital human cards are only exemplary descriptions and are not used to limit the introduction words contained in the digital human cards. The introduction words contained in the digital human cards can be set according to actual needs.

[0137] S503, after the digital human card of the first digital human is selected, receive the input user interaction voice.

[0138] Exemplarily, the first digital human is the selected digital human among the candidate digital humans, that is, the first digital human can be one of the candidate digital humans. Among them, the candidate digital humans can be a set of all or part of the characters in the currently played media asset.

[0139] The user can select the card of the first digital human by operating the control device 100 or the smart device 300. For example, by operating the control device 100 or the smart device 300, the focus can be placed on the card of the first digital human to realize the selection operation of the first digital human. The first digital human can also be selected by voice input. For example, the user can input the name of the character corresponding to the first digital human by voice to select the first digital human.

[0140] Exemplarily, after the digital human card of the first digital human is selected, the user can input user interaction voice. For example, the user's interaction voice can be "What's the weather like today"; the controller receives the user interaction voice input by the user.

[0141] S504, according to the user interaction voice, obtain the first reply voice of the first digital human for the user interaction voice from the server, and play the first reply voice.

[0142] Furthermore, the user interaction voice and the digital human identifier of the first digital human can be sent to the server, so that the server generates the first reply voice based on the user interaction voice and the digital human identifier, and feeds back the first reply voice to the display device, so that the display device plays the first reply voice. Among them, the timbre in the first reply voice is the timbre of the name of the character corresponding to the first digital human.

[0143] Optionally, while playing the first reply voice, the text corresponding to the first reply voice can also be visually displayed to improve the user's viewing experience and interaction experience.

[0144] In the above embodiments, by displaying the media details page associated with the currently playing media, and the digital human interaction control is displayed on the media details page, a bridge is provided for the user to interact with the digital human. Furthermore, during the video playback process, the user is no longer passively receiving interactive recommendation content, but can actively interact with the digital human through the digital human interaction control on the media details page, improving the initiative of the user to interact with the digital human; further, in response to a trigger operation on the digital human interaction control, by displaying the digital human card of the candidate digital human associated with the currently playing media, since different candidate digital humans represent different character roles in the currently playing media, the subsequent voice interaction is made more suitable for the currently playing media and provides a basis for subsequent personalized voice interaction; in addition, after the digital human card of the first digital human is selected, the input user interaction voice can be received, and according to the user interaction voice, the first reply voice of the first digital human for the user interaction voice is obtained from the server, and the first reply voice with the timbre of the character name corresponding to the first digital human is played, improving the fun of the interaction between the digital human and the user.

[0145] In some alternative implementation manners, when controlling the display device to display the media details page associated with the currently playing media, it may first be determined whether there is digital human information in the details page data of the currently playing media. If there is digital human information, a digital human interaction control may be generated to support the user to interact with the digital human based on the digital human interaction control.

[0146] Based on this, refer to Figure 8 , Figure 8 A flow diagram of displaying the media details page associated with the currently playing media is provided, which specifically includes the following steps:

[0147] S801, send a request for obtaining the details page of the currently playing media to the server.

[0148] Optionally, one or more media playback windows may be displayed in the navigation column of the display device. When the user selects a certain media and clicks on the playback window of the media, the selected media is the currently playing media.

[0149] The user can focus on a certain media by operating the control device 100 or the intelligent device 300, and operate the control device 100 or the intelligent device 300 to click on the playback window of the media; after clicking on the playback window of the media, the display device will send a request to the server to obtain the media details page associated with the currently playing media, and the media identifier of the currently playing media may be carried in the request; the server may obtain the details page data of the currently playing media based on the media identifier of the currently playing media. For example, the details page data of the playback media may be obtained on a third-party platform, or the corresponding port of the currently playing media may be called to obtain the details page data of the playback media.

[0150] S802, the receiving server obtains the detail page data in response to the detail page acquisition request.

[0151] Furthermore, the server feeds back the detail page data to the display device. The display device can receive the detail page data fed back by the server in response to the detail page acquisition request.

[0152] S803, when the detail page data contains digital human information, generate a media asset detail page containing digital human interaction controls according to the detail page data, and control the display to display the media asset detail page.

[0153] Exemplarily, when the detail page data contains digital human information, the controller in the display device can perform page rendering according to the detail page data to generate a media asset detail page containing digital human interaction controls, and the controller in the display device can control the display to display the media asset detail page. At this time, the digital human interaction controls are displayed on the media asset detail page. Among them, the digital human interaction control is a control that can provide interaction between the user and the digital human associated with the currently playing media asset.

[0154] In the above embodiment, when there is digital human information in the detail page data of the currently playing media asset, a media asset detail page containing digital human interaction controls can be generated, which provides a bridge for the user to interact with the digital human, facilitating the user to actively interact with the digital human and improving the initiative of the user to interact with the digital human.

[0155] In some optional implementation manners, when displaying the digital human cards of the candidate digital humans associated with the currently playing media asset, the display can be controlled to display the digital human cards of the candidate digital humans associated with the currently playing media asset in the digital human display order.

[0156] Exemplarily, the digital human display order can be randomly generated by the server or randomly generated by the display device; it can also be determined by the server according to at least one of the popularity information of the candidate digital humans, the historical interaction selection times, and the matching degree between the candidate digital humans and the currently playing media asset, or determined by the display device according to at least one of the popularity information of the candidate digital humans, the historical interaction selection times, and the matching degree between the candidate digital humans and the currently playing media asset.

[0157] Among them, the popularity information of the candidate digital humans can be determined according to the number of topics, discussion frequency, and number of participants related to the candidate digital humans on the social platform, and the candidate digital humans with higher popularity can be placed in the front for display.

[0158] The historical interactive selection count can be understood as the number of times a user interacts with a digital human within a historical period. Here, the historical period can be one month, half a month, etc., and can be specifically set according to actual circumstances. The candidate digital humans with a higher historical interactive selection count can be displayed first.

[0159] The matching degree between a candidate digital human and the currently playing media asset can be understood as the relevance of the candidate digital human to the episodes and plot of the currently playing media asset. The higher the relevance of the candidate digital human, the higher its matching degree with the currently playing media asset. The candidate digital humans with a high matching degree with the currently playing media asset can be displayed first.

[0160] Exemplarily, after controlling the display of the digital human cards of the candidate digital humans associated with the currently playing media asset on the display, after the digital human card of the first digital human is selected, the interactive guiding language of the first digital human can be obtained and played.

[0161] Exemplarily, the user can select the card of the first digital human by operating the control device 100 or the smart device 300. For example, by operating the control device 100 or the smart device 300, the focus can be placed on the card of the first digital human to achieve the selection operation of the first digital human. The first digital human can also be selected by voice input. For example, the user can select the first digital human by voice-inputting the name of the character role corresponding to the first digital human.

[0162] After the digital human card of the first digital human is selected, the interactive guiding language of the first digital human can be obtained and played. Taking the "Qiqi" in Figure 7 as an example, the interactive guiding language of the first digital human can be "Hi, I'm Qiqi. I'm so happy to have learned new knowledge today. Manage my own knowledge by myself. Come and chat with me!".

[0163] Exemplarily, when playing the guiding language of the first digital human, the voice of the character role name corresponding to the first digital human can be used for playing, and the text corresponding to the guiding language of the first digital human can be visually displayed.

[0164] In the above embodiments, on the one hand, by displaying the digital human cards of the candidate digital humans associated with the currently playing media asset in the digital human display order, the digital humans displayed first are more likely to meet the user's interaction needs; on the other hand, after the digital human card of the first digital human is selected, playing the interactive guiding language of the first digital human can improve the enthusiasm of user interaction.

[0165] In some alternative implementations, when playing the first reply voice for user interaction voice, it can be played according to the currently selected digital human or the digital human selected by the user independently.

[0166] Based on this, in the case where the user interaction voice does not contain digital human association information, according to the user interaction voice, the first reply voice of the first digital human for the user interaction voice can be obtained from the server and the first reply voice can be played. Among them, the digital human association information can be the information for selecting a digital human included in the user interaction voice. For example, it can be the name of the character role corresponding to the digital human in the user interaction voice, or the display sequence number of the digital human in the display order, etc. In the case where the user interaction voice does not contain digital human association information, that is, the currently selected first digital human can be used to play the first reply voice.

[0167] In the case where the user interaction voice contains digital human association information, if the second digital human represented by the digital human association information is different from the first digital human, the screen focus is focused on the digital human card corresponding to the second digital human, and according to the user interaction voice, the second reply voice of the second digital human for the user interaction voice is obtained from the server and the second reply voice is played; among them, the timbre in the second reply voice is the timbre of the name of the character role corresponding to the second digital human.

[0168] Exemplarily, in the case where the user interaction voice contains digital human association information, for example, the user interaction voice includes the name of the character role corresponding to the digital human, or the user interaction voice includes information such as the display sequence number of the digital human in the display order, which means that the user interaction voice contains digital human association information. In this case, it can be first determined whether the second digital human represented by the digital human association information is the same as the currently selected first digital human; if the second digital human represented by the digital human association information is different from the first digital human, the screen focus can be focused on the digital human card corresponding to the second digital human, and according to the user interaction voice, the second reply voice of the second digital human for the user interaction voice is obtained from the server and the second reply voice is played; in this way, it can be realized that the timbre in the second reply voice is the timbre of the name of the character role corresponding to the second digital human.

[0169] If the second digital human represented by the digital human association information is the same as the first digital human, the first reply voice of the first digital human for the user interaction voice can be obtained from the server and the first reply voice can be played.

[0170] In the above embodiments, when inputting the user interaction voice, digital human association information can be carried to personalized select the digital human for voice reply, making the process of user interaction with the digital human more personalized and improving the initiative of user interaction with the digital human.

[0171] In some alternative implementation manners, when obtaining the first reply voice of the first digital human for the user interaction voice from the server, the user interaction text corresponding to the user interaction voice and the digital human identifier may be sent to the server, so that the server determines the first reply voice based on the user interaction text and the digital human identifier.

[0172] Based on this, referring to Figure 9 , Figure 9 a flowchart diagram for obtaining the first reply voice is provided, which specifically includes the following steps:

[0173] S901, convert the user interaction voice into a user interaction text.

[0174] Before obtaining the relevant reply voice from the server, the user interaction voice may first be converted into a user interaction text. The conversion of the user interaction voice into a user interaction text can be achieved in various ways. For example, a speech-to-text tool can be used to convert the user interaction voice into a user interaction text; alternatively, the user interaction voice can be input into a speech recognition model to obtain the user interaction text corresponding to the user interaction voice. The above methods are only exemplary descriptions of the ways to convert the user interaction voice into a user interaction text, and are not used to limit the ways to convert the user interaction voice into a user interaction text.

[0175] S902, send a first inquiry request carrying the user interaction text and the digital human identifier of the first digital human to the server.

[0176] Furthermore, a first inquiry request carrying the user interaction text and the digital human identifier of the first digital human can be sent to the server. Among them, the digital human identifier is used to distinguish different digital humans, and the same digital human has a unique digital human identifier. The first inquiry request is used to instruct the server to generate a first reply voice based on the user interaction text and the digital human identifier of the first digital human.

[0177] After receiving the first inquiry request, the server can obtain the corresponding reply text information according to the user interaction text; furthermore, it can find the corresponding digital human according to the digital human identifier of the first digital human, and then call the first digital human to generate a first reply voice according to the user interaction text. For example, the reply text information can first be converted into reply voice data, and according to the voice characteristics of the first digital human, the voice color in the reply voice data can be adjusted to the voice color of the first digital human to obtain the first reply voice.

[0178] S903, receive the first reply voice fed back by the server based on the first inquiry request.

[0179] Among them, the first reply voice is determined by the server based on the digital human identifier of the first digital human and by invoking the first digital human according to the user interaction text.

[0180] The server feeds back the first reply voice to the display device. The display device receives the first reply voice fed back by the server based on the first inquiry request. The first reply voice is determined by the server based on the digital human identifier of the first digital human and by invoking the first digital human according to the user interaction text. In this way, the timbre in the first reply voice can be the timbre of the character name corresponding to the first digital human.

[0181] In the above embodiment, when obtaining the first reply voice, a first inquiry request carrying the user interaction text and the digital human identifier of the first digital human is sent to the server, so that the server generates the first reply voice based on the user interaction text and the digital human identifier of the first digital human. The timbre in the first reply voice is the timbre of the character name corresponding to the first digital human, making the first reply voice more in line with the digital human image, thereby improving the fun of interacting with the user.

[0182] In some alternative implementation manners, a voice service application and a desktop application for playing the currently played media asset may also be integrated in the display device; among them, the desktop application for playing the currently played media asset may be an application that is opened by default when the display device is turned on and can be used to support the display device in playing media data. In some embodiments, the desktop application is a pre-set application that can be used as a signal source for the display device. The voice service application is an application in the display device that can provide a voice interaction function. Optionally, in some embodiments, the voice service application can provide two voice processing functions. One is to directly output the text converted from the user interaction voice, and the other is to output the reply voice corresponding to the user interaction voice. Specifically, the voice service application can interact with the desktop application to obtain the current scene information, user interaction voice, etc. from the desktop application, and then process the user interaction voice based on the current scene information by using any of the above voice processing functions.

[0183] Based on this, before receiving the input user interaction voice, the desktop application can send the user wake-up voice and the current scene information to the voice service application; among them, the user wake-up voice is used to wake up the voice service application. Exemplarily, the user wake-up voice can be pre-set voice data. For example, it can be "Xiaoju Xiaoju". When the user inputs "Xiaoju Xiaoju" in voice, the desktop application can send the input "Xiaoju Xiaoju" of the user to the voice service application, and the voice service application is woken up.

[0184] The current scene information is used to characterize the scene where the desktop application is currently located. Optionally, the desktop application can determine the current scene information based on the information of whether the digital human interaction control is triggered. For example, if the digital human interaction control is triggered, the generated current scene information can be the first identifier; if the digital human interaction control is not triggered, the generated current scene information can be the second identifier; the first identifier corresponds to the role interaction scene, and the second identifier corresponds to the non-role interaction scene.

[0185] After the desktop application sends the current scene information to the voice service application, the voice service application can determine whether the current scene information is a role interaction scene or a non-role interaction scene based on the identifier corresponding to the current scene information and the corresponding relationship between the identifier and the interaction scene. For example, if the identifier corresponding to the current scene information is the first identifier, it is determined that the current scene information is a role interaction scene; if the identifier corresponding to the current scene information is the second identifier, it is determined that the current scene information is a non-role interaction scene.

[0186] Exemplarily, referring to Figure 10 , Figure 10 a flowchart of a method for processing user interaction voice by a voice service application is provided, which specifically includes the following steps:

[0187] S1001, the desktop application sends user interaction voice to the voice service application.

[0188] S1002, after the voice service application recognizes that the current scene information is a role interaction scene, it converts the user interaction voice into user interaction text.

[0189] After the voice service application is awakened, after the desktop application receives the user interaction voice input by the user, it can send the user interaction voice to the voice service application. Furthermore, the voice service application first determines whether it is a role interaction scene based on the current scene information. For example, if the voice service application recognizes that the current scene information includes the information that the digital human interaction control is triggered, it determines that the current scene information is a role interaction scene. That is, after the voice service application recognizes that the current scene information is a role interaction scene, it converts the user interaction voice into user interaction text, and sends the user interaction text to the desktop application. The desktop application sends a first inquiry request carrying the user interaction text and the digital human identifier of the first digital human to the server, so that the server can feedback a first reply voice based on the first inquiry request. After the desktop application receives the first reply voice feedback by the server based on the first inquiry request, it can call the multimedia player to play the first reply voice.

[0190] For example, a voice service application converts the received user interaction voice into text to obtain user interaction text, such as "What's the weather like today?", and sends it to a desktop application. The desktop application then sends it to a server. At this time, the server invokes a relevant digital human to generate a first reply voice based on the user interaction text, such as "It's sunny today", and sends it to the desktop application. After receiving the first reply voice, the desktop application can invoke a multimedia player to play the first reply voice of "It's sunny today", and the voice color in the first reply voice is the voice color of the character name corresponding to the first digital human.

[0191] S1003. After the voice service application recognizes that the current scene information is a non-role interaction scene, it obtains the third reply voice corresponding to the user interaction voice and plays the third reply voice.

[0192] After the desktop application sends the user interaction voice to the voice service application, if the voice service application recognizes that the current scene information is a non-role interaction scene, for example, the voice service application recognizes that the current scene information includes information that the digital human interaction control is not triggered, then it determines that the current scene information is a non-role interaction scene. That is, after the voice service application recognizes that the current scene information is a non-role interaction scene, the voice service application may not convert the user interaction voice into user interaction text, but directly send the user interaction voice to the server, so that the server generates a third reply voice based on the user interaction voice and sends the third reply voice to the voice service application. The voice service application obtains the third reply voice corresponding to the user interaction voice and plays the third reply voice.

[0193] Alternatively, if the voice service application recognizes that the current scene information is a non-role interaction scene, the voice service application may also convert the user interaction voice into user interaction text, and then send the user interaction text corresponding to the user interaction voice to the server, so that the server generates a third reply voice based on the user interaction text and sends the third reply voice to the voice service application. The voice service application obtains the third reply voice corresponding to the user interaction text and plays the third reply voice through the voice service application.

[0194] For example, the voice service application receives a user interaction voice, such as "What's the weather like today?", and can send the user interaction voice to the server. The server generates a third reply voice based on the user interaction voice, such as "It's sunny today"; then, the server can send the third reply voice to the voice service application, and the voice service application will play the third reply voice of "It's sunny today".

[0195] In the above embodiments, the display device is integrated with a voice service application and a desktop application for playing the currently played media asset. The desktop application can send a user wake-up voice and current scene information to the voice service application, so that the voice service application can determine whether it is a role interaction scene based on the current scene information, and further facilitate determining the way to play the reply voice corresponding to the user interaction voice. On the one hand, the function of the voice service is broadened; on the other hand, the distinction between the role interaction scene and the non-role interaction scene is realized, making the interaction method with the user more in line with the user's interaction needs, thereby improving the user interaction experience.

[0196] In some optional implementation manners, when the display device is integrated with a desktop application for playing the currently played media asset, when playing the first reply voice, the desktop application can be used to call a multimedia player to play the first reply voice. In this way, the purpose of playing the first reply voice with the timbre of the character name corresponding to the first digital human can be achieved, thereby improving the interest of the interaction with the user.

[0197] In the above embodiments, using the desktop application to call a multimedia player to play the first reply voice, on the one hand, improves the interest of the interaction with the user, and on the other hand, reduces the development cost; and the multimedia player usually supports multiple audio formats, reducing the playback problems caused by format incompatibility and improving the stability and reliability of the first reply voice playback.

[0198] In some optional implementation manners, after the user wakes up the voice service application, the voice service application can monitor the user interaction voice input by the user in real time, so as to timely feedback the reply voice corresponding to the user interaction voice to improve the real-time performance of the human-computer interaction.

[0199] Exemplarily, referring to Figure 11 , Figure 11 a flowchart of a method for providing an encore prompt is provided, which specifically includes the following steps:

[0200] S1101, after the voice service application monitors that the time without voice input exceeds a preset duration, it sends an encore prompt message to the desktop application.

[0201] However, if the voice service application does not monitor the user interaction voice within a period of time, that is, after the voice service application monitors that the time without voice input exceeds the preset duration, it can send an encore prompt message to the desktop application. Among them, the preset duration can be set according to the actual situation. For example, it can be set to 10 minutes, and no specific limitation is made on the preset duration here. The encore prompt message is an instruction message used to indicate outputting to attract the user to watch the currently played media asset or interact with the digital human.

[0202] S1102. Based on the return prompt information, the desktop application obtains the prompt voice of the third digital human from the server and calls the multimedia player to play the prompt voice.

[0203] After receiving the return prompt information, the desktop application can obtain the prompt voice of the third digital human from the server based on the return prompt information and call the multimedia player to play the prompt voice. Herein, the third digital human can be the currently selected digital human, any one of the candidate digital humans involved in the currently played media asset, or the first digital human in the digital human list. No specific limitation is made on the third digital human herein, and it can be set according to the actual situation.

[0204] Exemplarily, the prompt voice can be a voice prompting the user to continue interacting with the digital human. For example, it can be "Dear user, please continue to interact with me"; it can also be a voice prompting the user to continue watching the subsequent played media asset data. For example, it can be "Don't go away, the following content is more wonderful"; it can also be a voice corresponding to the overview content of the wonderful content of the subsequent played media asset data. For example, it can be "Next, Qiqi is about to start an adventure journey. Let's go on an adventure with Qiqi". The above prompt voices are only exemplary descriptions and are not used to limit the content of the prompt voice.

[0205] In the above embodiment, after the voice service application monitors that the time without voice input exceeds the preset duration, it sends a return prompt information to the desktop application; so that the desktop application obtains the prompt voice of the third digital human from the server based on the return prompt information and calls the multimedia player to play the prompt voice to attract the user to continue watching the media asset or continue interacting with the digital human, thereby improving the user's enthusiasm for watching the media asset and interacting with the digital human.

[0206] In some optional implementation manners, during the process that the desktop application obtains the prompt voice of the third digital human from the server based on the return prompt information, the prompt voice of the third digital human can be generated based on the media asset identifier of the currently played media asset, the currently played frame image, and the third digital human.

[0207] Based on this, see Figure 12 , Figure 12 A flow diagram for obtaining and playing the prompt voice of the third digital human is provided, which specifically includes the following steps:

[0208] S1201. Based on the return prompt information, the desktop application sends a prompt voice acquisition request to the server.

[0209] Exemplarily, after the voice service application monitors that the time without voice input exceeds the preset duration, it sends a return prompt message to the desktop application. After receiving the return prompt message, the desktop application sends a prompt text acquisition request to the server based on the return prompt message. Among them, the prompt text acquisition request may carry the media identifier of the currently played media and the image of the currently played frame. The server can obtain the media drama and plot of the currently played media, as well as the drama and plot of the media to be played next, based on the media identifier of the currently played media and the image of the currently played frame, so as to generate a prompt voice that can attract users according to the media drama and plot of the currently played media, as well as the drama and plot of the media to be played next.

[0210] Exemplarily, the server can extract the key elements in the currently played frame image. Among them, the key elements in the currently played frame image include but are not limited to the theme, plot, core character emotions, etc. Furthermore, the server can obtain the prompt text and prompt voice frequency associated with the plot and character emotions based on the media identifier and the identified key elements of the picture.

[0211] S1202, the desktop application, obtains the prompt voice of the third digital human feedback by the server.

[0212] Exemplarily, the prompt voice can be determined by the server based on the digital human identifier of the third digital human and invoking the third digital human according to the media identifier and the currently played frame image. For example, after receiving the prompt text acquisition request, the server can determine the media drama and plot of the currently played media, as well as the drama and plot of the media to be played next, based on the media identifier of the currently played media and the image of the currently played frame carried in the prompt text acquisition request, and determine the voice content that can attract users to continue watching the media or continue to interact with the digital human based on the media drama and plot of the currently played media, as well as the drama and plot of the media to be played next; and can convert the timbre in the voice content into the timbre of the third digital human.

[0213] Among them, the third digital human can be selected from the candidate digital humans. For example, the third digital human can be the currently selected digital human among the candidate digital humans, or the digital human specified by the user; it can also be one of the digital humans included in the currently played frame image, or any one of the digital humans included in the drama after the currently played frame image.

[0214] S1203, the desktop application, plays the prompt voice when the third digital human is the same as the candidate digital human corresponding to the digital human card where the screen focus is located.

[0215] Exemplarily, before playing the prompt voice, it can first be determined whether the third digital human is the same as the candidate digital human corresponding to the digital human card where the screen focus is located. If they are the same, the desktop application can call the multimedia player to play the prompt voice.

[0216] S1204, the desktop application, in the case where the third digital human is not the same as the candidate digital human corresponding to the digital human card where the screen focus is located, focuses the screen focus on the digital human card corresponding to the third digital human and plays the prompt voice.

[0217] Exemplarily, in the case where the third digital human is not the same as the candidate digital human corresponding to the digital human card where the screen focus is located, the desktop application needs to first focus the screen focus on the digital human card corresponding to the third digital human and call the multimedia player to play the prompt voice. In this way, it is possible to play the prompt voice using the voice color of the character name corresponding to the third digital human, making the played voice more matching the image of the digital human.

[0218] In the above embodiments, the prompt voice is generated based on the digital human identifier of the third digital human, the media asset identifier of the currently played media asset, and the currently played frame image. On the one hand, the generated prompt voice is more in line with the current plot and the subsequent plot of the currently played media asset, which can improve the enthusiasm of users to continue watching the media asset; on the other hand, playing the prompt voice using the voice color of the character name corresponding to the third digital human not only realizes switching digital humans according to the screen focus, but also makes the played voice more matching the image of the switched digital human.

[0219] In some alternative implementation manners, the server in the above embodiments can be implemented by a single server or by a server cluster. Exemplarily, if implemented by a server cluster, the server cluster can be composed of a desktop application, a voice service application, a terminal-facing subsystem, a content subsystem, an image recognition service, and a digital human model. Among them, the terminal-facing subsystem provides a bridge for the interaction between the desktop application and the content subsystem, the image recognition service, and the digital human model; the content subsystem is used to store the guiding text and audio data of the digital human; the image recognition service is used to provide functions such as image recognition; the digital human model is used to provide digital human-related services.

[0220] Among them, the digital human model can be pre-trained according to sample media asset data, sample audio data, and sample text data.

[0221] Exemplarily, see Figure 13 , Figure 13 A timing diagram for training a digital human model is provided, which specifically includes the following steps:

[0222] S1301, The digital human model sends the sample audio-visual data and the drama script of the character corresponding to the digital human to the large model service.

[0223] S1302, The large model service conducts voice replication of the digital human and script knowledge learning, generates a trained digital human, and assigns a unique digital human identifier to each digital human.

[0224] S1303, The large model service sends the digital human and the digital human identifier to the digital human model.

[0225] S1304, The operation system conducts digital human maintenance.

[0226] Among them, digital human maintenance includes setting the guiding language for each digital human, associating each digital human with the corresponding digital human identifier, and setting the media asset identifier. The digital human identifier can be the digital human ID, and the media asset identifier can be the media asset IP.

[0227] S1305, The operation system sends the digital human, the corresponding digital human identifier, the guiding language for each digital human, and the media asset identifier to the content subsystem.

[0228] S1306, The content subsystem associates the digital human, the corresponding digital human identifier, the guiding language for each digital human, with the media asset identifier.

[0229] S1307, The content subsystem sends the guiding language text for each digital human and the corresponding digital human identifier to the digital human model.

[0230] S1308, The digital human model generates the interaction guiding language corresponding to each digital human identifier based on each digital human identifier and the corresponding guiding language text.

[0231] S1309, The digital human model feeds back each digital human identifier and the corresponding interaction guiding language to the content subsystem.

[0232] S1310, The content subsystem stores each digital human identifier and the corresponding interaction guiding language.

[0233] S1311, The third-party content sends a notice of drama update to the content subsystem.

[0234] S1312, The content subsystem sends a message of receiving the notice of drama update to the third-party content.

[0235] S1313, The content subsystem sends a notice of drama update to the large model service.

[0236] S1314, The large model service sends a request for obtaining the newly added script content to the content subsystem.

[0237] S1315. The content subsystem feeds back the newly added script content to the large model service.

[0238] S1316. The large model service updates the digital human based on the newly added script content.

[0239] S1317. The large model service sends the updated digital human to the digital human model.

[0240] In the above embodiments, digital humans corresponding to the character roles in the media asset data can be trained based on the media asset data, so that when the user views the media asset data, they can interact with the digital human corresponding to the media asset data, thereby improving the intelligence and interest of human-computer interaction.

[0241] In some alternative implementation manners, refer to Figure 14 , Figure 14 A timing diagram of voice interaction is provided, which specifically includes the following steps:

[0242] S1401. The user triggers a request to obtain the media asset details page associated with the currently playing media asset.

[0243] S1402. After receiving the request to obtain the media asset details page associated with the currently playing media asset, the desktop application sends the request to obtain the media asset details page associated with the currently playing media asset carrying the media asset identifier to the terminal-facing subsystem.

[0244] S1403. The terminal-facing subsystem sends a digital human query request carrying the media asset identifier to the content subsystem.

[0245] S1404. The content subsystem feeds back a digital human list identical to the media asset identifier to the terminal-facing subsystem.

[0246] S1405. The content subsystem feeds back the relevant data of the media asset details page and the digital human list to the desktop application.

[0247] S1406. The desktop application displays the media asset details page based on the relevant data of the media asset details page and the digital human list.

[0248] S1407. The user triggers a click operation instruction on the digital human interaction control in the media asset details page.

[0249] S1408. In response to the operation triggered by the digital human interaction control, the desktop application displays the digital human card of the candidate digital human associated with the currently playing media asset.

[0250] S1409. The desktop application sends a request to obtain the guiding language to the terminal-facing subsystem according to the digital human identifier corresponding to the digital human card where the page focus is located.

[0251] S1410, The terminal-oriented subsystem sends the digital human identifier corresponding to the digital human card where the page focus is located and a request for obtaining the guiding speech corresponding to the digital human identifier to the content subsystem.

[0252] S1411, The content subsystem feeds back the interactive guiding speech of the corresponding digital human to the terminal-oriented subsystem.

[0253] S1412, The terminal-oriented subsystem feeds back the interactive guiding speech of the corresponding digital human to the desktop application.

[0254] S1413, The desktop application plays the interactive guiding speech using a multimedia player.

[0255] S1414, The user inputs a user wake-up voice to the desktop application.

[0256] S1415, The desktop application sends a wake-up request carrying the user wake-up voice and the current scene information to the voice service application.

[0257] S1416, The voice service application starts.

[0258] S1417, The user inputs an interactive voice for the target digital human.

[0259] Among them, the target digital human can be the currently selected digital human or the digital human represented by the digital human association information included in the user interactive voice.

[0260] S1418, The voice service application recognizes the current scene information. In the case where the recognized current scene information is a role interaction scene, it converts the user input user interactive voice into a user interactive text.

[0261] S1419, The voice service application sends the user interactive text to the desktop application.

[0262] S1420, The desktop application sends the user interactive text and the identifier of the target digital human to the digital human model.

[0263] S1421, The digital human model calls the target digital human based on the digital human identifier of the target digital human, so that the target digital human outputs a reply voice and a reply text according to the text.

[0264] S1422, The digital human model feeds back the reply voice and the reply text to the desktop application.

[0265] S1423, The desktop application displays the reply text and plays the reply voice using a player.

[0266] S1424, The voice service application continuously monitors the user voice.

[0267] S1425, after the voice service application monitors that the user has no voice input for more than a preset duration, it sends a recall prompt message to the desktop application.

[0268] S1426, the desktop application sends the digital human identifier, the media identifier of the currently played media asset, and the current playing frame image to the terminal subsystem.

[0269] S1427, the terminal subsystem sends a request to the image recognition service to obtain key elements in the current playing frame image.

[0270] S1428, the image recognition service feeds back the key elements in the current playing frame image to the terminal subsystem.

[0271] S1429, the terminal subsystem sends a prompt speech acquisition request containing the digital human identifier, the episode identifier, and the recognized key elements of the image to the digital human model.

[0272] S1430, the digital human model feeds back the prompt speech and the corresponding text to the desktop application.

[0273] S1431, the desktop application displays the text of the prompt speech and plays the obtained prompt speech using a player.

[0274] As described above in conjunction with Figures 1 to 14 , the voice interaction method of the embodiment of the present application has been described in detail. Next, in conjunction with Figure 15 , the device embodiment of the present application will be described in detail. It should be understood that the voice interaction device in the embodiment of the present application can execute various voice interaction methods of the foregoing embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the foregoing method embodiments.

[0275] Figure 15 is a schematic diagram of the voice interaction device provided by the embodiment of the present application.

[0276] It should be understood that the voice interaction device can execute Figures 1 to 14 the voice interaction method shown; the voice interaction device 600 includes:

[0277] The first control unit 610 is used to control the display to display the media details page associated with the currently played media asset; wherein, a digital human interaction control is displayed on the media details page;

[0278] The second control unit 620 is used to, in response to a trigger operation on the digital human interaction control, control the display to display the digital human card of the candidate digital human associated with the currently played media asset; wherein, each digital human card of the candidate digital human contains the digital human image corresponding to the candidate digital human and / or the name of the character role in the currently played media asset represented by the candidate digital human;

[0279] A receiving unit 630, configured to receive input user interaction voice after a digital human card of a first digital human is selected, where the first digital human is one of candidate digital humans;

[0280] A playing unit 640, configured to obtain a first reply voice of the first digital human for the user interaction voice from a server according to the user interaction voice, and play the first reply voice; wherein, the timbre in the first reply voice is the timbre of the character name corresponding to the first digital human.

[0281] In the above embodiments, the voice interaction device provides a bridge for user interaction with a digital human by displaying a media resource details page associated with the currently played media resource, and a digital human interaction control is displayed on the media resource details page. Further, during the video playback process, the user is no longer passively receiving interactive recommendation content, but can actively interact with the digital human through the digital human interaction control on the media resource details page, improving the initiative of user interaction with the digital human; further, in response to a trigger operation on the digital human interaction control, by displaying a digital human card of a candidate digital human associated with the currently played media resource, since different candidate digital humans represent different character roles in the currently played media resource, subsequent voice interaction is made more suitable for the currently played media resource, and a basis for subsequent personalized voice interaction is provided; in addition, after the digital human card of the first digital human is selected, input user interaction voice can be received, and according to the user interaction voice, a first reply voice of the first digital human for the user interaction voice is obtained from the server, and the first reply voice with the timbre of the character name corresponding to the first digital human is played, improving the interest of interaction between the digital human and the user.

[0282] In some embodiments, the playing unit 640 is specifically configured to:

[0283] In the case where the user interaction voice does not contain digital human association information, obtain a first reply voice of the first digital human for the user interaction voice from the server according to the user interaction voice, and play the first reply voice; in the case where the user interaction voice contains digital human association information, if the second digital human represented by the digital human association information is different from the first digital human, focus the screen focus on the digital human card corresponding to the second digital human, and obtain a second reply voice of the second digital human for the user interaction voice from the server according to the user interaction voice, and play the second reply voice; wherein, the timbre in the second reply voice is the timbre of the character name corresponding to the second digital human.

[0284] In some embodiments, the playing unit 640 includes:

[0285] A conversion sub-unit, configured to convert the user interaction voice into a user interaction text;

[0286] A sending subunit, configured to send a first inquiry request carrying user interaction text and a digital human identifier of a first digital human to a server;

[0287] A receiving subunit, configured to receive a first reply voice feedback by the server based on the first inquiry request; wherein, the first reply voice is determined by the server based on the digital human identifier of the first digital human and by invoking the first digital human according to the user interaction text.

[0288] In some embodiments, a voice service application and a desktop application for playing currently played media assets are integrated in a display device; the apparatus further includes a sending unit, before receiving an input user interaction voice, the sending unit is configured to support the desktop application to send a user wake-up voice and current scene information to the voice service application; wherein, the user wake-up voice is used to wake up the voice service application;

[0289] A conversion subunit supports the desktop application to send a user interaction voice to the voice service application; and supports the voice service application to convert the user interaction voice into user interaction text after recognizing that the current scene information is a role interaction scene; supports the voice service application to obtain a third reply voice corresponding to the user interaction voice and play the third reply voice after recognizing that the current scene information is a non-role interaction scene.

[0290] In some embodiments, the playing unit 640 is specifically configured to:

[0291] Support the desktop application to invoke a multimedia player to play the first reply voice.

[0292] In some embodiments, the apparatus further includes an invoking unit, after playing the first reply voice, the sending unit is configured to: after the voice service application monitors that the time without voice input exceeds a preset duration, send a return prompt message to the desktop application;

[0293] The invoking unit is configured to support the desktop application to obtain a prompt voice of a third digital human from the server based on the return prompt message and invoke a multimedia player to play the prompt voice.

[0294] In some embodiments, the invoking unit is specifically configured to:

[0295] Support the desktop application to send a prompt message acquisition request to the server based on the encore prompt message; wherein, the prompt message acquisition request carries the media identifier of the currently played media and the image of the currently played frame; support the desktop application to obtain the prompt voice of the third digital person fed back by the server; wherein, the third digital person is selected from the candidate digital persons; the prompt voice is determined by the server based on the digital person identifier of the third digital person and called the third digital person according to the media identifier and the image of the currently played frame; support the desktop application to play the prompt voice when the third digital person is the same as the candidate digital person corresponding to the digital person card where the screen focus is located; support the desktop application to focus the screen focus on the digital person card corresponding to the third digital person and play the prompt voice when the third digital person is different from the candidate digital person corresponding to the digital person card where the screen focus is located.

[0296] In some embodiments, the first control unit 610 is specifically configured to:

[0297] Send a detail page acquisition request for the currently played media to the server; receive the detail page data fed back by the server based on the detail page acquisition request; in the case that the detail page data contains digital person information, generate a media detail page containing digital person interaction controls according to the detail page data, and control the display to display the media detail page.

[0298] In some embodiments, the first control unit 610 is specifically configured to:

[0299] Control the display to display the digital person cards of the candidate digital persons associated with the currently played media in the digital person display order; after controlling the display to display the digital person cards of the candidate digital persons associated with the currently played media, further include: after the digital person card of the first digital person is selected, obtain the interaction guiding language of the first digital person and play the interaction guiding language of the first digital person.

[0300] Each unit module of the voice interaction device can respectively execute the corresponding steps in the above method embodiments, so the unit modules will not be described in detail here. For details, please refer to the description of the above corresponding steps.

[0301] It should be noted that the above voice interaction device is embodied in the form of functional units. The term "unit" here can be implemented in the form of software and / or hardware, and no specific limitation is made thereto.

[0302] For example, a "unit" may be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group of processors, etc.) for executing one or more software or firmware programs, and a memory, a combined logic circuit, and / or other suitable components that support the described functions.

[0303] Therefore, the units of the various examples described in the embodiments of the present application can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0304] As Figure 1 , Figure 2 shown, the display device 200 can be used to implement the display control method described in the above method embodiments.

[0305] The display device 200 may include one or more memories, on which there is a program that can be run by the controller 250 to generate instructions, so that the controller 250 executes the display control method described in the above method embodiments according to the instructions.

[0306] Optionally, data may also be stored in the memory. Optionally, the controller 250 may also read the data stored in the memory. The data may be stored at the same storage address as the program, or the data may be stored at a different storage address from the program.

[0307] The controller 250 and the memory may be provided separately or integrated together; for example, integrated on a system on chip (SOC) of a terminal device.

[0308] The present application also provides a computer program product, which implements the display control method of any method embodiment in the present application when executed by the controller 250.

[0309] The computer program product may be stored in the memory, for example, it is a program that is finally converted into an executable target file that can be executed by the controller 250 after processes such as preprocessing, compilation, assembly, and linking.

[0310] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, it implements the display control method of any method embodiment in the present application. The computer program can be a high-level language program or an executable target program.

[0311] The computer-readable storage medium is, for example, a memory. The memory can be a volatile memory or a non-volatile memory, or the memory can include both a volatile memory and a non-volatile memory at the same time. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0312] In the present application, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following (items)" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c can be single or multiple.

[0313] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0314] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0315] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0316] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of units is only a logical function division, and there can be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0317] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0318] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0319] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application and should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A voice interaction method, characterized in that: Applied to display devices, including: Controlling the display to display a media resource detail page associated with the currently playing media resource; wherein the media resource detail page displays a digital human interaction control; In response to a trigger operation on the digital human interactive control, the display is controlled to display digital human cards of candidate digital humans associated with the currently playing media asset; wherein the digital human card of each candidate digital human includes a digital human image corresponding to the candidate digital human and / or a character name in the currently playing media asset represented by the candidate digital human; After a digital person card of a first digital person is selected, receiving an input user interaction voice, wherein the first digital person is one of the candidate digital persons; According to the user interaction voice, a first reply voice of the first digital human to the user interaction voice is obtained from the server, and the first reply voice is played; wherein the timbre in the first reply voice is the timbre of the character name corresponding to the first digital human.

2. The method according to claim 1, characterized in that The step of acquiring, from a server, a first reply voice of the first digital human to the user interaction voice according to the user interaction voice, and playing the first reply voice includes: In the case that the user interaction voice does not contain the digital human associated information, obtaining a first reply voice of the first digital human to the user interaction voice from the server according to the user interaction voice, and playing the first reply voice; In the case where the user interaction voice contains digital human association information, if the second digital human represented by the digital human association information is different from the first digital human, the screen focus is placed on the digital human card corresponding to the second digital human, and based on the user interaction voice, a second reply voice of the second digital human to the user interaction voice is obtained from the server, and the second reply voice is played; wherein the timbre in the second reply voice is the timbre of the character name corresponding to the second digital human.

3. The method according to claim 1 or 2, characterized in that: The acquiring, from the server, a first reply voice of the first digital human to the user interaction voice according to the user interaction voice comprises: Converting the user interaction speech into user interaction text; Sending a first inquiry request carrying the user interaction text and the digital human identifier of the first digital human to the server; Receive the first reply voice fed back by the server based on the first inquiry request; wherein the first reply voice is determined by the server based on the digital human identifier of the first digital human and calling the first digital human according to the user interaction text.

4. The method according to claim 3, characterized in that The display device is integrated with a voice service application and a desktop application for playing the currently playing media asset; Before receiving the input user interaction voice, the method further includes: The desktop application sends a user wake-up voice and current scene information to the voice service application; wherein the user wake-up voice is used to wake up the voice service application; The converting the user interaction voice into user interaction text comprises: The desktop application sends the user interaction voice to the voice service application; After the voice service application recognizes that the current scene information is a role interaction scene, the user interaction voice is converted into a user interaction text; After the voice service application recognizes that the current scene information is a non-role interaction scene, it obtains a third reply voice corresponding to the user interaction voice and plays the third reply voice.

5. The method according to claim 4, characterized in that The playing of the first reply voice includes: The desktop application calls a multimedia player to play the first reply voice.

6. The method according to claim 5, characterized in that After playing the first reply voice, the method further includes: After the voice service application detects that there is no voice input for a period exceeding a preset time, the voice service application sends a return prompt message to the desktop application; The desktop application obtains the prompt voice of the third digital human from the server based on the return prompt information, and calls a multimedia player to play the prompt voice.

7. The method according to claim 6, characterized in that The desktop application obtains the prompt voice of the third digital human from the server based on the return prompt information, and calls a multimedia player to play the prompt voice, including: The desktop application sends a prompt acquisition request to the server based on the return prompt information; wherein the prompt acquisition request carries the media asset identifier of the currently playing media asset and the currently playing frame image; The desktop application obtains the prompt voice of the third digital person fed back by the server; wherein the third digital person is selected from the candidate digital persons; the prompt voice is determined by the server based on the digital person identifier of the third digital person, calling the third digital person according to the media resource identifier and the current playing frame image; The desktop application plays the prompt voice when the third digital human is the same as the candidate digital human corresponding to the digital human card where the screen focus is; When the third digital human is different from the candidate digital human corresponding to the digital human card where the screen focus is, the desktop application focuses the screen on the digital human card corresponding to the third digital human and plays the prompt voice.

8. The method according to any one of claims 1 to 7, characterized in that: The control display displays the media asset details page associated with the currently playing media asset, including: Sending a request to the server for obtaining the details page of the currently playing media asset; Receiving the detail page data fed back by the server based on the detail page acquisition request; In the case where the detail page data includes digital human information, a media resource detail page including the digital human interactive control is generated according to the detail page data, and the display is controlled to display the media resource detail page.

9. The method according to any one of claims 1 to 7, characterized in that: The controlling the display to display the digital human card of the candidate digital human associated with the currently playing media asset comprises: Controlling the display to display digital human cards of candidate digital humans associated with the currently playing media resource in the order in which digital humans are displayed; After controlling the display to display the digital human card of the candidate digital human associated with the currently playing media asset, the method further includes: After the digital human card of the first digital human is selected, the interaction guide words of the first digital human are obtained, and the interaction guide words of the first digital human are played.

10. A display device, characterized in that: include: Display and controller; The controller is configured to: Controlling the display to display a media resource detail page associated with the currently playing media resource; wherein the media resource detail page displays a digital human interaction control; In response to a trigger operation on the digital human interactive control, the display is controlled to display digital human cards of candidate digital humans associated with the currently playing media asset; wherein the digital human card of each candidate digital human includes a digital human image corresponding to the candidate digital human and / or a character name in the currently playing media asset represented by the candidate digital human; After a digital person card of a first digital person is selected, receiving an input user interaction voice, wherein the first digital person is one of the candidate digital persons; According to the user interaction voice, a first reply voice of the first digital human to the user interaction voice is obtained from the server, and the first reply voice is played; wherein the timbre in the first reply voice is the timbre of the character name corresponding to the first digital human.

Images