Television equipment and session processing method

Abstract

The invention relates to television equipment and a session processing method. Wherein the television device comprises: a display component; a remote control signal receiving assembly; an audio output component; a network communication device; a voice acquisition assembly; the controller is configured to perform voiceprint recognition on the voice of the current user to determine a voiceprint identifier corresponding to the current user, and display a voice text; extracting user feature information from the knowledge graph based on the voiceprint identifier; obtaining a dialogue text based on a generative model in a first server of the user feature information; dialogue voice data are acquired; and displaying the dialogue text through the first interface, outputting the dialogue voice data through the audio output component, and driving a preset digital human image to make an action corresponding to the dialogue voice data through the digital human driving system. The television equipment and the session processing method can provide a non-Chinese voice dialogue function with better quality.

Description

Technical Field

[0001] The present application relates to the technical field of television equipment, and in particular to a television equipment and a session processing method. Background Art

[0002] Currently, there is a trend towards intelligent development of television devices. Users can conduct dialogue training with devices such as televisions to improve their foreign language skills, such as spoken English. However, currently, the quality of dialogue that television devices can provide is poor. Summary of the invention

[0003] The present application provides a television device and a session processing method to improve the quality of a session between the television device and a user.

[0004] In a first aspect, some embodiments provide a television device, comprising:

[0005] Display components;

[0006] Remote control signal receiving component;

[0007] Audio output component;

[0008] Network communication devices;

[0009] The voice collection component is used to collect the current user's voice when the remote control signal receiving component receives a key signal corresponding to a target key in the remote control; the target key is a preset key for starting the TV voice recognition function;

[0010] The controller is configured as:

[0011] When the display component displays the first interface and the voice collection component collects the current user's voice, voiceprint recognition is performed on the current user's voice, the voiceprint identifier corresponding to the current user is determined based on the voiceprint recognition result, and user feature information corresponding to the current user is extracted from the preset knowledge graph based on the voiceprint identifier, and the current user's voice is converted into corresponding voice text, and the voice text is displayed through the first interface; the first interface is a preset interface for conducting voice dialogue with a digital human;

[0012] Initiating a first request to a first server through a network communication device; the first request carries a dialogue text request instruction, a voice text, and user feature information; receiving a dialogue text returned by the first server; wherein the dialogue text is generated based on the first request and a generative model preset in the first server;

[0013] Initiating a second request to a second server through a network communication device; the second request carries a text-to-speech instruction and a conversation text;

[0014] Receiving the conversation voice data returned by the second server; the conversation voice data is generated based on the second request and the text-to-speech service preset in the second server;

[0015] The conversation text is sent to the display component, and the conversation voice data is sent to the audio output component and the preset digital human driving system respectively, so as to display the conversation text through the first interface, output the conversation voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the conversation voice data through the digital human driving system.

[0016] In one embodiment, the controller is further configured to:

[0017] When the remote control signal receiving component receives a confirmation instruction sent by the remote control, if the current display focus in the display component is located on a preset dialogue mode option, the display component is controlled to display the first interface; or,

[0018] When the remote control signal receiving component receives the confirmation command sent by the remote control, if the current display focus in the display component is located on the preset guessing picture mode option, the display component is controlled to display the second interface; the second interface is the preset interface for guessing the picture with the digital human by voice;

[0019] The first interface and the second interface are interfaces included in the target application; the target application is an application including a digital human voice dialogue function.

[0020] In one embodiment, the controller is further configured to:

[0021] Based on the historical user voices collected by the voice collection component, the voiceprint information corresponding to each historical user is analyzed, and the voiceprint identification of each historical user is determined based on the voiceprint information;

[0022] Convert historical user voices into corresponding historical voice texts, and extract entity information corresponding to predefined entity words and the association relationship between different entity information from the historical voice texts; the predefined entity words include one or more of name, gender, age, interest, and character relationship;

[0023] A knowledge graph is constructed based on the voiceprint identification, entity information and association relationship of at least one historical user; the nodes in the knowledge graph represent the voiceprint identification and entity information, and the edges between the nodes represent the association relationship.

[0024] In one embodiment, entity information corresponding to predefined entity words and association relationships between different entity information are extracted from historical speech texts, including:

[0025] Initiate a third request to the first server through the network communication device; the third request carries historical voice text and entity information extraction instructions;

[0026] Entity information returned by the first server and associations between different entity information are received; the entity information and the associations are generated based on the third request, a set of entity words preset in the first server, and a generative model.

[0027] In one embodiment, the controller is further configured to:

[0028] After initiating a first request to the first server through a network communication device, the entity information contained in the voice text of the current user's voice and the association relationship between different entity information returned by the first server based on the first request are received, and the knowledge graph is updated.

[0029] In one embodiment, extracting user feature information corresponding to the current user from a preset knowledge graph based on the voiceprint identification includes:

[0030] Determine the node corresponding to the voiceprint identifier from the preset knowledge graph as the target node;

[0031] Determine at least one associated node of the target node, where there is an edge between the associated node and the target node;

[0032] The entity information represented by the associated node and the association relationship between the associated node and the target node are obtained as user feature information corresponding to the current user.

[0033] In one embodiment, the controller is further configured to:

[0034] When the display component switches from other interfaces to display the second interface, a preset number of images are extracted from the preset image database and cached; the preset number is an integer greater than 1;

[0035] Determine a picture from the cached pictures as the first target picture, and initiate a fourth request to the first server through the network communication device; the fourth request carries the opening text request instruction and the first target picture;

[0036] Receiving the opening text returned by the first server; the opening text is generated based on the fourth request and a generative model preset in the first server;

[0037] Initiate a fifth request to the second server through the network communication device; the fifth request carries the text-to-speech instruction and the opening text;

[0038] Receiving opening voice data returned by the second server; the opening voice data is generated based on the fifth request and a text-to-speech service preset in the second server;

[0039] Sending the first target image and the opening text to the display component, and sending the opening voice data to the audio output component and the digital human driving system respectively, so as to display the first target image and the opening text through the second interface, output the opening voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the dialogue voice data through the digital human driving system;

[0040] The opening text and the opening voice data correspond to the same non-Chinese language.

[0041] In one embodiment, the controller is further configured to:

[0042] When the display component displays the second interface and the voice collection component collects the current user's voice, the current user's voice is converted into a corresponding picture guessing answer text, and the picture guessing answer text is displayed through the second interface;

[0043] Initiate a sixth request to the first server through the network communication device; the sixth request carries a result determination request instruction, a guessing picture answer text, and a first target picture;

[0044] Receiving a determination result text returned by the first server; the determination result text is generated based on the sixth request and a generative model preset in the first server;

[0045] Initiate a seventh request to the second server through the network communication device; the seventh request carries a text-to-speech instruction and a determination result text;

[0046] Receiving the picture guessing result voice data returned by the second server; the picture guessing result voice data is generated based on the seventh request and the text-to-speech service preset in the second server;

[0047] Determine another picture from the cached pictures as the second target picture;

[0048] Obtaining a preset prompt text and its corresponding voice data; the prompt text and its corresponding voice data are used to prompt the user to continue guessing the picture;

[0049] The guessing result text, the second target image and the prompt text are sent to the display component, and the combined data of the guessing result voice data and the voice data corresponding to the prompt text are sent to the audio output component and the digital human driving system respectively, so as to display the guessing result text, the second target image and the prompt text through the second interface, output the combined data through the audio output component, and drive the preset digital human image to perform actions corresponding to the opening voice data through the digital human driving system;

[0050] Among them, the guess picture result text and guess picture result voice data correspond to non-Chinese languages.

[0051] In one of the embodiments, the first server and the second server are servers deployed in the cloud, and the first server and the second server are the same server or different servers; the generative model preset in the first server is a model based on the Transformer architecture and has a text-generating text function.

[0052] The above-mentioned television device has the following effects:

[0053] When the TV device enters the function page related to the digital human oral speech training, when the user operates the target button in the remote control, it indicates that the voice recognition function of the TV device needs to be activated to conduct a foreign language oral dialogue with the digital human. In this regard, the TV device collects the current user voice through the voice collection component, and converts the current user voice into voice text through the controller and displays it on the first interface so that the user can know the user speech content captured by the TV device; at the same time, the controller performs voiceprint recognition to determine the corresponding voiceprint identifier, and extracts the corresponding user feature information from the knowledge graph according to the voiceprint identifier. The controller of the TV device sends a first request to the first server to obtain the dialogue text for the user speech content. Since the dialogue text is generated based on the first request and the generative model preset in the first server, and the first request carries the voice text and user feature information, it is possible to obtain a dialogue text with better quality. The controller of the TV device sends a second request to the second server to obtain dialogue voice data, and the obtained dialogue voice data is consistent with the content of the dialogue text generated by the generative model. Finally, the dialogue text is displayed through the display component of the television device, the dialogue voice data is output through the audio output component, and the digital human is driven by the digital human driving system to make corresponding actions. Based on the three-end collaboration of the controller of the television device, the generative model in the first server, and the text-to-speech service in the second server, the foreign language dialogue practice between the user and the digital human is realized. Since the dialogue text of the digital human for the user's speech content is generated based on the generative model and user feature information, it can provide users with personalized dialogue content, and the quality of the dialogue text is better. At the same time, through the text-to-speech service and the digital human driving system, the dialogue text, digital human action, and digital human voice are coordinated and consistent, so it can provide a better foreign language oral dialogue experience.

[0054] In a second aspect, some embodiments further provide a session processing method, which is applied to the controller provided in the first aspect, and the method includes:

[0055] When the display component of the television device displays the first interface and the voice collection component of the television device collects the voice of the current user, voiceprint recognition is performed on the voice of the current user, a voiceprint identifier corresponding to the current user is determined based on the voiceprint recognition result, and user feature information corresponding to the current user is extracted from a preset knowledge graph based on the voiceprint identifier, and the current user voice is converted into a corresponding voice text, and the voice text is displayed through the first interface; the first interface is a preset interface for conducting a voice dialogue with a digital human;

[0056] Initiating a first request to a first server through a network communication device; the first request carries a dialogue text request instruction, a voice text, and user feature information; receiving a dialogue text returned by the first server; wherein the dialogue text is generated based on the first request and a generative model preset in the first server;

[0057] Initiating a second request to a second server through a network communication device; the second request carries a text-to-speech instruction and a conversation text;

[0058] Receiving the conversation voice data returned by the second server; the conversation voice data is generated based on the second request and the text-to-speech service preset in the second server;

[0059] The conversation text is sent to the display component, and the conversation voice data is sent to the audio output component of the television device and the preset digital human driving system respectively, so as to display the conversation text through the first interface, output the conversation voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the conversation voice data through the digital human driving system.

[0060] The above-mentioned telephone conversation processing method has the following effects:

[0061] When the TV device enters the function page related to the digital human oral speech training, when the user operates the target button in the remote control, it indicates that the voice recognition function of the TV device needs to be activated to conduct a foreign language oral dialogue with the digital human. In response, the controller in the TV device converts the current user voice into voice text and displays it on the first interface so that the user can know the user speech content captured by the TV device; at the same time, the controller performs voiceprint recognition to determine the corresponding voiceprint identifier, and extracts the corresponding user feature information from the knowledge graph according to the voiceprint identifier. The controller sends a first request to the first server through a network communication device to obtain a dialogue text for the user speech content. Since the dialogue text is generated based on the first request and the generative model preset in the first server, and the first request carries the voice text and user feature information, a dialogue text with better quality can be obtained. The controller sends a second request to the second server through a network communication device to obtain dialogue voice data, and the obtained dialogue voice data is consistent with the content of the dialogue text generated by the generative model. Finally, the controller sends the above-mentioned dialogue text to the display component of the TV device to display the dialogue text, sends the dialogue voice data to the audio output component to output the dialogue voice data, and sends the dialogue voice data to the digital human driving system to drive the digital human to make corresponding actions, thereby realizing the foreign language dialogue practice between the user and the digital human based on the three-end collaboration of the controller of the TV device, the generative model in the first server, and the text-to-speech service in the second server. Since the dialogue text of the digital human for the user's speech content is generated based on the generative model and user feature information, it can provide users with personalized dialogue content, and the quality of the dialogue text is better. At the same time, through the text-to-speech service and the digital human driving system, the three aspects of dialogue text, digital human action, and digital human voice are coordinated and consistent, so it can provide a better foreign language oral dialogue experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0064] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0065] Figure 3A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0066] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0067] Figure 5 A schematic diagram of a flow chart of a controller configured to execute in a television device provided in some embodiments of the present application;

[0068] Figure 6 A schematic diagram of another flow chart of a controller configured to execute in a television device provided in some embodiments of the present application;

[0069] Figure 7 A timing diagram of non-Chinese voice dialogue practice provided for some embodiments of the present application;

[0070] Figure 8 Statistical data graphs of non-Chinese voice dialogue exercises provided for some embodiments of the present application;

[0071] Fig. 9 A timing diagram of voice guessing provided in some embodiments of the present application;

[0072] Fig.10 An interface display diagram for guessing pictures by voice provided in some embodiments of the present application;

[0073] Fig.11 A first schematic diagram of a knowledge graph provided for some embodiments of the present application;

[0074] Fig.12 A second schematic diagram of a knowledge graph provided for some embodiments of the present application;

[0075] Fig.13 A third schematic diagram of a knowledge graph provided for some embodiments of the present application;

[0076] Fig.14 A fourth schematic diagram of a knowledge graph provided for some embodiments of the present application. DETAILED DESCRIPTION

[0077] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0078] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0079] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way are interchangeable under appropriate circumstances. Unless otherwise specified, the reference to multiple in this document can be understood as two or more. Unless otherwise specified, id and ID in this document can be the same.

[0080] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0081] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0082] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0083] Figure 1 Schematic diagram of operation scenarios between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0084] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0085] like Figure 1As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0086] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0087] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0088] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0089] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0090] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0091] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0092] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0093] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0094] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0095] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0096] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0097] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0098] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0099] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0100] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0101] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0102] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0103] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0104] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0105] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0106] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0107] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0108] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0109] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for a display device.

[0110] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0111] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0112] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0113] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0114] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0115] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0116] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0117] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0118] In an exemplary embodiment, a television device is provided, and a user can interact with the television device through a remote control, voice, etc. The television device can provide dialogue training for the user to improve the user's foreign language ability. The television device includes: a display component, a remote control signal receiving component, an audio output component, a network communication device, a voice acquisition component, a controller, etc. The television device may be the display device described above. For an understanding of the television device and its components, please refer to the relevant description of the display device in the previous text. For example, for an understanding of the display component, please refer to the display component in the display device in the previous text, for an understanding of the network communication device, please refer to the communication device in the display device in the previous text, etc.; for an understanding of the remote control, please refer to the description of the control device in the previous text. The same content will not be repeated here. The following mainly describes the specific technical solutions related to how the television device can provide users with better dialogue quality:

[0119] Regarding the voice collection component in the TV device: it is used to collect the current user's voice when the remote control signal receiving component receives the key signal corresponding to the target key in the remote control; the target key is a preset key for starting the TV voice recognition function.

[0120] The target button may be a physical button or a virtual button. The target button may be a dedicated button or a general button, such as a "confirm" or "OK" button, as long as the user can start the TV voice recognition function by pressing the target button.

[0121] In some possible embodiments, the user only needs to press the target button once, and the voice collection component can continue to collect the current user's voice until the user presses the target button again, and the voice collection component no longer collects the current user's voice.

[0122] In some possible embodiments, when the user holds down the target button and speaks, the voice collection component can collect the current user's voice. When the user no longer holds down and releases the target button, the voice collection component stops collecting the current user's voice.

[0123] like Figure 5 As shown, regarding the controller in the television device, it is configured to perform steps S101 to S106, specifically as follows:

[0124] Step S101: When the display component displays the first interface and the voice collection component collects the current user's voice, voiceprint recognition is performed on the current user's voice, and the voiceprint identifier corresponding to the current user is determined based on the voiceprint recognition result. Based on the voiceprint identifier, user feature information corresponding to the current user is extracted from the preset knowledge graph, and the current user's voice is converted into corresponding voice text, and the voice text is displayed through the first interface; the first interface is a preset interface for voice dialogue with a digital human.

[0125] The voiceprint identification may be an identification obtained by performing voiceprint recognition on the collected user language. Since different users have different voiceprints, different users have different corresponding voiceprint identifications.

[0126] In some possible embodiments, in order to provide users with foreign language oral dialogue services, a foreign language oral training or dialogue function may be pre-installed in the television device, in which the user can have a voice dialogue with a digital human in a foreign language. The opening or entry method of this function may be various, for example, a shortcut key may be set in the remote control, and when the user presses the shortcut key, the function is entered or opened. For another example, an interface option for opening the function may be set in the power-on interface or the initial default interface of the television device, such as when the user selects the interface option through the remote control and confirms it, the function is entered or opened, etc. The function may have a corresponding interface, and the television device displays the corresponding interface to the user.

[0127] The first interface may be an interface corresponding to the function of providing foreign language oral training or voice dialogue to the user on the television device, and the content of the dialogue between the user and the digital human may be displayed in the interface. In some possible embodiments, the image of the digital human is also displayed in the first interface, and the image of the digital human may be determined based on the user's selection operation, or may be determined based on relevant configuration information, or may be determined by a large model from a plurality of pre-set images to be suitable for the current scene.

[0128] In some possible embodiments, the texts or words displayed in the first interface are all in the same non-Chinese language, such as English.

[0129] In some possible embodiments, converting the current user voice into the corresponding language may be accomplished on a television device (locally) or by utilizing a server (cloud).

[0130] Step S102: Initiate a first request to a first server via a network communication device; the first request carries a dialogue text request instruction, voice text and user feature information.

[0131] Among them, the knowledge graph can be pre-constructed information involving user voiceprint identification and user characteristics, such as user's name, hobbies, family relationships, etc.

[0132] Among them, the conversation text request instruction is an instruction used to instruct the first server to perform related actions. For example, the conversation text request instruction of the present application is used to instruct the first server to generate and return the corresponding conversation text based on the voice text and user feature information.

[0133] For the understanding of the first server, please refer to the above description of “server 400”.

[0134] In some possible embodiments, the knowledge graph may be a user feature-related graph built around a voiceprint identifier. When the voiceprint identifier is determined, corresponding user feature information may be extracted based on the determined voiceprint identifier.

[0135] In some possible embodiments, the knowledge graph can be updated based on the conversation records between the user and the television device and the digital human.

[0136] Exemplarily, user feature information corresponding to the current user can be extracted from the knowledge graph based on the voiceprint identification, and then the user feature information, voice text and corresponding dialogue text request instructions can be packaged into a first request, and the first request can be initiated to the first server.

[0137] Step S103: receiving a dialog text returned by the first server; wherein the dialog text is generated based on the first request and a generative model preset in the first server.

[0138] Among them, the dialogue text can be the text used to respond to the current user's voice, and it is also the data basis for the digital human to have a verbal dialogue with the user, thereby forming a dialogue exchange with the user in terms of content.

[0139] The generative model may be an artificial intelligence model preset in the first server, which has the function of generating text from words and can generate a dialogue text according to the first request. For example, the voice text in the first request includes "how are you?" and the user feature information includes the user name "Tom", then the generative model can generate "Hi, Tom, I'm fine, and you?" accordingly.

[0140] In some possible embodiments, user characteristic information can be integrated into the prompt information of the generative model, for example, a prompt word is generated based on the current user's voice text, user characteristic information and dialogue text request instructions to prompt the generative model to generate more accurate dialogue text based on the voice text.

[0141] After the above steps, data for replying to users can be obtained, that is, the dialogue text, but this is in written form and is the data basis for the digital human to have a verbal dialogue with the user. In order to complete the verbal dialogue with the user, it is also necessary to obtain the digital human's audio data, that is, to provide a dialogue form of text and audio at the same time. This can be achieved through the following steps.

[0142] Step S104: Initiate a second request to a second server via a network communication device; the second request carries a text-to-speech instruction and a conversation text.

[0143] The text-to-speech instruction may be used to indicate the content of a specific request to the second server, that is, to request that the conversation text be converted into speech and returned.

[0144] Exemplarily, the received conversation text and text-to-speech instruction may be packaged and organized into a second request, and the second request may be initiated to the second server.

[0145] Step S105: receiving the conversation voice data returned by the second server; the conversation voice data is generated based on the second request and the text-to-speech service preset in the second server.

[0146] The text-to-speech service may be a service preset in the second server for converting text into speech.

[0147] For the understanding of the second server, please refer to the above description of “server 400”.

[0148] In some possible embodiments, the second server and the first server may be the same server that can provide both text-to-speech service and conversation text generation service. Exemplarily, the server may be preset with a large model that has both text-to-speech function and conversation text generation function.

[0149] Step S106: Send the dialogue text to the display component, and send the dialogue voice data to the audio output component and the preset digital human driving system respectively, so as to display the dialogue text through the first interface, output the dialogue voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the dialogue voice data through the digital human driving system.

[0150] Among them, the digital human can be understood as a virtual character image displayed on a television device. Accordingly, the digital human driving system can be a system used to drive the digital human. For example, driving the digital human to make actions can be understood in a broad sense. It can be actions with large changes such as waving and bowing, or micro-movements such as blinking, tilting the head, smiling, crying, yawning, etc. It can be body movements or facial movements, etc.

[0151] In some possible embodiments, user voice, voice text, dialogue text, and dialogue voice data may correspond to the same non-Chinese language, such as English. In some possible embodiments, based on the aforementioned same non-Chinese language, corresponding translation content may also be provided, for example, while displaying the English dialogue text, the corresponding Chinese translation text is displayed.

[0152] In some possible embodiments, after the dialogue text and dialogue voice data are obtained through the aforementioned steps, the dialogue text can be displayed through the display component for the user to read, and the corresponding audio can be played through the audio output component - that is, the dialogue voice data is output to drive the digital human to make corresponding changes.

[0153] In this embodiment, when the TV device enters the digital human oral speech training function page, when the user operates the target button in the remote control, it indicates that the voice recognition function of the TV device needs to be activated to conduct a foreign language oral dialogue with the digital human. In this regard, the TV device collects the current user voice through the voice collection component, and converts the current user voice into voice text through the controller and displays it on the first interface so that the user can know the user speech content captured by the TV device; at the same time, the controller performs voiceprint recognition to determine the corresponding voiceprint identifier, and extracts the corresponding user feature information from the knowledge graph according to the voiceprint identifier. The controller of the TV device sends a first request to the first server to obtain the dialogue text for the user speech content. Since the dialogue text is generated based on the first request and the generative model preset in the first server, and the first request carries the voice text and user feature information, a dialogue text with better quality can be obtained. The controller of the TV device sends a second request to the second server to obtain dialogue voice data, and the obtained dialogue voice data is consistent with the content of the dialogue text generated by the generative model. Finally, the dialogue text is displayed through the display component of the television device, the dialogue voice data is output through the audio output component, and the digital human is driven by the digital human driving system to make corresponding actions. Based on the three-end collaboration of the controller of the television device, the generative model in the first server, and the text-to-speech service in the second server, the foreign language dialogue practice between the user and the digital human is realized. Since the dialogue text of the digital human for the user's speech content is generated based on the generative model and user feature information, it can provide users with personalized dialogue content, and the quality of the dialogue text is better. At the same time, through the text-to-speech service and the digital human driving system, the dialogue text, digital human action, and digital human voice are coordinated and consistent, so it can provide a better foreign language oral dialogue experience.

[0154] In one of the embodiments, the controller in the aforementioned embodiment is further configured to: when the remote control signal receiving component receives a confirmation instruction from the remote control, if the current display focus in the display component is located on a preset dialogue mode option, control the display component to display a first interface; or, when the remote control signal receiving component receives a confirmation instruction from the remote control, if the current display focus in the display component is located on a preset picture guessing mode option, control the display component to display a second interface; the second interface is a preset interface for voice guessing with a digital human;

[0155] The first interface and the second interface are interfaces included in the target application; the target application is an application including a digital human voice dialogue function.

[0156] Among them, the confirmation instruction issued by the remote control can be a command issued by the user through the remote control, and the corresponding remote control operation is not restricted. For example, the user can click the "OK / Confirm" button on the remote control, of course, it can also be other buttons. The buttons can be physical, virtual, etc., there is no restriction here.

[0157] Exemplarily, when the user presses a confirmation button in the remote control, the remote control issues a confirmation instruction.

[0158] Among them, as mentioned above, the television device can be pre-installed with a digital human dialogue function, and the interface of the digital human dialogue function includes mode options, which specifically include dialogue mode options and picture guessing mode options. The dialogue mode option is used to enter the interface for voice chat with the digital human, and the picture guessing mode option is used to enter the interface for interactive guessing pictures with the digital human, and practice oral language during the interaction.

[0159] The following is an explanation of the guessing picture mode. For example, the TV device can display a picture and a text description and / or play related audio to guide the user to answer the content of the picture through voice and give feedback on the correctness of the user's answer. This process can use the same foreign language, thereby helping the user improve his foreign language speaking ability and vocabulary in the process.

[0160] In some possible embodiments, the conversation mode option and the guessing picture mode option may be displayed simultaneously in the same interface of the television device, or only the conversation mode option may be displayed in the voice conversation mode and only the guessing picture mode option may be displayed in the guessing picture mode. When the user selects the currently displayed mode option, another option may pop up for the user to select.

[0161] The current display focus may be a position coordinate or a position coordinate range of the screen displayed by the display component. For example, the current display focus may be a selection box of an operating system in a television device, and the color of the selection box may be white, yellow, etc. to highlight the selection box; for another example, the current display focus may be a highlighted, enlarged, or other highlighted portion of the display screen.

[0162] When the current display focus is on the preset dialogue mode option, if a confirmation instruction from the remote controller is received, the controller in the television device will control the display component to display the first interface to the user.

[0163] In some possible embodiments, the voice dialogue interface corresponding to the dialogue mode option may be provided by the target application, and the voice guessing picture interface corresponding to the guessing picture mode option may be provided by the same target application. Accordingly, the first interface may be an interface included in the target application, and the second interface is an interface included in the same target application; the target application is an application including a digital human voice dialogue function.

[0164] In this embodiment, when the current display focus in the display component is located on the preset dialogue mode option or the guessing picture mode option, and the remote control signal receiving component receives the confirmation command issued by the remote control, the display component is controlled to display the corresponding first interface or the second interface, thereby enabling the user to select to enter the corresponding interactive mode through the remote control, and practice speaking with the digital human through a variety of different interactive modes, which is conducive to enriching the user experience.

[0165] In one embodiment, if Figure 6 As shown, the controller in the above embodiment is further configured to execute steps S201 to S204:

[0166] Step S201: Based on the historical user voices collected by the voice collection component, the voiceprint information corresponding to each historical user is analyzed, and the historical user voices are converted into corresponding historical voice texts.

[0167] The historical user may be a user who used the TV device in the past; accordingly, the historical user voice is the user voice collected by the voice collection component in the past. For example, the historical user voice may be the user voice collected when the TV device enters the first interface or the second interface.

[0168] Step S202: Determine the voiceprint identifier of each historical user based on the voiceprint information corresponding to the historical users.

[0169] Exemplarily, the voiceprint identifier corresponding to the same user is the same and unique, and the voiceprint identifiers corresponding to different users are different. If there are multiple historical users who have talked with the digital human, the voiceprint identifiers corresponding to the multiple users can be analyzed and obtained.

[0170] Step S203: extracting entity information corresponding to predefined entity words and association relationships between different entity information from the historical voice text; the predefined entity words include one or more of name, gender, age, interest, and character relationship.

[0171] The entity information may be a name, personal information, hobbies, etc.; correspondingly, the association relationship may represent the relationship between different entity information.

[0172] In some possible embodiments, artificial intelligence related technologies, such as large language models, can be used to analyze historical speech texts to obtain entity information and association relationships therein.

[0173] Step S204: construct a knowledge graph based on the voiceprint identifier, entity information and association relationship of at least one historical user; the nodes in the knowledge graph represent the voiceprint identifier and entity information, and the edges between the nodes represent the association relationship.

[0174] In some possible embodiments, the voiceprint identifier itself is also used as a node. Two nodes representing voiceprint identifiers are not directly connected, but can be indirectly connected through one or more nodes representing entity information.

[0175] like Fig.14 As shown, a possible knowledge graph is provided, in which the nodes representing voiceprint identification are "voiceprint id 1" and "voiceprint id 2"; the nodes representing entity information are "Alice", "Tom", "badminiton", etc.; for the representation of association relationships, for example, the name of "voiceprint id 1" is "Tom", and the Friend of "voiceprint id 1" is "Alice", that is, Tom's friend is Alice, and for another example, the Love of "voiceprint id 1" is "badminiton", that is, Tom loves to play badminton, etc.

[0176] In this embodiment, the television device processes the user's voice collected in the past to determine the user's voiceprint identification, and extracts entity information and association relationships from the historical voice text, and then constructs a knowledge graph based on the voiceprint identification, entity information and association relationships. This enables the knowledge graph to form a comprehensive information network about the user's name, gender, interests, relationships with people, etc. based on the voiceprint identification. Based on the knowledge graph, more comprehensive, three-dimensional and accurate user feature information can be provided for subsequent steps.

[0177] In one of the embodiments, the "extracting entity information corresponding to pre-defined entity words from historical voice text, and the association relationship between different entity information" in the aforementioned embodiment may include: initiating a third request to the first server through a network communication device; the third request carries historical voice text and entity information extraction instructions; receiving the entity information returned by the first server and the association relationship between different entity information; the entity information and the association relationship are generated based on the third request, a preset entity word set in the first server and a generative model.

[0178] As mentioned above, entity information and association relationships can be extracted from historical speech texts. To improve the accuracy and efficiency of extraction, a large model can be used for extraction. In this embodiment, as mentioned above, a generative model can be configured in the first server, so that the entity information and association relationships can be extracted by interacting with the first server.

[0179] Exemplarily, the historical voice text and entity information extraction instructions can be packaged into a third request and sent to the first server. After receiving the third request, the first server parses it, calls the built-in generative model to generate corresponding entity information and association relationships, and returns it to the TV device.

[0180] In some possible embodiments, the entity word set may be preset in the first server or other servers, and when necessary, the first server may directly obtain or obtain from other servers. Of course, the entity word set may also be preset in the television device, and when necessary, the television device may send the entity word set to the first server, for example, by also putting the entity word set in the third request. In order to improve the efficiency of the use of each hardware in the television device and reduce the hardware requirements, the entity word set may be preset in the first server.

[0181] In some possible embodiments, the entity word set may be updated, such as by adding new entity words.

[0182] In this embodiment, by sending a request to the first server, the entity information returned by the first server and the association relationship between different entity information can be obtained. Since the entity information and the association relationship are generated by the preset entity word set and generative model in the first server, the entity information and the association relationship can more accurately reflect the actual situation of each historical user of the TV device.

[0183] As described in the above embodiments, the knowledge graph can be constructed based on historical user voices. In order to ensure its timeliness, the knowledge graph can also be updated with the voice information of the current user. In one embodiment, the controller in the above embodiment is also configured to: after initiating a first request to the first server through a network communication device, receive the entity information contained in the voice text of the current user voice and the association relationship between different entity information returned by the first server based on the first request, and update the knowledge graph.

[0184] In this embodiment, for the current user, in the process of obtaining the conversation text of the digital person through the first server, the entity information contained in the voice text of the current user's voice and the association relationship between different entity information are returned by the first server to update the knowledge graph. For example, if the current user mentions a new family member in the voice, the name of the family member can be updated to the knowledge graph as a new node, and the association relationship between the node and the voiceprint identifier corresponding to the current user is added as a family member. This enables the knowledge graph to be updated in real time and dynamically, thereby ensuring the accuracy and real-time nature of the knowledge graph.

[0185] As described in the above embodiments, there is a node representing a voiceprint identifier in the knowledge graph. At the same time, the voiceprint identifier corresponding to the current user can be extracted through the steps in the above embodiments. Therefore, the node representing the voiceprint identifier can be found in the knowledge graph, and the user feature information corresponding to the current user can be extracted based on the node. In one embodiment, the "extracting user feature information corresponding to the current user from the preset knowledge graph based on the voiceprint identifier" in the above embodiment can include: determining the node corresponding to the voiceprint identifier from the preset knowledge graph as the target node; determining at least one associated node of the target node, and there is an edge between the associated node and the target node; obtaining the entity information represented by the associated node and the association relationship between the associated node and the target node as the user feature information corresponding to the current user.

[0186] Among them, the understanding of the target node can be combined with the above-mentioned embodiment Fig.14 For example, if the node corresponding to the voiceprint identifier is "voiceprint id 1", then "voiceprint id 1" can be used as the target node, and accordingly, "Tom" and the like can be associated nodes.

[0187] In some possible embodiments, there is an edge between the associated node and the target node, which may correspond to an associated relationship between the associated node and the target node. For example, the edge between "voiceprint id 1" and "Tom" corresponds to an associated relationship of "name", that is, the name of "voiceprint id 1" is "Tom".

[0188] In this embodiment, the node corresponding to the voiceprint identifier in the knowledge graph is used as the target node, and the associated node of the target node is determined, so as to obtain the entity information represented by the associated node and the association relationship between the associated node and the target node, and then obtain the user feature information corresponding to the current user. Since the user feature information is determined based on the voiceprint identifier, this avoids the confusion of user feature information between different users; at the same time, the user feature information contains the entity information and association relationship of the target node and the associated node, so that the user feature information can reflect the more comprehensive user involved, which helps to provide better quality non-Chinese dialogue services.

[0189] In the aforementioned embodiment, the television device can provide a dialogue service to the user through the first interface. In addition to the dialogue service, the television device can also provide a voice guessing picture service, and the voice guessing picture can be non-Chinese, thereby helping to provide foreign language ability training services for users.

[0190] In one embodiment, the controller in the above embodiment is further configured to perform the following steps:

[0191] Step 1: When the display component switches from other interfaces to display the second interface, a preset number of images are extracted from a preset image database and cached; the preset number is an integer greater than 1; a picture is determined from the cached pictures as the first target picture, and a fourth request is initiated to the first server through a network communication device; the fourth request carries an opening text request instruction and the first target picture.

[0192] The preset picture database may be pre-set in the television device, and the controller may directly or indirectly extract a preset number of pictures from the preset picture database and cache them.

[0193] The case of switching from other interfaces to displaying the second interface indicates that the digital human initiates the first round of dialogue just after entering the second interface. Based on this, the opening text is the text of the first round, and the opening text request instruction can be an instruction representing the opening text requested from the first server. In the first interface and the second interface of the embodiment of the present application, the first round of dialogue is initiated by the digital human first.

[0194] In some possible embodiments, the preset image database may be updated by adding new image data locally or by interacting with a cloud server.

[0195] In some possible embodiments, a picture may be determined as the first target picture from the cached pictures by random selection.

[0196] In some possible embodiments, in order to avoid displaying the same image to the user through the second interface multiple times within a short period of time, after the first target image is determined, the selected image can be put back into the preset image database or no longer participate in subsequent steps; of course, an upper limit on the number of times the same image can be selected within a preset time range can also be set, such as being selected once within 10 minutes. When the upper limit is reached, it cannot be selected and enters a waiting period, such as 10 minutes. After the waiting period, it can be selected again, and the cycle repeats.

[0197] Step 2: Receive the opening text returned by the first server; the opening text is generated based on the fourth request and a generative model preset in the first server.

[0198] The opening text may be information used to explain and describe to the user that the second interface displayed by the television device is an interface for voice guessing with a digital person, so as to guide the user to participate in voice guessing.

[0199] Through the above steps, the opening text of the voice guessing picture can be obtained. In order to improve the quality of interaction with the user, the corresponding audio can be added at the same time. This can be achieved by the following steps:

[0200] Step 3: Initiate a fifth request to the second server via the network communication device; the fifth request carries the text-to-speech instruction and the opening text.

[0201] Step 4: Receive the opening voice data returned by the second server; the opening voice data is generated based on the fifth request and the text-to-speech service preset in the second server. The opening voice data is the voice data initiated by the digital human in the first round.

[0202] Step 5: Send the first target image and the opening text to the display component, and send the opening voice data to the audio output component and the digital human driving system respectively, so as to display the first target image and the opening text through the second interface, output the opening voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the opening voice data through the digital human driving system; wherein the opening text and the opening voice data correspond to the same non-Chinese language.

[0203] In this embodiment, when the television device enters the voice guessing picture mode, that is, the display component switches from other interfaces to display the second interface, extracts multiple pictures from the preset picture database at one time and caches them, thereby determining the first target picture, and requests the opening text from the first server, which is generated based on the preset generative model, so that a better quality opening text can be obtained. At the same time, the television device requests text-to-speech from the second server, so that the opening voice data corresponding to the opening text returned by the second server can be received. Finally, the opening text is displayed by the display component, the opening voice data is output by the audio output component, and the digital human is driven by the digital human driving system to make corresponding actions, thereby realizing the collaborative provision of foreign language dialogue services to users from the three aspects of opening text, digital human action, and digital human voice. Since the contents of the three aspects of opening text, digital human action, and digital human voice are coordinated and consistent, better quality non-Chinese dialogue can be provided.

[0204] In one embodiment, the controller in the above embodiment is further configured to perform the following steps:

[0205] Step 1: When the display component displays the second interface and the voice collection component collects the current user's voice, the current user's voice is converted into a corresponding picture guessing answer text, and the picture guessing answer text is displayed through the second interface.

[0206] It can be seen from the above embodiments that when the display component displays the second interface, it can indicate that the TV device currently provides the user with voice guessing picture related services. The TV device displays the first target picture, the opening text, and plays the opening voice data to the user, and the user can respond to this, for example, the user speaks the content in the first target picture. In this regard, the TV device can collect the current user's voice through the voice collection component, and convert the current user's voice into the corresponding guessing picture answer text, and display the guessing picture answer text through the second interface.

[0207] When the guessing picture answer text is obtained, the digital human service of the TV device can make a corresponding response, and the response content can be obtained by the TV device from the cloud server. This can be achieved through the following steps:

[0208] Step 2: Initiate a sixth request to the first server via a network communication device; the sixth request carries a result determination request instruction, a guessing picture answer text, and a first target picture.

[0209] Step three: receiving the determination result text returned by the first server; the determination result text is generated based on the sixth request and the generative model preset in the first server.

[0210] The judgment result text may be a text indicating whether the guess or description about the first target image is accurate. In addition, it may also include a text describing the subsequent interactions related to the judgment result. For example: The guess is correct, let's move on to the next guess.

[0211] Step 4: Initiate a seventh request to the second server via the network communication device; the seventh request carries a text-to-speech instruction and a determination result text.

[0212] Step 5: receiving the image guessing result voice data returned by the second server; the image guessing result voice data is generated based on the seventh request and the text-to-speech service preset in the second server.

[0213] Step 6: Determine another picture from the cached pictures as the second target picture, wherein the second target picture is different from the first target picture.

[0214] Regarding “determining another picture from the cached pictures”, reference may be made to the aforementioned related technical solutions for “determining a picture from the cached pictures”, such as setting an upper limit on the number of times selected.

[0215] In some possible embodiments, determining another picture from the cached pictures and the aforementioned step of sending a request to the first server or the second server may be performed synchronously or asynchronously.

[0216] Step 7: Obtain a preset prompt text and its corresponding voice data; the prompt text and its corresponding voice data are used to prompt the user to continue guessing the picture.

[0217] In some possible embodiments, the prompt text and its corresponding voice data may be pre-stored in the television device.

[0218] Step 8: Send the guessing result text, the second target image and the prompt text to the display component, and send the combined data of the guessing result voice data and the voice data corresponding to the prompt text to the audio output component and the digital human driving system respectively, so as to display the guessing result text, the second target image and the prompt text through the second interface, output the combined data through the audio output component, and drive the preset digital human image to perform actions corresponding to the current voice data through the digital human driving system; wherein, the guessing result text and the guessing result voice data correspond to non-Chinese languages.

[0219] In this embodiment, by displaying the second interface on the display component and collecting the current user's voice, a corresponding guessing picture answer text is generated according to the current user's voice; according to the guessing picture answer text, the judgment result text and the guessing picture result voice data corresponding to the judgment result text are requested from the server, and another picture is determined from the cached pictures as the second target picture; and the preset prompt text and its corresponding voice data are obtained. Finally, the judgment result text, the second target picture and the prompt text are displayed through the display component, the guessing picture result voice data and the voice data corresponding to the prompt text are output through the audio output component, and the action corresponding to the current voice data is performed by the digital human image. This enables the digital human service of the television device to provide the user with a voice guessing picture service, and to make corresponding feedback according to the user's answer, such as whether the guessing picture is accurate and guessing the next picture. At the same time, since the judgment result text is generated by the generative model in the first server, the accuracy of the judgment result text is better.

[0220] In one of the embodiments, the first server and the second server are servers deployed in the cloud, and the first server and the second server are the same server or different servers; the generative model preset in the first server is a model based on the Transformer architecture and has a text-generating text function.

[0221] Among them, the Transformer architecture can be an architecture of an artificial intelligence model, which can include an input part, an encoder, a decoder, and an output part, etc.

[0222] Among them, the text-generating function can be the ability to generate another text from one paragraph of text. In terms of content, the two paragraphs of text can correspond and connect. For example, if one paragraph of text is "Hi, I am Tom", then the other paragraph of text generated can be "Nice to meet you, Tom, I'm Alice."

[0223] In this embodiment, the first server and the second server are deployed in the cloud, so the TV device can interact with the first server and the second server through a network communication device, thereby obtaining corresponding data to provide foreign language dialogues for users. At the same time, the generative model preset in the first server is a model based on the Transformer architecture and has a text-generating text function, so the TV device can obtain data such as dialogue texts with better quality generated based on the model from the first server, thereby providing users with better quality foreign language dialogues.

[0224] In some possible embodiments, the current user's voice can be converted into corresponding text locally on the television device without the need to interact with the server to achieve voice-to-text conversion, thereby improving processing efficiency; for the text returned by the first server, such as the opening text, judgment result text, dialogue text, etc., the second server can be used to achieve text-to-speech conversion, thereby obtaining more accurate voice, while also meeting the digital human driving system's demand for corresponding data.

[0225] In some possible embodiments, when requesting text from the first server, corresponding prompt words may be constructed based on the text type currently requested to guide the generative model to generate data with better quality, such as conversation text.

[0226] In addition to the television device provided in the above embodiment, a session processing method is provided below by way of an embodiment. The method can be applied to a controller of a television device, as described below.

[0227] In an exemplary embodiment, the method applied to the controller of the television device in the above embodiment includes:

[0228] When the display component of the television device displays the first interface and the voice collection component of the television device collects the voice of the current user, voiceprint recognition is performed on the voice of the current user, a voiceprint identifier corresponding to the current user is determined based on the voiceprint recognition result, and user feature information corresponding to the current user is extracted from a preset knowledge graph based on the voiceprint identifier, and the current user voice is converted into a corresponding voice text, and the voice text is displayed through the first interface; the first interface is a preset interface for conducting a voice dialogue with a digital human;

[0229] Initiating a first request to a first server through a network communication device; the first request carries a dialogue text request instruction, a voice text, and user feature information; receiving a dialogue text returned by the first server; wherein the dialogue text is generated based on the first request and a generative model preset in the first server;

[0230] Initiating a second request to a second server through a network communication device; the second request carries a text-to-speech instruction and a conversation text;

[0231] Receiving the conversation voice data returned by the second server; the conversation voice data is generated based on the second request and the text-to-speech service preset in the second server;

[0232] The conversation text is sent to the display component, and the conversation voice data is sent to the audio output component of the television device and the preset digital human driving system respectively, so as to display the conversation text through the first interface, output the conversation voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the conversation voice data through the digital human driving system.

[0233] For understanding of the technical solution of this embodiment, reference may be made to the relevant parts of the aforementioned embodiments, which will not be repeated here.

[0234] In this embodiment, when the TV device enters the function page related to the digital human oral speech training, when the user operates the target button in the remote control, it indicates that the voice recognition function of the TV device needs to be activated to conduct a foreign language oral dialogue with the digital human. In response, the controller in the TV device converts the current user voice into voice text and displays it on the first interface so that the user can know the user speech content captured by the TV device; at the same time, the controller performs voiceprint recognition to determine the corresponding voiceprint identifier, and extracts the corresponding user feature information from the knowledge graph according to the voiceprint identifier. The controller sends a first request to the first server through a network communication device to obtain a dialogue text for the user speech content. Since the dialogue text is generated based on the first request and the generative model preset in the first server, and the first request carries the voice text and user feature information, a dialogue text with better quality can be obtained. The controller sends a second request to the second server through a network communication device to obtain dialogue voice data, and the obtained dialogue voice data is consistent with the content of the dialogue text generated by the generative model. Finally, the controller sends the above-mentioned dialogue text to the display component of the TV device to display the dialogue text, sends the dialogue voice data to the audio output component to output the dialogue voice data, and sends the dialogue voice data to the digital human driving system to drive the digital human to make corresponding actions, thereby realizing the foreign language dialogue practice between the user and the digital human based on the three-end collaboration of the controller of the TV device, the generative model in the first server, and the text-to-speech service in the second server. Since the dialogue text of the digital human for the user's speech content is generated based on the generative model and user feature information, it can provide users with personalized dialogue content, and the quality of the dialogue text is better. At the same time, through the text-to-speech service and the digital human driving system, the three aspects of dialogue text, digital human action, and digital human voice are coordinated and consistent, so it can provide a better foreign language oral dialogue experience.

[0235] In an exemplary embodiment, a television device is provided, which is equipped with an English speaking coach to provide English speaking training services to users. The English speaking coach can be a virtual character image presented in the television device, that is, a digital human or a 3D digital human. The driving principle of the 3D digital human mainly includes speech recognition, motion capture and expression recognition technology. These technologies enable the digital human to respond to actions and expressions according to the user's input and achieve interaction with the user. Speech recognition technology can convert the user's speech into text, motion capture technology obtains the user's action data, and expression recognition technology recognizes the user's expression, thereby driving the digital human to perform corresponding actions and expression reactions.

[0236] The current English chat products are based on traditional chat models. The conversation content is stiff and not smooth enough, and pronunciation and grammatical errors in the conversation process cannot get timely feedback. The technical solution of this embodiment is based on speech recognition, large model text generation, speech evaluation technology, 3D digital human technology, voiceprint recognition technology and the construction and use of conversation-related knowledge graphs, etc., which makes the conversation more fluent and natural. The body language of 3D digital people during the conversation makes the conversation more human, and the conversation records allow users to clearly understand their weaknesses and learning progress. The technical solution of this embodiment mainly involves three parts: 1. Dialogue practice; 2. Picture guessing game; 3. Knowledge graph. The specific contents are as follows:

[0237] 1. Dialogue Practice

[0238] like Figure 7 As shown, a possible timing for realizing dialogue practice / chat interaction is provided, wherein the digital human central control-related content corresponds to the controller-related content in the aforementioned embodiment, the large model service and TTS service (Text-to-Speech Service) can correspond to the first server and the second server-related content in the aforementioned embodiment, and the digital human service can correspond to the digital human drive system-related content in the aforementioned embodiment.

[0239] The main functions of dialogue practice include English dialogue, grammar correction, advanced expression (prompting users to express more advanced methods), dialogue duration statistics, word count statistics, and pronunciation fluency score statistics. The specific process is described as follows:

[0240] 1. The user initiates a voice request, and the TV terminal calls the ASR (Automatic Speech Recognition) service to perform voice recognition and pronunciation evaluation to obtain the corresponding text and pronunciation score;

[0241] 2. The TV requests the Digital Human Central Control Service, which calls the Big Model Service to obtain the dialogue results. At the same time, the Big Model is used asynchronously to obtain the grammatical error correction results and advanced expression results, which are then sent to the TV terminal for display.

[0242] 3. Based on the dialogue results of the large model, the digital human central control calls the TTS service to obtain audio, uses the audio to call the digital human service to obtain driving data, and sends it to the TV terminal to drive the digital human; the interaction process between the digital human central control and the large model and the digital human service is streaming, without blocking, and the digital human's dialogue expression is more fluent;

[0243] 4. During the interaction process, the conversation duration, conversation content, user pronunciation score and other data are saved and analyzed. Figure 8As shown, a possible form of statistical data is provided.

[0244] 2. Picture Guessing Game

[0245] The guessing picture game corresponds to the voice guessing picture mode in the above-mentioned embodiment. The guessing picture game adds the function of guessing pictures to learn English words and phrases on the basis of dialogue practice. The interaction logic here also includes the interaction process with the big model service, TTS service, and digital human driving service. For relevant details, please refer to the above-mentioned embodiment and the attached Fig. 9 As shown, no further details will be given. Fig. 9 As shown in Figure 1, a possible sequence of the guessing game is provided. Fig.10 As shown, a possible interface display of the guessing picture game is provided, which may correspond to the second interface in the aforementioned embodiment. The "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" shown in the figure may be an opening text in a foreign language, such as "Hello, let's start the game of guessing the English content from pictures", i.e., Hello, let's start the game of guessing the English content from pictures. Regarding the first interface in the aforementioned embodiment, the overall framework may also refer to Fig.10 , but the specific content displayed can be different. For example, the "topic" in the picture can be displayed in dialogue mode, there can be no picture in the dialog box, the opening text can be different, etc. The specific process of the guessing picture game is as follows:

[0246] 1. Enter the guessing picture function. The TV actively requests the digital human central control to trigger the card generation logic. The digital human central control randomly obtains 100 card information from the preset database and puts them into redis (Remote Dictionary Server), that is, extracts a preset number of pictures from the preset picture database and caches them; randomly obtains 1 picture information from redis, that is, determines the first target picture, assembles the picture information into prompt, and asynchronously requests the large model service to obtain the opening remarks, which are returned to the TV terminal for display, that is, the first target picture and the opening text are displayed through the second interface, and the opening voice data is output through the audio output component, and the preset digital human image is driven by the digital human driving system to perform actions corresponding to the opening voice data.

[0247] 2. The user answers the card content, assembles the user answer and the original card information into a prompt, asynchronously requests the big model service, obtains the judgment result, that is, the judgment result text; at the same time, asynchronously obtains a card information assembly result from redis, that is, determines the second target image. If there is no data in redis, it is retrieved from the database and put into redis, and sent to the TV terminal for display. That is, the second target image and the guessing result text are displayed through the second interface, and the guessing result voice data is output through the audio output component, and the preset digital human image is driven by the digital human driving system to make actions corresponding to the guessing result voice data.

[0248] 3. The above process is repeated for the entire interactive logic of the guessing game.

[0249] 3. Knowledge Graph

[0250] In this embodiment, voiceprint technology is used to distinguish different users and establish corresponding knowledge graphs to form long-term memory capabilities, thereby helping to improve the quality of conversations. The specific solution is as follows:

[0251] 1. Use voiceprint recognition to distinguish different users. When users are talking, use voiceprint recognition to identify the user's voiceprint ID and report it to the digital human central control together with the recognized text;

[0252] 2. Use the big model service to extract entities and summarize content, and save the extracted entities in the graph database; the extracted entities include name, gender, age, interests, character relationships, etc. The TV terminal reports the voiceprint ID and text content to the digital human central control, which asynchronously extracts entity relationships from the text based on the original logic and builds a knowledge graph for the task portrait.

[0253] The following example illustrates the construction of a knowledge graph:

[0254] User 1 inputs the content by voice or other means: my name is Tom, that is, my name is Tom. Fig.11 As shown, a corresponding knowledge graph is provided, in which voiceprint id 1 corresponds to user 1.

[0255] User 1 inputs the content by voice or other means: I love to play badminton, that is, I love to play badminton. Fig.12 As shown, the corresponding knowledge graph is provided.

[0256] User 1 uses voice or other methods: My friendname isAlice, which means my friend's name is Alice. Fig.13 As shown, the corresponding knowledge graph is provided.

[0257] At this time, user 2 speaks, user 2: hi, I am Alice, that is, hi, I am Alice. Fig.14 As shown, the corresponding knowledge graph is provided.

[0258] In some possible embodiments, different colors may be used to represent different labels, such as yellow corresponding to the voiceprint ID, blue corresponding to the name, and so on.

[0259] 3. The extracted entities and relationships are saved in the image database. When the user initiates a conversation, the corresponding user information can be retrieved based on the voiceprint ID and the entities extracted from the user conversation, and integrated into the prompt of the large model, thereby achieving long-term memory based on the voiceprint ID and improving the quality of the conversation.

[0260] In multi-person conversation scenarios, different users can be distinguished based on their voiceprint IDs, and personalized conversations can be conducted based on the collected user information.

[0261] In some possible embodiments, the big model service may use a self-developed specific big model and adopt a server-side deployment method.

[0262] In some possible embodiments, the digital human 3D model is deployed in a terminal, and the terminal pre-downloads relevant components of the 3D model in advance according to the image selected by the user.

[0263] In some possible embodiments, the process of defining a digital human and issuing instructions is as follows:

[0264] 1. Define the digital human image

[0265] Different clothes, hairstyles, accessories, shoes, and props are combined into different digital human images. The same digital human image can be designed for different users and different emotions by changing the color of the clothes and the movements.

[0266] 2. Definition of Digital Human Action Animation

[0267] Define the digital human body motion animation parameters in advance, where the body motion parameters include: arm swing angle, knee flexion angle, facial expression parameters, etc. There are no restrictions here and they can be defined according to the actual situation of the digital human model.

[0268] 3. Establish the relationship between topics, emotions and digital human images

[0269] Different digital human images are used for different topics, such as sportswear and football props for digital humans in sports topic scenarios. Different digital human images are used according to different user emotions for the same chat topic. For example, if the user's emotion is happy in the chat mode, a happy digital human image is used. If the user's emotion is sad, an empathetic image is used to make the user feel the emotional resonance of the digital human. The user emotion column common represents that emotions are not restricted.

[0270] 4. According to the chat topic selected by the user and the user emotion recognized by the large model, the corresponding digital human image and action are matched, and the digital human image parameters, action parameters and broadcast language are input into the digital human driving system. The specific parameters of the digital human are obtained according to the digital human image parameters, and the lip shape parameters are obtained through the digital human facial driving algorithm. The image parameters, action parameters, face, expression parameters and voice are sent to the terminal, and the terminal drives the digital human model to make corresponding expression actions.

[0271] In some possible embodiments, there may be multiple images corresponding to the digital human, for example, a joyful image in a sports topic, an empathetic image in a chat, etc.; each digital human image may have its own digital human image ID; different digital human images may be different in appearance, for example, clothes, accessories, movements, etc. may be different.

[0272] In some possible embodiments, a matching digital human image may be determined from multiple digital human images based on the topic, the user's intention, emotion, etc., and displayed to the user.

[0273] The technical solution in this embodiment is based on speech recognition, large model text generation, speech evaluation technology, and 3D digital human technology, making the conversation more fluent and natural. The body language of the 3D digital human during the conversation makes the conversation more humanized, and the conversation record allows users to clearly understand their weaknesses and learning progress. By using voice intelligent evaluation, pronunciation scores are provided to users to facilitate the correction of user pronunciation; English conversations are combined with large models to make the conversation content more fluent and natural; 3D digital human driving technology is used to make the interaction process more humanized; and learning record statistics are provided so that users can understand the learning process more intuitively. At the same time, voiceprint technology is used to distinguish different users and build knowledge graphs. At the same time, large model technology and prompt engineering (prompt) technology are combined to achieve long-term memory based on voiceprint ID and improve the quality of conversation; in multi-person conversation scenarios, different users are distinguished based on voiceprint ID, and personalized conversations are conducted based on the collected user information.

[0274] In an exemplary embodiment, a session processing device is provided, which is applied to a controller of a television device, and includes:

[0275] The recognition module is used to perform voiceprint recognition on the current user's voice when the display component of the television device displays the first interface and the voice collection component of the television device collects the current user's voice, determine the voiceprint identifier corresponding to the current user based on the voiceprint recognition result, extract user feature information corresponding to the current user from the preset knowledge graph based on the voiceprint identifier; and convert the current user's voice into corresponding voice text, and display the voice text through the first interface; the first interface is a preset interface for conducting voice dialogue with a digital human;

[0276] A conversation content generation module is used to initiate a first request to a first server through a network communication device; the first request carries a conversation text request instruction, voice text and user feature information; receives the conversation text returned by the first server; wherein the conversation text is generated based on the first request and a generative model preset in the first server; initiate a second request to a second server through the network communication device; the second request carries a text-to-speech instruction and the conversation text; receives conversation voice data returned by the second server; the conversation voice data is generated based on the second request and a text-to-speech service preset in the second server;

[0277] The dialogue content output module is used to send the dialogue text to the display component, and send the dialogue voice data to the audio output component of the television device and the preset digital human driving system respectively, so as to display the dialogue text through the first interface, output the dialogue voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the dialogue voice data through the digital human driving system.

[0278] The modules in the above-mentioned session processing device may also have other functions, or the above-mentioned session processing device may also include other modules, so that the above-mentioned session processing device may have the function corresponding to the session processing method in the above-mentioned embodiment.

[0279] Each module in the above session processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0280] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0281] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0282] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0283] It should be understood that each step, flow, etc. in each embodiment as described above are not necessarily performed in sequence according to the order of indication. Unless there is clear explanation in this article, the execution of these steps and flow does not have strict order restriction, and these steps can be performed in other orders. Moreover, at least a part of the steps in each embodiment as described above may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a part of the steps or stages in other steps or other steps.

Claims

1. A television device, characterized in that: include: Display components; Remote control signal receiving component; Audio output component; Network communication devices; A voice collection component, used for collecting the current user's voice when the remote control signal receiving component receives a key signal corresponding to a target key in the remote control; the target key is a preset key for starting a voice recognition function of the television; The controller is configured as: When the display component displays the first interface and the voice collection component collects the voice of the current user, voiceprint recognition is performed on the voice of the current user, a voiceprint identifier corresponding to the current user is determined based on the voiceprint recognition result, and user feature information corresponding to the current user is extracted from a preset knowledge graph based on the voiceprint identifier, and the voice of the current user is converted into a corresponding voice text, and the voice text is displayed through the first interface; the first interface is a preset interface for conducting a voice dialogue with a digital human; Initiate a first request to a first server through the network communication device; the first request carries a dialogue text request instruction, the voice text and the user feature information; Receiving the conversation text returned by the first server; The dialogue text is generated based on the first request and a generative model preset in the first server; Initiate a second request to a second server through the network communication device; the second request carries a text-to-speech instruction and the conversation text; Receiving the conversation voice data returned by the second server; The conversation voice data is generated based on the second request and a text-to-speech service preset in the second server; The dialogue text is sent to the display component, and the dialogue voice data is sent to the audio output component and the preset digital human driving system respectively, so as to display the dialogue text through the first interface, output the dialogue voice data through the audio output component, and drive the preset digital human image through the digital human driving system to perform actions corresponding to the dialogue voice data.

2. The television device according to claim 1, characterized in that The controller is further configured to: When the remote control signal receiving component receives a confirmation instruction sent by the remote control, if the current display focus in the display component is located on a preset dialogue mode option, the display component is controlled to display the first interface; or, When the remote control signal receiving component receives a confirmation instruction sent by the remote control, if the current display focus in the display component is located on the preset guessing picture mode option, the display component is controlled to display a second interface; the second interface is a preset interface for guessing pictures with a digital person by voice; The first interface and the second interface are interfaces included in the target application; The target application is an application including a digital human voice dialogue function.

3. The television device according to claim 1, characterized in that The controller is further configured to: Based on the historical user voices collected by the voice collection component, analyzing the voiceprint information corresponding to each historical user, and determining the voiceprint identification of each historical user based on the voiceprint information; Convert the historical user voice into a corresponding historical voice text, and extract entity information corresponding to predefined entity words and association relationships between different entity information from the historical voice text; the predefined entity words include one or more of name, gender, age, interest, and character relationship; The knowledge graph is constructed based on the voiceprint identifier of at least one of the historical users, the entity information and the association relationship; the nodes in the knowledge graph represent the voiceprint identifier and the entity information, and the edges between the nodes represent the association relationship.

4. The television device according to claim 3, characterized in that The extracting entity information corresponding to the predefined entity words and the association relationship between different entity information from the historical speech text includes: Initiate a third request to the first server through the network communication device; the third request carries the historical voice text and entity information extraction instruction; Receive entity information returned by the first server and associations between different entity information; the entity information and the associations are generated based on the third request, a set of entity words preset in the first server, and a generative model.

5. The television device according to claim 4, characterized in that The controller is further configured to: After initiating a first request to the first server through the network communication device, the entity information contained in the voice text of the current user's voice and the association relationship between different entity information returned by the first server based on the first request are received, and the knowledge graph is updated.

6. The television device according to claim 3, characterized in that The extracting user feature information corresponding to the current user from a preset knowledge graph based on the voiceprint identifier includes: Determine a node corresponding to the voiceprint identifier from a preset knowledge graph as a target node; Determine at least one associated node of the target node, where there is an edge between the associated node and the target node; The entity information represented by the associated node and the associated relationship between the associated node and the target node are obtained as user feature information corresponding to the current user.

7. The television device according to claim 2, characterized in that The controller is further configured to: When the display component switches from other interfaces to display the second interface, extracting a preset number of images from a preset image database and caching them; The preset number is an integer greater than 1; Determine a picture from the cached pictures as a first target picture; Initiate a fourth request to the first server through the network communication device; The fourth request carries an opening text request instruction and the first target image; receiving an opening text returned by the first server; the opening text is generated based on the fourth request and a generative model preset in the first server; the opening text is a text of the first round; Initiate a fifth request to the second server through the network communication device; the fifth request carries the text-to-speech instruction and the opening text; receiving the opening voice data returned by the second server; the opening voice data is generated based on the fifth request and the text-to-speech service preset in the second server; the opening voice data is the voice data of the first round; Sending the first target image and the opening text to the display component, and sending the opening voice data to the audio output component and the digital human driving system respectively, so as to display the first target image and the opening text through the second interface, output the opening voice data through the audio output component, and drive the preset digital human image to perform actions corresponding to the opening voice data through the digital human driving system; The opening text and the opening voice data correspond to the same non-Chinese language.

8. The television device according to claim 7, characterized in that The controller is further configured to: When the display component displays the second interface and the voice collection component collects the current user's voice, convert the current user's voice into a corresponding picture guessing answer text, and display the picture guessing answer text through the second interface; Initiate a sixth request to the first server through the network communication device; the sixth request carries a result determination request instruction, the guessing picture answer text and the first target picture; Receiving the determination result text returned by the first server; The determination result text is generated based on the sixth request and a generative model preset in the first server; Initiate a seventh request to the second server through the network communication device; the seventh request carries a text-to-speech instruction and the determination result text; Receiving the image guessing result voice data returned by the second server; The guessing picture result voice data is generated based on the seventh request and the text-to-speech service preset in the second server; Determine another picture from the cached pictures as a second target picture; Get the preset prompt text and its corresponding voice data; The prompt text and its corresponding voice data are used to prompt the user to continue guessing the picture; The guessing result text, the second target image and the prompt text are sent to the display component, and the combined data of the guessing result voice data and the voice data corresponding to the prompt text are sent to the audio output component and the digital human driving system respectively, so as to display the guessing result text, the second target image and the prompt text through the second interface, output the combined data through the audio output component, and drive the preset digital human image to perform actions corresponding to the opening voice data through the digital human driving system; Wherein, the guessing picture result text and the guessing picture result voice data correspond to the non-Chinese language.

9. The television device according to any one of claims 1 to 8, characterized in that: The first server and the second server are servers deployed in the cloud, and the first server and the second server are the same server or different servers; The generative model preset in the first server is a model based on the Transformer architecture and having a text-generating text function.

10. A session processing method, characterized in that: Controllers for TV equipment, The method comprises: When the display component of the television device displays the first interface and the voice collection component of the television device collects the voice of the current user, voiceprint recognition is performed on the voice of the current user, a voiceprint identifier corresponding to the current user is determined based on the voiceprint recognition result, and user feature information corresponding to the current user is extracted from a preset knowledge graph based on the voiceprint identifier; and the voice of the current user is converted into a corresponding voice text, and the voice text is displayed through the first interface; the first interface is a preset interface for conducting a voice dialogue with a digital human; Initiate a first request to a first server through the network communication device; the first request carries a dialogue text request instruction, the voice text and the user feature information; Receiving a dialogue text returned by the first server; wherein the dialogue text is generated based on the first request and a generative model preset in the first server; Initiate a second request to a second server through the network communication device; the second request carries a text-to-speech instruction and the conversation text; Receiving conversation voice data returned by the second server; the conversation voice data is generated based on the second request and a text-to-speech service preset in the second server; The dialogue text is sent to the display component, and the dialogue voice data is sent to the audio output component of the television device and a preset digital human driving system respectively, so as to display the dialogue text through the first interface, output the dialogue voice data through the audio output component, and drive the preset digital human image through the digital human driving system to perform actions corresponding to the dialogue voice data.

Images