Display device, server and ancient poetry multimedia data display method

Abstract

The invention provides a display device, a server and an ancient poetry multimedia data display method. The method comprises the following steps: starting a first application in response to a received starting instruction for the first application; acquiring a first voice instruction which is input by a user and is used for displaying multimedia data on the first display page; in response to the first voice instruction, displaying text information corresponding to the input content of the first voice instruction in the input area; determining metadata of the ancient poetry matched with the text information according to the text information; generating multimedia data of the ancient poetry according to the metadata; after the multimedia data of the ancient poetry is generated, displaying a second display page of the first application; and in response to the selected multimedia display control, displaying the multimedia data on a third display page of the first application. According to the process, after the first application is started, the matched multimedia data of the ancient poetry can be generated according to the voice instruction, the expression form of the ancient poetry on the display equipment is increased, and the visual and auditory experience of the ancient poetry is provided for a user.

Description

Technical Field

[0001] The present application relates to the technical field of word processing, and in particular to a display device, a server, and a method for displaying multimedia data of ancient poetry. Background Art

[0002] The presentation of ancient poems on display devices is usually in the form of reading aloud. The display device can use text-to-speech technology to generate corresponding reading audio based on the content of ancient poems, so that the user can see the text of the ancient poems on the display and hear the synchronously played reading audio at the same time. However, in the form of reading aloud, the ancient poems cannot restore the real historical scenes depicted by the ancient poems, nor can they fully display the charm and artistic conception of the ancient poems, making it impossible for users to fully understand the ancient poems. In addition, the simple reading form only provides users with an auditory experience, ignoring the visual needs of users, making the user's experience of watching ancient poems based on display devices poor. Summary of the invention

[0003] The present application provides a display device, a server, and a method for displaying multimedia data of ancient poems. A first application can be started, metadata of the ancient poems can be obtained based on a control instruction input by a user for displaying multimedia data of ancient poems, multimedia data of the ancient poems can be generated according to the metadata of the ancient poems, and the multimedia data can be displayed on a display page of the first application. In this way, the expression forms of ancient poems on the display device can be increased, the audiovisual experience of ancient poems can be provided to users, and the user's experience of viewing ancient poems can be enriched.

[0004] In a first aspect, an embodiment of the present application provides a display device, comprising: a display; a controller coupled to the display and configured to: in response to receiving a start instruction for a first application, start the first application; the first application is at least used to display multimedia data of ancient poetry; the multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data; display a first display page of the first application, and obtain a first voice instruction for displaying multimedia data input by a user on the first display page, wherein the first display page at least includes an input area, a first guide word and a guide example for guiding the user to input the first voice instruction; in response to the first voice instruction, display text information corresponding to the input content of the first voice instruction in the input area; determine metadata of ancient poetry matching the text information according to the text information, wherein the metadata at least includes the title, author and content of the ancient poetry; generate multimedia data of ancient poetry according to the metadata; after generating the multimedia data of ancient poetry, display a second display page of the first application; wherein the second display page includes text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed; in response to selecting the multimedia display control, display the multimedia data on a third display page of the first application.

[0005] In the technical solution provided by the present application, the controller starts a first application for displaying multimedia data of ancient poems. After the first application is started, the controller obtains a first voice command for the user to select multimedia data for display on the first display page, identifies the first voice command and displays the corresponding text information in the input area, determines the metadata of ancient poems matching the input text information, and generates multimedia data corresponding to the ancient poems according to the metadata. After generating the multimedia data, the first application switches to the second display page, the second display page displays the text of the metadata of the ancient poems, and provides a multimedia display control for the user to select. When the user selects the multimedia display control on the second display page, the first application switches to the third display page to display the multimedia data selected by the user. In the first aspect of the process, the text information of the first voice command can be displayed in the input area, so that the user can confirm whether his command is correctly recognized, which enhances the user's sense of trust. In the second aspect, different contents (such as metadata, multimedia display control, multimedia data) are displayed respectively through multiple display pages of the first application, so that each page displays different key information, which is convenient for users to understand and operate. In the third aspect, multimedia data in various forms such as ancient poems image data, ancient poems audio data and ancient poems video data can be provided, enriching the learning experience of ancient poems and satisfying the preferences of different users.

[0006] In some feasible embodiments, the multimedia display control includes an audio and image display control; the controller executes in response to selecting the multimedia display control to display the multimedia data on the third display page of the first application, and is specifically configured as follows: in response to selecting the audio and image display control, all ancient poetry image data are arranged according to the description order of the text of the metadata to obtain an image carousel set; the ancient poetry audio data is played; according to the playback progress of the ancient poetry audio data, the first text in the text of the metadata displayed in the first display area of ​​the third display page is highlighted; the first text is the text in the metadata text corresponding to the playback progress; according to the playback progress and the first text, the ancient poetry image data in the image carousel set is automatically played in the second display area of ​​the third display page.

[0007] According to the above embodiment, the display device displays ancient poetry image data, ancient poetry audio data and ancient poetry metadata on the third display page of the first application. In this way, users can synchronously appreciate the audio data, text content and related images of ancient poetry, thereby obtaining a richer and more coherent multimedia experience.

[0008] In some feasible embodiments, the multimedia display control includes a video display control; the controller executes, in response to selecting the multimedia display control, displaying multimedia data on the third display page of the first application, and is also configured to: in response to selecting the video display control, display ancient poetry video data on the third display page.

[0009] According to the above embodiment, the display device displays the ancient poetry video data on the third display page of the first application, and the user can conveniently select and watch the multimedia video of the ancient poetry, thereby obtaining a richer and more intuitive learning experience.

[0010] In some feasible embodiments, the multimedia display control includes an image display control; the controller executes in response to selecting the multimedia display control to display the multimedia data on the third display page of the first application, and is specifically configured as follows: in response to selecting the image display control, all ancient poetry image data are arranged according to the description order of the text of the metadata to obtain an image carousel set; the text of the metadata is displayed in the third display area of ​​the third display page, and the image carousel set is displayed in the fourth display area of ​​the third display page; wherein, when displaying the ancient poetry image data in the image carousel set, the second text of the metadata is highlighted; the second text is the text in the text of the metadata corresponding to the ancient poetry image data being displayed.

[0011] According to the above embodiment, the display device displays the text of the ancient poetry metadata and the ancient poetry image data on the third display page, so that the user can synchronously appreciate the text content and related images of the ancient poetry, thereby obtaining a richer and more coherent multimedia experience.

[0012] In some feasible embodiments, the multimedia display control includes an audio playback control; the controller executes in response to selecting the multimedia display control to display multimedia data on the third display page of the first application, and is configured to: in response to selecting the audio playback control, play the ancient poetry audio data; according to the playback progress of the ancient poetry audio data, highlight the first text in the text of the metadata displayed in the fifth display area in the third display page; the first text is the text in the metadata text corresponding to the playback progress.

[0013] According to the above embodiment, the text of the ancient poetry metadata is displayed on the third display page, and the ancient poetry audio data is played, so that the user can synchronously listen to the ancient poetry audio data and see the currently playing verses highlighted, thereby obtaining a more coherent and rich multimedia experience.

[0014] In some feasible embodiments, the controller executes to generate multimedia data of ancient poetry according to metadata, and is specifically configured as follows: based on metadata, semantic understanding of ancient poetry is performed to obtain ancient poetry semantic data; based on metadata, background retrieval of ancient poetry is performed to obtain ancient poetry background data; using an ancient poetry image generation model, ancient poetry image data corresponding to the content of each sentence of ancient poetry is generated according to the ancient poetry semantic data and the ancient poetry background data; using an ancient poetry audio generation model, ancient poetry audio data is obtained according to the ancient poetry semantic data and the ancient poetry background data, the ancient poetry audio data includes background music audio corresponding to the ancient poetry background data, and text reading audio corresponding to the content of each sentence of ancient poetry with emotions consistent with the ancient poetry semantic data; generating ancient poetry video data according to the ancient poetry image data and the ancient poetry audio data, wherein the subtitles in the ancient poetry video data are determined according to the text of the metadata, the ancient poetry audio data is used as the audio of the ancient poetry display video data, the ancient poetry image data is automatically rotated to form a carousel video, and the carousel video is used as the video display of the ancient poetry video data.

[0015] According to the above embodiment, metadata can be processed to obtain ancient poetry semantic data and ancient poetry background data for analyzing ancient poetry. Afterwards, ancient poetry image data that conforms to the artistic conception, scene and emotion of ancient poetry can be generated based on the ancient poetry semantic data and ancient poetry background data, and background music that conforms to the creation background and the background of the times of ancient poetry can also be generated. Ancient poetry video data can also be generated based on the ancient poetry image data and ancient poetry audio data. In this way, the expression form of ancient poetry on the display device can be enriched to meet the visual and auditory experience of users.

[0016] In some feasible embodiments, the controller performs semantic understanding of ancient poetry based on metadata to obtain semantic data of ancient poetry, and is specifically configured as follows: preprocessing the content of the metadata to obtain a first content; the preprocessing includes word segmentation, removal of stop words and part-of-speech tagging; based on a semantic analysis model, analyzing the semantics of the first content to obtain first data; the first data is used to characterize the interpretation of the ancient poetry; determining a first keyword for the first content based on the first data; the first keyword includes words that describe scenes and scenery in the first content; determining a second keyword for the first content based on the first data; the second keyword includes words that describe emotions in the first content; and using the first data, the first keyword and the second keyword as semantic data of ancient poetry.

[0017] According to the above embodiment, the controller can perform semantic analysis on the metadata of the ancient poetry to determine the first data representing the interpretation of the ancient poetry, the first keyword data representing the scene of the ancient poetry, and the second keyword representing the emotion of the ancient poetry. The deep meaning of the poetry can be more comprehensively represented according to the first data, the first keyword, and the second keyword, providing data support for the subsequent generation of multimedia data.

[0018] In some feasible embodiments, the controller performs background retrieval of ancient poems based on metadata to obtain background data of the ancient poems, and is specifically configured as follows: retrieving the first background data of the author in the metadata, the first background data being used to represent background data related to the author; determining the creation time of the ancient poems based on the title and author in the metadata; determining the second background data of the ancient poems based on the creation time and the title; the second background data being used to represent the creation background data related to the author's creation of the ancient poems; determining the third background data of the ancient poems based on the creation time; the third background data being used to represent the era background data related to the creation time; and using the first background data, the second background data, and the third background data as the background data of the ancient poems.

[0019] According to the above embodiment, the controller can perform background search on the metadata of the ancient poetry to determine the first background data related to the author, the second background data related to the creation, and the third background data related to the era. Determining the background data of the ancient poetry can more comprehensively characterize the historical, cultural, and social background of the creation of the ancient poetry, and provide data support for the subsequent generation of multimedia data.

[0020] In some feasible embodiments, the controller executes in response to receiving a startup instruction for the first application, and before starting the first application, is also configured to: in response to receiving a second voice instruction for displaying ancient poetry, start the second application, and display the text of the metadata of the ancient poetry and the first control on the display page of the second application, wherein the second application is at least used to search for and display ancient poetry, and the first control is used to display the icon of the first application; maintain the metadata text displayed on the display page of the second application, obtain the user's startup instruction, and start the first application.

[0021] According to the above embodiment, displaying the icon of the first application on the display page of the second application allows the user to click the icon of the first application to start the first application, or to enter a voice command in the second application to start the first application. In this way, multiple ways of starting the first application are provided to meet the usage habits of different users.

[0022] In a second aspect, an embodiment of the present application provides a display device, comprising: a display; a communicator, configured to be communicatively connected to a server; a controller, coupled to the display and the communicator, and configured to: in response to receiving a start instruction for a first application, start the first application; the first application is at least used to display multimedia data of ancient poetry; the multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data; display a first display page of the first application, and obtain a first voice instruction input by a user for displaying multimedia data on the first display page, wherein the first display page at least includes an input area, a first guide word and a guide word for guiding the user to input the first voice instruction Example; in response to the first voice command, display text information corresponding to the input content of the first voice command in the input area; control the communicator to send text information to the server, so that the server obtains the metadata of the ancient poetry according to the text information and generates multimedia data of the ancient poetry; wherein the metadata at least includes the title, author and content of the ancient poetry; control the communicator to receive multimedia data from the server; display the second display page of the first application; wherein the second display page includes the text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed; in response to selecting the multimedia display control, display the multimedia data on the third display page of the first application.

[0023] In a third aspect, an embodiment of the present application provides a server, comprising: a communicator, configured to be connected to a display device for communication; a processor, coupled to the communicator, and configured to: control the communicator to receive text information from the display device, and obtain metadata of ancient poetry based on the text information; the metadata at least includes the title, author and content of the ancient poetry; generate multimedia data of the ancient poetry based on the metadata; control the communicator to send the multimedia data to the display device, so that the display device displays the multimedia data on the third display page of the first application.

[0024] In a fourth aspect, an embodiment of the present application provides a method for displaying multimedia data of ancient poetry, comprising: in response to receiving a startup instruction for a first application, starting the first application; the first application is at least used to display multimedia data of ancient poetry; the multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data; displaying a first display page of the first application, obtaining a first voice instruction for displaying multimedia data input by a user on the first display page, wherein the first display page at least includes an input area, a first guide word for guiding the user to input the first voice instruction, and a guide example; in response to the first voice instruction, displaying text information corresponding to the input content of the first voice instruction in the input area; determining metadata of ancient poetry matching the text information according to the text information, wherein the metadata at least includes the title, author and content of the ancient poetry; generating multimedia data of ancient poetry according to the metadata; after generating the multimedia data of ancient poetry, displaying a second display page of the first application; wherein the second display page includes text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed; in response to selecting the multimedia display control, displaying the multimedia data on a third display page of the first application.

[0025] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the second to fourth aspects mentioned above can refer to the beneficial effects in the first aspect and any optional implementation method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the hardware configuration of a display device provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of software configuration of a display device provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the application layer of a display device provided in an embodiment of the present application;

[0031] Figure 5 This is a schematic diagram of displaying ancient poems based on a voice assistant;

[0032] Figure 6A schematic diagram of displaying ancient poetry text on a display device;

[0033] Figure 7 A schematic diagram showing the display device playing ancient poetry voice data;

[0034] Figure 8 A schematic diagram of a process of displaying multimedia data based on a voice command for displaying multimedia data of ancient poetry provided by a display device according to an embodiment of the present application;

[0035] Fig. 9 A schematic diagram of a display device according to an embodiment of the present application starting a first application based on an operation instruction;

[0036] Fig.10 A schematic diagram of a display device according to an embodiment of the present application starting a first application based on a voice command;

[0037] Fig.11 A schematic diagram of a first display page of a first application provided in an embodiment of the present application;

[0038] Fig.12 A schematic diagram of another display page of the first application provided in an embodiment of the present application;

[0039] Fig.13 A schematic diagram of a second display page of a first application provided in an embodiment of the present application;

[0040] Fig.14 A schematic diagram of a process for displaying ancient poetry metadata based on a second application provided in an embodiment of the present application;

[0041] Fig.15a A schematic diagram of a second application provided in an embodiment of the present application displaying the content of ancient poetry on a second display page;

[0042] Fig.15b A schematic diagram of starting a first application based on a voice command on a second display page provided in an embodiment of the present application;

[0043] Fig.16 A schematic diagram of a process for generating multimedia data by a display device provided in an embodiment of the present application;

[0044] Fig.17 Another schematic diagram of a process for generating multimedia data by a display device provided in an embodiment of the present application;

[0045] Fig.18 A schematic diagram of displaying ancient poetry image data and ancient poetry audio data on a third display page provided in an embodiment of the present application;

[0046] Fig.19 A schematic diagram of displaying ancient poetry video data on a third display page provided in an embodiment of the present application;

[0047] Fig. 20 A schematic diagram of displaying ancient poetry image data on a third display page provided in an embodiment of the present application;

[0048] Fig.21 A schematic diagram of displaying ancient poetry audio data on a third display page provided in an embodiment of the present application;

[0049] Fig. 22 A schematic diagram of a process of starting a first application on a display device provided in an embodiment of the present application;

[0050] Fig.23 A schematic diagram of a process for a display device provided in an embodiment of the present application to display multimedia data of ancient poetry based on a first application;

[0051] Fig.24 A schematic diagram of a scenario of interaction between a display device and a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0053] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0054] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0055] The terms "including" and "having" and any variations thereof are intended to cover but not be exclusive, for example, a product or device comprising a series of components is not necessarily limited to all the components explicitly listed, but may include other components not explicitly listed or inherent to these products or devices. The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with the element.

[0056] The display device 200 provided in the embodiment of the present application generally refers to a device with image display and data processing capabilities, and can have various implementation forms, for example, it can be a television, a smart TV, a computer, a laser projection device, a display (monitor), an electronic whiteboard (electronic bulletin board), an electronic desktop (electronic table), a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0057] Figure 1 Schematic diagram of an operation scenario between a display device 200 and a control device 100 provided in an embodiment of the present application. Figure 1 As shown, the user can operate the display device 200 through the terminal device 300 or the control device 100 .

[0058] In some embodiments, the control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that the display device 200 can recognize and respond to. The control device 100 can be a remote control, a stylus, a handle, etc., wherein the communication between the remote control and the display device 200 includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user can input user instructions through buttons on the remote control, voice input, control panel input, etc. to control the display device 200.

[0059] In some embodiments, a terminal device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop computer, etc.) can also be used as a control device to perform human-computer interaction between a user and the display device 200. The terminal device 300 can also be used as a communication device to establish a communication connection with the display device 200 and perform data interaction. The mobile terminal 300 can install the same software application as the display device 200, and then achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0060] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0061] In some embodiments, the display device 200 may also be controlled in a manner other than the control device 100 and the terminal device 300, for example, the user's voice command control may be directly received through a module for acquiring voice commands configured inside the display device 200, or the user's voice command control may be received through a voice control device provided outside the display device 200. The display device 200 may also receive user control through touch or gestures.

[0062] In some embodiments, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0063] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0064] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0065] like Figure 2 As shown, the display device 200 may include at least one of a modem 210, a communication device 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user interface.

[0066] In some embodiments, the modem 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0067] In some embodiments, the communication device 220 is a component for communicating with an external device or server according to various communication protocol types. The display device 200 can be provided with a plurality of communication devices 220 according to the different communication modes supported. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function. The communication device 220 may include at least one of a WiFi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish the sending and receiving of control signals and data signals with the external control device 100 or the server 400 through the communication device 220.

[0068] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0069] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or the detector 230 includes a sound collector, such as a microphone, for receiving external sounds.

[0070] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0071] In some embodiments, the controller 250 controls the operation of the display device 200 and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0072] In some embodiments, the controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), a first interface to an nth interface for input / output, a communication bus, etc. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0073] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0074] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0075] In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0076] In some embodiments, the "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0077] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0078] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0079] In some embodiments, the user interface may be used to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0080] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface. For example, the operating system can directly control the display device to provide a user interface, or provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0081] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0082] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0083] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0084] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0085] Figure 3 A schematic diagram of the software configuration of the display device 200 provided in an embodiment of the present application.

[0086] like Figure 3 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0087] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0088] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0089] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WiFi driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.) and power driver, etc.

[0090] Figure 4 A schematic diagram of the application layer of a display device provided in an embodiment of the present application.

[0091] In some embodiments, Figure 4 As shown, the framework layer also includes a speech recognition service, a speech recognition service (Automatic Speech Recognition, ASR), that is, automatic speech recognition, which is used to capture voice data and convert the voice data into text data. Exemplarily, the user's voice request for the far field or near field of the display device can be converted into text data.

[0092] In some embodiments, the framework layer also includes a natural language processing cloud service, which is a natural language processing capability provided by a cloud computing platform, and can directly call the API interface provided by the cloud service to implement various natural language processing functions. Specifically, it can be used to understand the user's intention based on text data.

[0093] In some embodiments, the framework layer also includes an ancient poetry microservice, which can be used to manage the storage, retrieval, and display of ancient poetry. When a user inputs a control instruction to search for ancient poetry, the display device can call the ancient poetry microservice according to the control instruction to query the ancient poetry corresponding to the control instruction.

[0094] In some embodiments, the framework layer also includes a text-to-speech service. The text-to-speech (TTS) service can convert text data into voice data, and can provide voice interaction services to users based on the voice assistant service.

[0095] In some embodiments, the framework layer also includes an ancient poetry multimedia data service for generating ancient poetry multimedia data based on the metadata of the ancient poetry.

[0096] In some embodiments, the framework layer also includes an ancient poetry comprehension service for performing semantic analysis on ancient poetry to obtain ancient poetry semantic data.

[0097] In some embodiments, the framework layer also includes an ancient poetry background retrieval service for retrieving the background of ancient poetry and obtaining ancient poetry background data.

[0098] In some embodiments, the framework layer also includes an image generation service for generating ancient poetry image data based on ancient poetry semantic data and ancient poetry background data using a preset image generation model.

[0099] In some embodiments, the framework layer also includes an audio generation service for generating ancient poetry audio data based on ancient poetry semantic data and ancient poetry background data using a preset audio generation model.

[0100] In some embodiments, the framework layer also includes a video generation service for generating ancient poetry video data based on ancient poetry image data and ancient poetry audio data.

[0101] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0102] Based on the above hardware framework and software framework, in some embodiments, after starting the first application, the display device can obtain metadata of the ancient poetry based on the voice command input by the user for displaying multimedia data of the ancient poetry, generate multimedia data of the ancient poetry according to the metadata of the ancient poetry, and display the multimedia data in the display page of the first application. Currently, if a user wants to use a display device to search for ancient poetry, he can search and display the ancient poetry through the voice assistant of the display device.

[0103] For example, Figure 5 As shown, the user can wake up the voice assistant and issue a control command to the voice assistant to display ancient poetry. Based on the control command, the display device calls the ancient poetry microservice. The ancient poetry microservice searches for ancient poetry based on the control command and controls the display to display the content of the ancient poetry and the corresponding audio reading, annotations and appreciation of the ancient poetry.

[0104] Furthermore, if Figure 6 As shown, the display device can display the text of the metadata of the ancient poetry in a simple text form, that is, the full text of the ancient poetry is displayed on the display page of the voice assistant, and the displayed text can have different fonts, font sizes and colors. Figure 7 As shown, the display device can also read the text of the metadata of ancient poems when displaying the text of the metadata of ancient poems. The reading can be real-person dubbing or electronic synthesized sound, and the user can choose different reading voices.

[0105] That is to say, users can see the text of the metadata of ancient poems on the display device and hear the synchronously played audio reading at the same time. However, in the form of reading, the ancient poems cannot restore the real historical scenes depicted in the ancient poems, nor can they fully display the charm and artistic conception of the ancient poems, making it difficult for users to fully understand the ancient poems. In addition, the simple reading form only provides users with an auditory experience, ignoring the visual needs of users, making the user's experience of watching ancient poems based on display devices poor.

[0106] In view of the above problems, an embodiment of the present application provides a display device, on which a first application is set. After the first application is started, the display device can obtain metadata of ancient poems according to the user's voice instructions to generate multimedia data that conforms to the ancient poems, and display the multimedia data on the display page of the first application.

[0107] In some embodiments, Figure 8 A flow chart of a display device provided in an embodiment of the present application showing multimedia data based on a control instruction for showing multimedia data of ancient poetry. Figure 8 As shown, the embodiment of the present application provides a display device 200, which includes: a display 260 and a controller 250, wherein the controller 250 is coupled to the display 260 and can be configured to perform the following program steps:

[0108] S100: In response to receiving a start instruction for a first application, start the first application.

[0109] In some embodiments, the first application is at least used to display multimedia data of ancient poetry; the multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data.

[0110] For example, the start instruction may be a voice instruction or an operation instruction. When the start instruction is an operation instruction, the operation instruction may be an operation of selecting the first application icon, an operation input on a control device (such as a remote controller), or other pre-set operations for starting the first application. The present application embodiment does not limit the start method of the first application. Fig. 9 As shown, taking the start instruction as the operation of the user selecting the first application icon as an example, the display device starts the first application in response to receiving the operation of the user selecting the icon of the first application. Fig.10 As shown, taking the start instruction as a voice instruction issued by the user as an example, the display device starts the first application in response to the voice instruction.

[0111] S200: Display a first display page of a first application, and obtain a first voice instruction input by a user for displaying multimedia data on the first display page.

[0112] In some embodiments, the first display page is an initial page displayed after the first application is started, and the first display page at least includes an input area, a first guide word for guiding the user to input the first voice command, and a guide example. Fig.11 As shown, the input area is an area where the user can input or trigger the first voice command, and the first guide language can be used to guide the text prompts for how to input the first voice command, such as "Please say the ancient poetry you want to experience", and the guidance example is a specific first voice command that the user can output, such as "Play Su Shi's Water Melody" and the like.

[0113] When the display device displays the first display page, the first display page can guide the user to select multimedia data of ancient poems to be displayed through the first voice command. The user can input the first voice command to the display device based on the first guide language and the guide example, so that the display device displays the multimedia data. In this way, the user can interact with the first application intuitively, providing a convenient way of use.

[0114] S300: In response to the first voice instruction, display text information corresponding to the input content of the first voice instruction in the input area.

[0115] In some embodiments, when the user issues a semantic instruction in the first display page, the first application recognizes the user's first voice instruction, converts the first voice instruction into text information, and displays it in the input area. In this way, the user can confirm in the input area whether the input first voice instruction is correctly recognized, and understand that the output first voice instruction is received by the first application.

[0116] For example, when the first voice command input by the user is "play Li Bai's "Quiet Night Thoughts", the first application will recognize this sentence and display the corresponding text "play Li Bai's "Quiet Night Thoughts"" in the input area. In this way, the user can confirm in the input area that the first voice command is correctly recognized, and further confirm whether this command meets his intention.

[0117] S400: Determine metadata of the ancient poem that matches the text information according to the text information, wherein the metadata at least includes the title, author and content of the ancient poem.

[0118] In some embodiments, the text information may include relevant information about the ancient poetry that the user wants to display, such as "Quiet Night Thoughts" shown in the above example. The first application can extract keywords in the text information and find metadata of ancient poetry that matches the keywords in a database or index. Here, the metadata of ancient poetry is data that describes the basic information of ancient poetry, which can help identify, classify and index ancient poetry. The metadata of ancient poetry may include the title of the ancient poetry, the author of the ancient poetry, the content of the ancient poetry such as the text, the dynasty of the author of the ancient poetry, the explanation or annotation of the ancient poetry, the source of the ancient poetry, the genre of the ancient poetry, etc. This application does not limit the content included in the metadata of ancient poetry.

[0119] For example, when the text information is "play Li Bai's "Quiet Night Thoughts", the keyword extraction technology can be used to extract the keywords "Li Bai" and "Quiet Night Thoughts" from the text information "play Li Bai's "Quiet Night Thoughts", and the first application searches for ancient poems written by Li Bai and titled "Quiet Night Thoughts" in the database or index according to the keywords. After finding the matching ancient poems, the first application obtains metadata such as the title, author, and content of the ancient poems, that is, the title: "Quiet Night Thoughts", the author: Li Bai, and the content of the ancient poems: The moonlight shines on the bed, I wonder if it is frost on the ground. I look up at the bright moon and look down at my hometown.

[0120] In some embodiments, when it is determined according to the text information that there are multiple matching ancient poems, metadata of each ancient poem can be obtained, and the metadata of each ancient poem and a second guide word can be displayed on the display page of the first application. The second guide word can be used to guide the user to select the ancient poem for which multimedia data is to be generated, and can be "Please select the ancient poem you want to view". For example, Fig.12 As shown, when the text information is "playing untitled ancient poems", the first application searches for ancient poems related to untitled according to the text information and obtains the corresponding metadata. Afterwards, the first application displays trivial and matching ancient poems on the display page, such as "Untitled" (Li Shangyin), "Untitled" (Du Mu), etc. The display page can also display a second guide, such as "Please select the ancient poem you want to view", to guide the user to proceed to the next step.

[0121] S500: Generate multimedia data of ancient poems according to metadata.

[0122] In some embodiments, the first application can generate various forms of ancient poetry multimedia data based on metadata to enrich the display of ancient poetry. Here, the multimedia data of ancient poetry can include but is not limited to ancient poetry image data, ancient poetry audio data and ancient poetry video data.

[0123] In the process of generating multimedia data of ancient poems based on metadata, an image generation model can be used to generate ancient poems image data depicting the scenes of ancient poems based on the metadata of ancient poems. An audio generation model can be used to generate ancient poems audio data including background music and reading audio that conform to the era background of ancient poems based on the metadata of ancient poems. The above-mentioned ancient poems image data and ancient poems audio data can also be integrated into a video to form ancient poems video data that displays the content and audio of ancient poems.

[0124] For example, when the first application confirms the metadata of the ancient poem "Quiet Night Thoughts", it can generate multiple ancient poem image data depicting the moon and the scene in front of the bed according to the metadata of "Quiet Night Thoughts", and each ancient poem image data can correspond to a sentence of the ancient poem. The background music and reading audio of the time background at that time can also be determined according to the metadata of the ancient poem "Quiet Night Thoughts". The first application can also integrate the above-mentioned ancient poem image data and ancient poem audio data into a video to display the text, background music, reading audio and image of the ancient poem.

[0125] S600: After generating multimedia data of ancient poetry, displaying a second display page of the first application.

[0126] In some embodiments, after generating multimedia data of ancient poetry, the first application may switch to a second display page, which includes text of metadata and multimedia display controls, and the multimedia display controls are used to determine the multimedia data to be displayed. Here, the second display page may include multiple multimedia display controls, which may be audio playback controls, image viewing controls, and video viewing controls, etc. The user may select multimedia data to be displayed through the multimedia display controls. In this way, the user can select the multimedia data of interest through the multimedia display controls on the second display page, and obtain a richer and more diverse ancient poetry learning experience.

[0127] For example, Fig.13 As shown, after the multimedia data of the ancient poem "Quiet Night Thoughts" is generated, the first area of ​​the second display page of the first application can display the text of the metadata of "Quiet Night Thoughts", and the second area of ​​the second display page can display the control 1 for playing audio, the control 2 for viewing images, the control 3 for displaying audio images, and the control 4 for viewing videos. After the user clicks the control 1, the audio data of the ancient poem "Quiet Night Thoughts" can be played, after the user clicks the control 2, the image data of the ancient poem related to "Quiet Night Thoughts" can be viewed, after the user clicks the control 3, the image data and audio data of the ancient poem can be viewed, and after the user clicks the control 4, the video data of the ancient poem including the text, audio, and image of the metadata of the ancient poem can be viewed.

[0128] S700: In response to selecting the multimedia presentation control, displaying multimedia data on a third display page of the first application.

[0129] In some embodiments, when the user selects a multimedia display control on the second display page, such as clicking a control for playing audio, the first application detects the user's selection action, responds to the action, switches the first application to the third display page, and displays the multimedia data selected by the user on the third display page.

[0130] According to the above embodiment, a first application for displaying multimedia data of ancient poetry is started. After the first application is started, a first voice instruction for selecting multimedia data to be displayed by a user is obtained on a first display page, the first voice instruction is recognized and the corresponding text information is displayed in an input area, the metadata of ancient poetry matching the input text information is determined, and multimedia data corresponding to the ancient poetry is generated according to the metadata. After generating the multimedia data, the first application switches to a second display page, the second display page displays the text of the metadata of the ancient poetry, and provides a multimedia display control for the user to select. When the user selects the multimedia display control on the second display page, the first application switches to a third display page to display the multimedia data selected by the user. In the first aspect of the process, the text information of the first voice instruction can be displayed in the input area, so that the user can confirm whether his instruction is correctly recognized, thereby enhancing the user's sense of trust. In the second aspect, different contents (such as metadata, multimedia display control, multimedia data) are displayed respectively through multiple display pages of the first application, so that each page displays different key information, which is convenient for the user to understand and operate. In the third aspect, multimedia data in various forms such as ancient poetry image data, ancient poetry audio data, and ancient poetry video data can be provided, enriching the learning experience of ancient poetry and satisfying the preferences of different users.

[0131] In some embodiments, before executing step S100, the controller is also configured to: in response to receiving a second voice instruction for displaying ancient poetry, start the second application, and display the text of the metadata of the ancient poetry and the first control on the display page of the second application; maintain the display of the metadata text on the display page of the second application, obtain the user's startup instruction, and start the first application.

[0132] In some embodiments, the second application is at least used to search for ancient poetry and display the content of ancient poetry. The second application can be a voice assistant that searches and displays ancient poetry based on the user's second voice command of ancient poetry.

[0133] In some embodiments, Fig.14 A flow chart of a display device provided in an embodiment of the present application showing the text of ancient poetry metadata on a display page of a second application based on a second voice command. Fig.14As shown, the user can send a second voice command to the display device based on a far-field or near-field voice request, and the user data acquisition module obtains the second voice command and starts the second application based on the second voice command. The second application sends the second voice command to the voice recognition service, and the voice recognition service performs voice recognition on the second voice command, generates an ancient poetry voice interaction text, and sends the ancient poetry voice interaction text to the natural language processing service. After receiving the ancient poetry voice interaction text, the natural language processing service performs semantic understanding and intent recognition on the ancient poetry voice interaction text, obtains an ancient poetry microservice call instruction, and sends the call instruction to the ancient poetry microservice. Based on the call instruction, the ancient poetry microservice searches and obtains the ancient poetry metadata, and sends the ancient poetry metadata to the text-to-speech service. At the same time, the ancient poetry metadata is sent to the second application, and the text of the ancient poetry metadata is displayed on the display page of the second application. After receiving the ancient poetry metadata, the text-to-speech service uses the text-to-speech technology to convert the ancient poetry metadata into corresponding ancient poetry voice data, and sends the ancient poetry voice data to the speaker. The speaker plays the received ancient poetry voice data. In this way, the display device can start the second application according to the second voice instruction and display the text of the ancient poetry metadata on the display page of the second application.

[0134] For example, the user can directly speak a voice command word to display ancient poems to the display device, and the controller searches for ancient poems based on the voice command word and displays the text of the ancient poems metadata on the display page of the second application. Fig.15a As shown, the user sends a second voice command to the display device, such as "show XXX ancient poetry", and the display device displays the metadata of the ancient poetry on the display page of the voice assistant based on the command.

[0135] At the same time, the display page of the voice assistant will also display the first control, which is used to display the icon of the first application. The user can select the icon of the first application on the display page of the second application to start the first application, and can continue to send a voice start instruction to the voice assistant to start the first application, so as to start the first application. Fig.15b shown.

[0136] That is to say, there are two ways to start the first application. The user can start the first application based on the start instruction on the application interface of the display, such as selecting the icon of the first application, or issuing a voice command of "open the first application". The user can also use the voice assistant to search for ancient poems first, and display the content of the ancient poems and the icon of the first application on the second display page. The user can select the icon of the first application on the second display page of the voice assistant to start the first application, and can continue to issue voice commands to start the first application after the content of the ancient poems is displayed on the second display page.

[0137] According to the above embodiment, displaying the icon of the first application on the display page of the second application allows the user to click the icon of the first application to start the first application, or to enter a voice command in the second application to start the first application. In this way, multiple ways of starting the first application are provided to meet the usage habits of different users.

[0138] In some embodiments, Fig.16 A schematic diagram of a process for generating multimedia data by a display device provided in an embodiment of the present application. Fig.16 As shown, after the display device obtains the user's first voice command, the ancient poetry multimedia data service is called, and the service includes an ancient poetry understanding service and an ancient poetry background retrieval service. When the ancient poetry understanding service is called, the metadata of the ancient poetry is understood to obtain the ancient poetry semantic data. The ancient poetry semantic data includes the first data, the first keyword and the second keyword. When the ancient poetry background retrieval service is called, the ancient poetry background data is retrieved according to the metadata of the ancient poetry, including the first background data, the second background data and the third background data. Then, the semantic features and background features of the ancient poetry semantic data and the ancient poetry background data are extracted. The semantic features and background features are polished and input into the preset image generation model to obtain the ancient poetry image data. The semantic features and background features are feature encoded and input into the preset audio generation model to obtain the ancient poetry audio data. Then, the video generation service is called to integrate the ancient poetry image data and the ancient poetry audio data into the ancient poetry video data.

[0139] Furthermore, in some embodiments, Fig.17 Another schematic diagram of a process for generating multimedia data by a display device provided in an embodiment of the present application is as follows: Fig.17 As shown, the controller executing S500 may be configured to execute the following program steps:

[0140] S1701: Based on the metadata, semantic understanding of ancient poems is performed to obtain semantic data of ancient poems.

[0141] In some embodiments, the ancient poetry semantic data may be information extracted from the content of the ancient poetry, which may include sentiment analysis, theme extraction, keyword extraction, and sentence structure analysis of the ancient poetry. Among them, sentiment analysis can be used to analyze the sentiment expressed in the ancient poetry, theme extraction can be used to identify the main topic and central idea in the ancient poetry, keyword extraction can be used to find out the words that reflect the core content in the ancient poetry, and sentence structure analysis can be used to analyze the sentence structure of the ancient poetry.

[0142] In some embodiments, in order to make the analysis data more comprehensive to characterize the deep meaning of poetry, the metadata of ancient poetry can be semantically analyzed. In some examples, the metadata of ancient poetry can be semantically analyzed to obtain ancient poetry semantic data, which can include first data representing the interpretation of ancient poetry, first keywords describing the scene of ancient poetry, and second keywords describing the emotion of ancient poetry. Among them, the first data is an interpretation of the content of ancient poetry, including the data after the interpretation of the vocabulary, sentence meaning and allusions involved in ancient poetry. The first keyword is a word for a specific scene or environment depicted in ancient poetry. The described scene may include natural landscapes, character activities, seasonal changes, etc. The second keyword can reflect the emotional state expressed in ancient poetry, and reflect the author's emotional experience when creating, such as joy, anger, sorrow, joy, sadness, separation, etc.

[0143] In some embodiments, the content in the metadata of ancient poems can be preprocessed, that is, the text of the ancient poems can be segmented, stop words can be removed, and part-of-speech tagging can be performed. Among them, word segmentation refers to breaking down the text of the ancient poems into individual words or phrases, removing stop words refers to removing words that have no practical meaning, and part-of-speech tagging refers to marking the part of speech of the words after word segmentation. Then, the first content after preprocessing can be analyzed by the semantic analysis model in the deep learning model to obtain the first data of the interpretation of the ancient poems. In this way, by analyzing the content of the ancient poems through the preset semantic analysis model, a modern Chinese interpretation can be provided for each sentence or word in the ancient poems.

[0144] In some embodiments, after obtaining the first data of ancient poetry interpretation, a pre-trained keyword extraction model can also be used to identify and extract first keywords describing scenes and scenery in the first content, such as words related to the background and description objects of the ancient poetry.

[0145] In some embodiments, after obtaining the first keyword, a pre-trained keyword extraction model can be used to identify and extract a second keyword describing emotions in the first content, such as words related to the poet's emotional state and the emotional tendency expressed. Based on the second keyword, the emotions expressed in the ancient poetry can be analyzed, such as joy, sadness, anger, etc.

[0146] For example, taking "Quiet Night Thoughts" as an example, it is explained how to determine the semantic data of ancient poems according to the metadata of ancient poems. First, the main text of "Quiet Night Thoughts" is preprocessed to obtain the first content of "Quiet Night Thoughts", which includes: in front of the bed (directional word), bright moon (noun), light (noun), doubt (verb), yes (verb), on the ground (directional word), frost (noun), raise head (verb phrase), look (verb), bright moon (noun), lower head (verb phrase), think (verb), hometown (noun).

[0147] After obtaining the first content, the first data of "Quiet Night Thoughts" is determined according to the first content, and the first data may be "Beside the bed, bright moonlight shines in; I wonder if it is frost on the ground; I raise my head to look up at the bright moon; I lower my head to miss my hometown."

[0148] After determining the first data of "Quiet Night Thoughts", extract the first keyword describing the scene and scenery in the first content, the first keyword can be bright moon, light, ground, frost, bright moon, hometown. After determining the first keyword of "Quiet Night Thoughts", extract the second keyword describing the emotion in the first content, namely: doubt, hope, and thought.

[0149] Based on this, in some embodiments, the controller executes S1701 and can be configured to: pre-process the content of the metadata to obtain the first content; the pre-processing includes word segmentation, removal of stop words and part-of-speech tagging; based on the semantic analysis model, analyze the semantics of the first content to obtain the first data; the first data is used to characterize the interpretation of the ancient poetry; based on the first data, determine the first keyword of the first content; the first keyword includes the vocabulary describing the scene and scenery in the first content; based on the first data, determine the second keyword of the first content; the second keyword includes the vocabulary describing the emotion in the first content; use the first data, the first keyword and the second keyword as the semantic data of the ancient poetry.

[0150] According to the above embodiment, the controller can perform semantic analysis on the metadata of the ancient poetry to determine the first data representing the interpretation of the ancient poetry, the first keyword data representing the scene of the ancient poetry, and the second keyword representing the emotion of the ancient poetry. The deep meaning of the poetry can be more comprehensively represented according to the first data, the first keyword, and the second keyword, providing data support for the subsequent generation of multimedia data.

[0151] S1702: Based on the metadata, background search is performed on the ancient poems to obtain background data of the ancient poems.

[0152] In some embodiments, the ancient poetry background data may be background information related to the creation of ancient poetry, such as the social and historical circumstances of the creation of ancient poetry, the life experience of the author, the cultural atmosphere when the ancient poetry was created, and the literary influences that may have been received when the ancient poetry was created.

[0153] In some embodiments, in order to more comprehensively characterize the historical, cultural, and social background of the creation of ancient poems, a background search may be performed on the metadata of the ancient poems. In some examples, a background search may be performed on the metadata of the ancient poems to obtain background data of the ancient poems, which may include first background data related to the author, second background data related to the creation of the ancient poems, and third background data related to the era of the creation of the ancient poems.

[0154] Here, the first background data may be personal information related to the author, including but not limited to the author's life story such as the author's birthplace, growth experience, educational background, etc., the author's creative process, the author's thoughts and ideas, and the historical evaluation of the author, etc. The second background data may be specific contexts related to the creation of poetry, including but not limited to the motivation for creation, the location of creation, historical events that occurred during the creation, and the author's personal experiences during the creation period, etc. The third background data may be the social and historical background of the creation period of ancient poetry, including but not limited to major historical events that occurred during the creation period, the social economy, politics, and cultural conditions during the creation period, the literary trends during the creation period, the cultural atmosphere during the creation period, and popular musical instruments, etc.

[0155] In some embodiments, the background data of ancient poetry can be determined based on a preset background retrieval model and metadata. The preset background retrieval model can be implemented by combining a large language model (LLM) and a self-built retrieval-augmented generation (RAG) knowledge base. Such a combination can make full use of the powerful generation capability of LLM and the precise retrieval capability of RAG, thereby providing richer and more accurate background information. In addition, the preset background retrieval model can also be integrated with existing search engines to further improve the retrieval effect.

[0156] Exemplarily, relevant information about ancient poems, authors, historical events, etc. can be collected from various sources, and the data can be indexed using indexing techniques such as inverted indexing to establish a RAG knowledge base.

[0157] Exemplarily, a suitable search engine tool may be selected to construct an API interface, so that the ancient poetry background data may be obtained from the indexing tool, and the self-built knowledge base may be integrated with the ancient poetry background data of the search engine tool.

[0158] Exemplarily, a search operation may be performed on the metadata to obtain background data from a self-built knowledge base and a search engine tool, respectively, and the retrieved information may be integrated to generate final background data of ancient poetry.

[0159] In some embodiments, the author's name may be input into a pre-trained background data retrieval model, which searches for information related to the author in a database or knowledge base to obtain the first background data.

[0160] In some embodiments, after determining the first background data, the creation time of the ancient poem can be determined based on the title and author of the ancient poem. Furthermore, the creation time is input as a search condition into a preset search model, and the model searches for historical events, social changes, cultural trends, etc. related to the time period to determine the second background data of the ancient poem. At the same time, the model can also search for third background data, such as information on politics, economy, science and technology, art, etc.

[0161] Exemplarily, take "Quiet Night Thoughts" as an example to illustrate how to determine the background data of ancient poems based on the metadata of ancient poems. First, based on the author Li Bai and the preset background search model, the first background data is determined, that is, the author Li Bai (701-762), a famous poet in the Tang Dynasty, with the name Taibai, the name Qinglian Jushi, and the name "Poet Immortal". Biography: Li Bai was born in Suiye City (now in Kyrgyzstan), and later moved to Sichuan with his father. He traveled extensively throughout his life, and his footprints were all over China. His poems were deeply influenced by Taoist thoughts, with a bold and unrestrained style, and he was good at using exaggeration and romantic techniques to express emotions.

[0162] After determining the first background data, the creation time of the ancient poem can be determined based on the title and the author. That is, "Quiet Night Thoughts" is generally believed to have been written by Li Bai during his stay in Yangzhou, around 726 AD (the 24th year of Emperor Xuanzong of Tang Dynasty). At this time, Li Bai was in his youth, had not yet obtained an official position in the court, and was traveling around.

[0163] After determining the creation time of "Quiet Night Thoughts", the second background data of "Quiet Night Thoughts" and the third background data of "Quiet Night Thoughts" are retrieved according to the creation time. The creation background of "Quiet Night Thoughts" is that Li Bai may be in a foreign land at that time, far away from his hometown and relatives. When he was alone at night, he was moved by the scene and wrote this poem expressing his homesickness. Li Bai's life status at this time was in the stage of traveling and he had not yet settled down, so he often missed his hometown. The era background data is during the Kaiyuan period, when the Tang Dynasty was in a prosperous and stable period, with a strong cultural atmosphere and literary creation reaching its peak. At that time, musical instruments could be used for accompaniment when reciting poems, such as traditional instruments such as pipa, guzheng, and flute.

[0164] Based on this, in some embodiments, the controller executes S1702 and can be configured to: retrieve the first background data of the author in the metadata, the first background data is used to represent the background data related to the author; determine the creation time of the ancient poetry according to the title and author in the metadata; determine the second background data of the ancient poetry according to the creation time and the title; the second background data is used to represent the creation background data related to the author's creation of the ancient poetry; determine the third background data of the ancient poetry according to the creation time; the third background data is used to represent the era background data related to the creation time; use the first background data, the second background data and the third background data as the background data of the ancient poetry.

[0165] According to the above embodiment, the controller can perform background search on the metadata of the ancient poetry to determine the first background data related to the author, the second background data related to the creation, and the third background data related to the era. Determining the background data of the ancient poetry can more comprehensively characterize the historical, cultural, and social background of the creation of the ancient poetry, and provide data support for the subsequent generation of multimedia data.

[0166] S1703: Generate ancient poetry image data corresponding to the content of each sentence of ancient poetry based on the ancient poetry semantic data and ancient poetry background data using the ancient poetry image generation model.

[0167] In some embodiments, the ancient poetry image data may include image data that presents the content of the ancient poetry or its artistic conception in the form of an image. The ancient poetry image data may be static or dynamic (such as animation), and the present application does not limit this. Further, the ancient poetry image data may express the artistic conception and emotion of the ancient poetry through visual elements such as landscapes, flowers and birds, characters, calligraphy, etc., and may also include the original text of the ancient poetry or its translation. Exemplarily, an ancient poetry image data depicting "the smoke rises straight in the desert, the setting sun is round in the long river" may show a vast desert and the setting sun.

[0168] In the process of generating ancient poetry image data, key nouns, verbs, adjectives, etc. that can represent the main imagery, emotions, and themes of ancient poetry can be extracted from the ancient poetry semantic data as semantic features. Semantic features can determine the overall style and specific content of ancient poetry image data.

[0169] Next, background features can be extracted from the ancient poetry background data. The background features can determine the generation style of the ancient poetry image data based on the historical background of the creation of the ancient poetry, the regional cultural characteristics described in the ancient poetry, and the personal experience of the author.

[0170] After obtaining the semantic features and background features of the ancient poetry, the semantic features and background features can also be polished to ensure that they are more accurate when input into the preset image model. For example, the semantic features and background features can be input into the polishing model, and then the polished semantic features and background features can be input into the preset image generation model to generate ancient poetry image data that is consistent with the artistic conception of the ancient poetry. Here, the preset image generation model is a deep learning model, which can be a generative adversarial network (GANs), a variational autoencoder (VAEs), etc.

[0171] Exemplarily, take "Quiet Night Thoughts" as an example to illustrate the polishing of semantic features and background features. First, extract the semantic features of the semantic data of ancient poems, which may include bright moonlight, suspected frost, looking up, thinking with head down, etc., and extract the background features of the background data of ancient poems, which may include creation in the 24th year of Emperor Xuanzong of Tang Dynasty Kaiyuan, accompaniment of popular musical instruments, etc. Then, polish the semantic features to obtain the polished semantic features, including: bright and clear moonlight, the ground seems to be covered with silver frost, looking up at the bright moon, and looking down at home. Polish the background features to obtain the polished background features. In the Tang Dynasty, on a quiet night in the 24th year of Emperor Xuanzong of Tang Dynasty Kaiyuan, the famous poet Li Bai wrote this poem in a strange city.

[0172] Furthermore, the preset image generation model can determine the main elements of the image, such as the moon, the ground, the mission, the musical instrument, etc., based on the polished semantic features and background features, and generate ancient poetry image data based on the above elements.

[0173] Based on this, in some embodiments, the controller executes S1703 and can be configured to: extract semantic features of ancient poetry semantic data; extract background features of ancient poetry background data; and use a preset image generation model to generate ancient poetry image data based on semantic features and background features.

[0174] According to the above embodiment, based on the semantic features and background features of ancient poems, a preset image generation model can be used to generate ancient poetry image data that is consistent with the interpretation, scene and emotion described in the ancient poems, thereby increasing the expression forms of ancient poems and enhancing the user's visual experience.

[0175] S1704: Using an ancient poetry audio generation model, ancient poetry audio data is obtained according to the ancient poetry semantic data and the ancient poetry background data. The ancient poetry audio data includes background music audio corresponding to the ancient poetry background data, and text reading audio corresponding to the emotion of the ancient poetry semantic data and the content of each sentence of the ancient poetry.

[0176] In some embodiments, the ancient poetry audio data may include audio data that presents the text of the ancient poetry in the form of sound. The ancient poetry audio data may be a recitation, singing, or a recitation containing background music that is the same as the background of the times. The ancient poetry audio data may convey the emotions and beauty of the ancient poetry through the intonation, rhythm, and rhyme of the sound. Background music that is consistent with the background of the times when the ancient poetry was created may also be added.

[0177] In the process of generating audio data of ancient poems, it is also necessary to extract the semantic features and background features of ancient poems. Afterwards, the audio data of ancient poems is generated according to the semantic features and background features of ancient poems. Next, the semantic features and background features of ancient poems are encoded to conform to the first feature of the input format of the preset audio generation model. Here, the preset audio generation model can be a deep learning model, such as a recurrent neural network (RNN), a transformer (Transformer) or a generative adversarial network (GAN) that can generate audio according to the input features. Further, the semantic features and background features are input into the preset audio generation model, and the corresponding music audio can be generated with the background features, that is, music that matches the emotional atmosphere with the background of the ancient poems, popular instruments, etc. The preset audio generation model can also be used to generate text reading audio according to the semantic features of ancient poems, which may include the correct emotions and tone contained in the ancient poems, and match the rhythm and rhythm of the ancient poems. In other words, the audio data of ancient poems includes background music corresponding to the background data of ancient poems to create an atmosphere, and also includes text reading corresponding to the semantic data of ancient poems to convey the content and emotions of the verses.

[0178] Based on this, the controller executes S1704 and can be configured to: extract semantic features of ancient poetry semantic data; extract background features of ancient poetry background data; feature encode semantic features and background features to obtain a first feature; input the first feature into a preset audio generation model to obtain ancient poetry audio data, the ancient poetry audio data including music audio corresponding to the ancient poetry background data and text reading audio corresponding to the ancient poetry semantic data.

[0179] According to the above embodiment, the controller can generate an audio work that combines music and reading according to the semantics and background data of ancient poetry, which can not only enhance the auditory experience of ancient poetry, but also help users better understand and feel the deep meaning of ancient poetry.

[0180] S1705: Generate ancient poetry video data based on the ancient poetry image data and the ancient poetry audio data.

[0181] In some embodiments, subtitles in ancient poetry video data are determined based on the text of metadata, the ancient poetry audio data is used as the audio of the ancient poetry display video data, the ancient poetry image data is automatically rotated to form a carousel video, and the carousel video is used as the video display of the ancient poetry video data.

[0182] In the process of generating ancient poetry video data, subtitles in the video are generated according to the metadata of the ancient poetry, and the ancient poetry audio data is used as the audio part of the video data. According to the order of the verses or the relevance of the content, the ancient poetry image data is arranged in a certain order to form a continuously played carousel video, and the subtitles, audio and carousel video are synthesized to form the final ancient poetry video data.

[0183] In this way, the ancient poetry video data includes subtitles, audio and automatically rotated images, providing users with a complete multimedia experience. This method not only allows users to appreciate ancient poetry through both visual and auditory senses, but also allows them to better understand the content of the poem through subtitles.

[0184] According to the above embodiments, ancient poetry image data that conforms to the artistic conception, scene and emotion of ancient poetry can be generated based on the ancient poetry semantic data and ancient poetry background data, and background music that conforms to the creation background and era background of ancient poetry can also be generated. Ancient poetry video data can also be generated based on the ancient poetry image data and ancient poetry audio data. In this way, the expression form of ancient poetry on the display device can be enriched to meet the visual and auditory experience of users.

[0185] After obtaining the multimedia data of ancient poetry, the text and multimedia display controls of the ancient poetry metadata can be displayed on the second display page of the first application. The multimedia display controls may include audio and image display controls, video display controls, image display controls and audio playback controls.

[0186] In some embodiments, after the above-mentioned audio and image display control is selected, the controller executes S700 and can be configured to: in response to selecting the audio and image display control, arrange all ancient poetry image data according to the description order of the text of the metadata to obtain an image carousel set; play the ancient poetry audio data; according to the playback progress of the ancient poetry audio data, highlight the first text in the text of the metadata displayed in the first display area of ​​the third display page; the first text is the text in the metadata text corresponding to the playback progress, and according to the playback progress and the first text, automatically play the ancient poetry image data in the image carousel set in the second display area of ​​the third display page.

[0187] In the process of displaying audio data and image data of ancient poems, when the user selects the audio image display control on the second display page, the first application switches to the third display page, and arranges all relevant image data in the order of verses according to the order of text description in the metadata of the ancient poems to form an image carousel set. The first application starts to play the audio data of ancient poems. While playing the audio data of ancient poems, the currently playing verses are highlighted in the first display area of ​​the third display page according to the playback progress of the audio. Here, the highlighting method can be to highlight or bold the currently playing verses in the text. And according to the playback progress of the audio and the currently highlighted verses, the corresponding images in the image carousel set are automatically played in the second display area of ​​the third display page. The display result of the third display page is as follows. Fig.18 shown.

[0188] According to the above embodiment, the display device displays ancient poetry image data, ancient poetry audio data and ancient poetry metadata on the third display page of the first application. In this way, users can synchronously appreciate the audio data, text content and related images of ancient poetry, thereby obtaining a richer and more coherent multimedia experience.

[0189] In some embodiments, after the above-mentioned video display control is selected, the controller executes S700 and can also be configured to: in response to the selection of the video display control, display the ancient poetry video data on the third display page.

[0190] In the process of displaying the ancient poetry video data, when the user selects the video display control on the second display page, the first application switches to the third display page and displays the ancient poetry video data selected by the user on the page. The display result of the third display page is as follows: Fig.19 In this way, users can easily select and watch multimedia videos of ancient poems, thereby obtaining a richer and more intuitive learning experience.

[0191] In some embodiments, after the above-mentioned image display control is selected, the controller executes S700 and can also be configured to: in response to selecting the image display control, arrange all ancient poetry image data according to the description order of the text of the metadata to obtain an image carousel set; display the text of the metadata in the third display area of ​​the third display page, and display the image carousel set in the fourth display area of ​​the third display page; wherein, when displaying the ancient poetry image data in the image carousel set, the second text of the metadata is highlighted; the second text is the text in the text of the metadata corresponding to the ancient poetry image data being displayed.

[0192] In the process of displaying image data of ancient poems, when the user selects the image display control on the second display page, the first application switches to the third display page, and arranges all relevant image data in the order of verses according to the order of text description in the metadata of the ancient poems to form an image carousel set. Afterwards, the text of the metadata of the ancient poems is displayed in the third display area on the third display page, and the image carousel set is displayed in the fourth display area, that is, the images are displayed in the order of arrangement. Here, when displaying images in the image carousel set, the application will highlight the second text in the metadata text that corresponds to the image according to the content of the currently displayed image. The display result of the third display page is as follows. Fig. 20 In this way, the display device displays the text of the ancient poetry metadata and the ancient poetry image data on the third display page, so that the user can synchronously appreciate the text content and related images of the ancient poetry, thereby obtaining a richer and more coherent multimedia experience.

[0193] In some embodiments, after the above-mentioned audio playback control is selected, the controller executes S700 and can also be configured to: in response to the selection of the audio playback control, play the ancient poetry audio data; according to the playback progress of the ancient poetry audio data, highlight the first text in the text of the metadata displayed in the fifth display area in the third display page; the first text is the text in the metadata text corresponding to the playback progress.

[0194] In the process of displaying the audio data of ancient poetry, when the user selects the audio playback control on the second display page, the first application can start playing the audio data of the ancient poetry. While playing the audio data of the ancient poetry, the first application can highlight the first text currently being played in the fifth display area of ​​the third display page according to the progress of the audio playback. The display result of the third display page is as follows: Fig.21 As shown, in this way, the text of the ancient poetry metadata is displayed on the third display page, and the ancient poetry audio data is played, so that the user can synchronously listen to the ancient poetry audio data and see the currently playing verses highlighted, thereby obtaining a more coherent and rich multimedia experience.

[0195] In some embodiments, Fig. 22 A schematic diagram of a process of starting a first application on a display device provided in an embodiment of the present application. Fig. 22 As shown, there are two ways to start the first application. The first way to start the first application is to display the text of the ancient poetry metadata on the display page of the second application of the display device, and start the first application based on the second application receiving the start instruction of the first application. Here, the start instruction of the first application received by the second application can be a voice instruction or an operation instruction. The second way to start the first application is that the display device receives the operation instruction of the first application on the application display interface or directly outputs a voice instruction to the display device.

[0196] In some embodiments, Fig.23 A schematic diagram of a process flow of displaying ancient poetry multimedia data based on a first application on a display device provided in an embodiment of the present application. Fig.23 As shown, after starting the first application, the first application can obtain the voice command input by the user for displaying multimedia data, recognize the text information of the command and display it in the input area of ​​the first display page. Then, the first application sends the text information to the ancient poetry multimedia data service. The ancient poetry multimedia data service determines the metadata of the matched ancient poetry based on the text information, and sends the ancient poetry metadata to the ancient poetry semantic understanding service and the ancient poetry background search service.

[0197] The ancient poetry understanding service performs semantic analysis on metadata to obtain ancient poetry semantic data, and sends the ancient poetry semantic data to the ancient poetry multimedia data service. The ancient poetry background search service searches the ancient poetry metadata to obtain ancient poetry background data, and then sends the ancient poetry background data to the ancient poetry multimedia service.

[0198] The ancient poetry multimedia service sends the ancient poetry semantic data and the ancient poetry background data to the image generation service and the music generation service. The image generation service extracts the semantic features of the ancient poetry semantic data and the background features of the ancient poetry background data; uses the preset image generation model to generate the ancient poetry image data for describing the ancient poetry according to the ancient poetry semantic data and the ancient poetry background data and sends it to the first application and the video generation service. After receiving the ancient poetry image data, the first application displays the ancient poetry image data on the third display page. The audio generation service extracts the semantic features of the ancient poetry semantic data and the background features of the ancient poetry background data, performs feature encoding on the semantic features and the background features, and obtains the first feature; inputs the first feature into the preset audio generation model, generates the ancient poetry audio data matching the ancient poetry content and sends it to the speaker and the video generation service, and controls the speaker to play the ancient poetry audio data on the third display page. The video generation service generates the ancient poetry video data according to the ancient poetry image data and the ancient poetry audio data, and sends the ancient poetry video data to the first application, and plays the ancient poetry video data on the third display page of the first application.

[0199] In some embodiments, Fig.24 As shown, the present application provides a display device 200, including: a display 260, a communicator, configured to be connected to a server for communication, and a controller, coupled to the display and the communicator, and configured to:

[0200] In response to receiving a start instruction for a first application, the first application is started; the first application is at least used to display multimedia data of ancient poetry; the multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data;

[0201] Displaying a first display page of the first application, obtaining, on the first display page, a first voice instruction input by a user for displaying multimedia data, wherein the first display page at least includes an input area, a first guiding word for guiding the user to input the first voice instruction, and a guiding example;

[0202] In response to the first voice command, displaying text information corresponding to the input content of the first voice command in the input area;

[0203] Controlling the communicator to send text information to the server, so that the server obtains metadata of the ancient poetry according to the text information and generates multimedia data of the ancient poetry; wherein the metadata at least includes the title, author and content of the ancient poetry;

[0204] Controlling the communicator to receive multimedia data from the server;

[0205] Displaying a second display page of the first application; wherein the second display page includes text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed;

[0206] In response to selecting the multimedia presentation control, the multimedia data is displayed on the third display page of the first application.

[0207] The display device 200 may be a smart TV with high computing power, which can recognize control instructions and generate multimedia data through its own computing chip. However, in some application scenarios, the display device 200 may be a low computing device such as an advertising machine, which generates multimedia data slowly or cannot carry a computing model.

[0208] Based on the above problems, in some embodiments, the display device 200 can also interact with the server in real time to generate multimedia through the server, receive multimedia data sent by the server, and then display the multimedia data through the display device 200. The server can be a server arranged on the cloud, a server arranged on the local network, or a server searched based on edge computing. The method of communicating with the server to obtain multimedia data alleviates the problem that the display device 200 cannot generate multimedia data in time according to the control instructions input by the user due to insufficient computing power.

[0209] In some embodiments, the present application provides a server, including:

[0210] A communicator configured to be communicatively connected with a display device;

[0211] A processor is coupled to the communicator and is configured to:

[0212] Controlling the communicator to receive text information from the display device, and obtaining metadata of the ancient poetry according to the text information; the metadata at least includes the title, author and content of the ancient poetry;

[0213] Generate multimedia data of ancient poems based on metadata;

[0214] The communicator is controlled to send the multimedia data to the display device, so that the display device displays the multimedia data on the third display page of the first application.

[0215] The server can receive the control instruction forwarded by the display device 200, and generate multimedia data according to the control instruction. And send the generated multimedia data to the display device 200, so that the display device 200 displays the multimedia data for the user to watch. The problem of low computing power display device 200 generating multimedia data is alleviated.

[0216] In some embodiments, the present application provides a method for displaying ancient poetry multimedia data, including:

[0217] In response to receiving a start instruction for a first application, the first application is started; the first application is at least used to display multimedia data of ancient poetry; the multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data;

[0218] Displaying a first display page of the first application, obtaining, on the first display page, a first voice instruction input by a user for displaying multimedia data, wherein the first display page at least includes an input area, a first guiding word for guiding the user to input the first voice instruction, and a guiding example;

[0219] In response to the first voice command, displaying text information corresponding to the input content of the first voice command in the input area;

[0220] Determine metadata of the ancient poem that matches the text information according to the text information, wherein the metadata at least includes the title, author and content of the ancient poem;

[0221] Generate multimedia data of ancient poems based on metadata;

[0222] After the multimedia data of the ancient poetry is generated, a second display page of the first application is displayed; wherein the second display page includes text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed;

[0223] In response to selecting the multimedia presentation control, the multimedia data is displayed on the third display page of the first application.

[0224] The method is applied to a display device 200 equipped with a computing chip with high computing power, such as a smart TV, a computer, etc. The control command input by the user can be received, and the controller of the display device 200 can obtain the metadata of the ancient poetry according to the control command, generate analysis data according to the metadata, and generate multimedia data of the ancient poetry according to the analysis data.

[0225] According to the above embodiment, the controller can start the first application based on the start instruction, and determine the multimedia data based on the voice instruction for displaying the multimedia data of the ancient poetry. This process can generate multimedia data related to the ancient poetry according to the user's voice instruction, which can fully reflect the meaning and artistic conception of the ancient poetry, and can also provide the user with a visual experience of the ancient poetry, thereby improving the user experience.

[0226] It is understandable that in some embodiments, a method for displaying ancient poetry multimedia data applied to a display device with low computing power is also provided. The controller 250 of the display device 200 forwards the control instruction input by the user to the server, and the server executes the process of generating multimedia data and sends the generated multimedia data to the display device 200. The display device 200 displays the multimedia data.

[0227] As can be seen from the above technical content, the present application provides a display device, a server, and a method for displaying multimedia data of ancient poems. The process controller starts a first application for displaying multimedia data of ancient poems. After the first application is started, a first voice instruction for selecting multimedia data to be displayed by a user is obtained on a first display page, the first voice instruction is recognized and the corresponding text information is displayed in the input area, the metadata of ancient poems matching the input text information is determined, and multimedia data corresponding to the ancient poems is generated according to the metadata. After generating the multimedia data, the first application switches to a second display page, the second display page displays the text of the metadata of the ancient poems, and provides a multimedia display control for the user to select. When the user selects the multimedia display control on the second display page, the first application switches to a third display page to display the multimedia data selected by the user. The first aspect of this process is that the text information of the first voice command can be displayed in the input area, so that the user can confirm whether his command is correctly recognized, thereby enhancing the user's sense of trust. The second aspect is that different content (such as metadata, multimedia display controls, multimedia data) can be displayed through multiple display pages of the first application, so that each page displays different key information, which is convenient for users to understand and operate. The third aspect is that various forms of multimedia data such as ancient poetry image data, ancient poetry audio data and ancient poetry video data can be provided, enriching the learning experience of ancient poetry and meeting the preferences of different users.

[0228] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A display device, characterized in that: include: monitor; A controller is coupled to the display and is configured to: In response to receiving a start instruction for a first application, start the first application; The first application is at least used to display multimedia data of ancient poetry; the multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data; Displaying a first display page of the first application, and acquiring a first voice instruction for displaying multimedia data input by a user on the first display page, wherein the first display page at least includes an input area, a first guide word for guiding the user to input the first voice instruction, and a guide example; In response to the first voice instruction, displaying text information corresponding to the input content of the first voice instruction in the input area; Determine metadata of the ancient poem that matches the text information according to the text information, wherein the metadata at least includes the title, author and content of the ancient poem; Generating multimedia data of the ancient poetry according to the metadata; After the multimedia data of the ancient poetry is generated, a second display page of the first application is displayed; wherein the second display page includes the text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed; In response to selecting the multimedia presentation control, the multimedia data is displayed on a third display page of the first application.

2. The display device according to claim 1, characterized in that The multimedia display control includes an audio and image display control; The controller executes, in response to selecting the multimedia presentation control, displaying the multimedia data on the third display page of the first application, and is specifically configured as follows: In response to selecting the audio image display control, all the ancient poetry image data are arranged according to the description order of the text of the metadata to obtain an image carousel set; Playing the ancient poetry audio data; According to the playback progress of the ancient poetry audio data, highlighting the first text in the text of the metadata displayed in the first display area in the third display page; the first text is the text in the metadata corresponding to the playback progress; The ancient poetry image data in the image carousel set is automatically played in the second display area of ​​the third display page according to the play progress and the first text.

3. The display device according to claim 1, characterized in that The multimedia display control includes a video display control; The controller is configured to display the multimedia data on a third display page of the first application in response to selecting the multimedia presentation control, and is further configured to: In response to selecting the video display control, the ancient poetry video data is displayed on the third display page.

4. The display device according to claim 1, characterized in that The multimedia display control includes an image display control; The controller executes, in response to selecting the multimedia presentation control, displaying the multimedia data on the third display page of the first application, and is specifically configured as follows: In response to selecting the image display control, all the ancient poetry image data are arranged according to the description order of the text of the metadata to obtain an image carousel set; Displaying the text of the metadata in a third display area of ​​the third display page, and displaying the image carousel set in a fourth display area of ​​the third display page; When displaying the ancient poetry image data in the image carousel set, the second text of the metadata is highlighted; the second text is the text in the metadata that corresponds to the ancient poetry image data being displayed.

5. The display device according to claim 1, characterized in that The multimedia display controls include audio playback controls; The controller executes, in response to selecting the multimedia presentation control, displaying the multimedia data on the third display page of the first application, and is configured to: In response to selecting the audio playback control, playing the ancient poetry audio data; According to the playback progress of the ancient poetry audio data, the first text in the text of the metadata displayed in the fifth display area in the third display page is highlighted; the first text is the text in the metadata text corresponding to the playback progress.

6. The display device according to claim 1, characterized in that The controller generates multimedia data of the ancient poetry according to the metadata, and is specifically configured as follows: Based on the metadata, semantic understanding is performed on the ancient poetry to obtain semantic data of the ancient poetry; Based on the metadata, background search is performed on the ancient poetry to obtain background data of the ancient poetry; Using an ancient poetry image generation model, generating the ancient poetry image data corresponding to the content of each sentence of the ancient poetry according to the ancient poetry semantic data and the ancient poetry background data; Using an ancient poetry audio generation model, the ancient poetry audio data is obtained according to the ancient poetry semantic data and the ancient poetry background data, wherein the ancient poetry audio data includes background music audio corresponding to the ancient poetry background data, and text reading audio corresponding to the content of each sentence of the ancient poetry and in accordance with the emotion of the ancient poetry semantic data; The ancient poetry video data is generated according to the ancient poetry image data and the ancient poetry audio data, wherein the subtitles in the ancient poetry video data are determined according to the text of the metadata, the ancient poetry audio data is used as the audio of the ancient poetry display video data, the ancient poetry image data is automatically rotated to form a carousel video, and the carousel video is used as the video display of the ancient poetry video data.

7. The display device according to claim 6, characterized in that The controller performs semantic understanding of the ancient poetry based on the metadata to obtain the semantic data of the ancient poetry, and is specifically configured as follows: Preprocessing the content of the metadata to obtain first content; the preprocessing includes word segmentation, removal of stop words and part-of-speech tagging; Analyze the semantics of the first content based on the semantic analysis model to obtain first data; The first data is used to represent the interpretation of the ancient poetry; determining a first keyword for the first content based on the first data; the first keyword includes words describing scenes and scenery in the first content; Determine a second keyword of the first content based on the first data; the second keyword includes a word describing emotion in the first content; The first data, the first keyword and the second keyword are used as the ancient poetry semantic data.

8. The display device according to claim 6, characterized in that The controller performs background retrieval on the ancient poetry based on the metadata to obtain the background data of the ancient poetry, and is specifically configured as follows: Retrieving first background data of the author in the metadata, where the first background data is used to represent background data related to the author; Determine the creation time of the ancient poem according to the title and the author in the metadata; Determine the second background data of the ancient poem according to the creation time and the title; The second background data is used to represent the creation background data related to the author's creation of the ancient poem; Determine the third background data of the ancient poem according to the creation time; The third background data is used to represent the era background data related to the creation time; The first background data, the second background data and the third background data are used as the ancient poetry background data.

9. The display device according to claim 1, characterized in that: The controller executes, in response to receiving a start instruction for a first application, before starting the first application, the controller is further configured to: In response to receiving a second voice instruction for displaying an ancient poem, starting a second application, and displaying the text of the metadata of the ancient poem and a first control on a display page of the second application, wherein the second application is at least used to search for and display the ancient poem, and the first control is used to display an icon of the first application; The text of the metadata is kept displayed on the display page of the second application, the startup instruction from the user is obtained, and the first application is started.

10. A display device, characterized in that: include: monitor; A communicator configured to communicate with the server; A controller is coupled to the display and the communicator and is configured to: In response to receiving a start instruction for a first application, the first application is started; the first application is at least used to display multimedia data of ancient poems; the multimedia data includes at least one of ancient poems image data, ancient poems audio data and ancient poems video data; Displaying a first display page of the first application, and acquiring a first voice instruction for displaying multimedia data input by a user on the first display page, wherein the first display page at least includes an input area, a first guide word for guiding the user to input the first voice instruction, and a guide example; In response to the first voice instruction, displaying text information corresponding to the input content of the first voice instruction in the input area; Controlling the communicator to send the text information to the server, so that the server obtains metadata of the ancient poetry according to the text information and generates multimedia data of the ancient poetry; wherein the metadata at least includes the title, author and content of the ancient poetry; Controlling the communicator to receive multimedia data from the server; Displaying a second display page of the first application; wherein the second display page includes text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed; In response to selecting the multimedia presentation control, the multimedia data is displayed on a third display page of the first application.

11. A server, characterized in that: include: A communicator configured to be communicatively connected with a display device; A processor is coupled to the communicator and is configured to: Controlling the communicator to receive text information from the display device, and obtaining metadata of the ancient poetry according to the text information; the metadata at least includes the title, author and content of the ancient poetry; Generate multimedia data of ancient poetry according to the metadata; The communicator is controlled to send the multimedia data to the display device, so that the display device displays the multimedia data on the third display page of the first application.

12. A method for displaying multimedia data of ancient poetry, characterized in that: include: In response to receiving a start instruction for a first application, start the first application; The first application is at least used to display multimedia data of ancient poetry; The multimedia data includes at least one of ancient poetry image data, ancient poetry audio data and ancient poetry video data; Displaying a first display page of the first application, and acquiring a first voice instruction for displaying multimedia data input by a user on the first display page, wherein the first display page at least includes an input area, a first guide word for guiding the user to input the first voice instruction, and a guide example; In response to the first voice instruction, displaying text information corresponding to the input content of the first voice instruction in the input area; Determine metadata of the ancient poem that matches the text information according to the text information, wherein the metadata at least includes the title, author and content of the ancient poem; Generating multimedia data of the ancient poetry according to the metadata; After the multimedia data of the ancient poetry is generated, a second display page of the first application is displayed; wherein the second display page includes the text of the metadata and a multimedia display control, and the multimedia display control is used to determine the multimedia data to be displayed; In response to selecting the multimedia presentation control, the multimedia data is displayed on a third display page of the first application.

Images