Display device and content play mode switching method

Abstract

The application relates to a display device and a content playing mode switching method, and relates to the technical field of display devices. The display device comprises a display, an audio output device and a controller, wherein the controller is configured to: in response to a playing mode setting instruction during playing of media content in a first mode in the display device, control the display to display at least an option of the first mode and an option of a second mode in a current playing interface; in response to a selection instruction of a user, control the display to continue playing the current media content in the second mode, display a content picture of the current media content at a target playing position, and control the audio output device to output audio at the target playing position; wherein the target playing position is a playing progress position of the current media content when the user triggers the option of the second mode. The application can provide multiple content playing modes for the user, take into account the habits of different watching users, and maintain the continuity of content playing.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a method for switching content playback modes. Background Art

[0002] A display device is a device with image display and data processing capabilities, such as a smart TV. A smart TV is a television product based on Internet application technology, with an open application platform, integrating multiple functions such as audio, video, entertainment, and data.

[0003] Display devices have traditionally been used as video and audio playback devices. To further develop the value of display devices, more applications can be configured for them, such as configuring an application for viewing picture books on a smart TV.

[0004] The application program in the display device can play predefined picture book content, but the playback mode is relatively fixed and cannot take into account the habits of different viewing users. Summary of the Invention

[0005] Based on this, it is necessary to provide a display device and content playback mode switching method that can take into account the different viewing habits of users in order to address the above technical issues.

[0006] In a first aspect, the present application provides a display device, comprising:

[0007] A display configured to display information, the information including at least a content image of the media content played in the target application; the target application includes at least one item of media content;

[0008] an audio output device configured to output audio, the audio including at least audio of media content played in the target application;

[0009] The controller is coupled to the display and the audio output device respectively and is configured to:

[0010] During playback of current media content in a first mode, in response to a playback mode setting instruction, displaying content playback mode options on a current playback interface via a display, the content playback mode options including at least a first mode option and a second mode option, the first mode and the second mode corresponding to different content playback formats; and

[0011] In response to a trigger instruction for the second mode option, the current media content continues to be played from the target playback position in the second mode through the display, the target content picture at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device, and the duration of the target content picture displayed on the display corresponds to the broadcast duration of the audio corresponding to the target content picture; wherein, the target playback position corresponds to the playback progress of the current media content when the user triggers the second mode option.

[0012] Based on this embodiment, the display device can provide viewers with multiple content playback modes, and users can choose to switch content playback modes while viewing picture books. Based on the user's playback mode selection, the display device switches the currently playing media content to the new playback mode while maintaining the current media content playback progress. This accommodates the habits of different viewers while maintaining the continuity of content playback.

[0013] In one embodiment, one of the first mode and the second mode is a flipping book mode, which is a playback mode including a flipping book animation and a flipping book audio during content playback;

[0014] The controller is also configured to:

[0015] When the display plays the current media content in flipping mode, when the playback progress reaches a preset flipping animation time node or in response to a user's flipping instruction, the display displays a flipping animation and outputs flipping audio through the audio output device.

[0016] Based on this embodiment, the display device can play media content in a flip book mode, and play flip book animation and flip book audio automatically or based on user instructions, which is conducive to enhancing the interactivity with the user during the content playback process and improving the content playback effect.

[0017] In one embodiment, the other of the first mode and the second mode is a video mode;

[0018] The controller is also configured to:

[0019] In response to a start play instruction for the current media content, the current media content is played through the display in a preset default play mode; the default play mode is a video mode or a book flipping mode.

[0020] Based on this embodiment, the target application in the display device can pre-set a default playback mode. When starting to play media content, it will be played in the default playback mode first. The user does not need to set the playback mode every time the media content is played, which simplifies the user's setting operation and improves the continuity of content playback.

[0021] In one embodiment, the controller is specifically configured to:

[0022] After responding to a trigger instruction for the option of the second mode, if the second mode is different from the default play mode, the default play mode of the target application is updated with the second mode.

[0023] Based on this embodiment, the default playback mode of the target application is dynamically updated based on the user's mode switching operation, which can better meet the media content viewing habits of different users and reduce the user's operation of switching playback modes.

[0024] In one embodiment, the controller is specifically configured to:

[0025] In response to the playback mode setting instruction, pausing playback of the current media content via the display and displaying content playback mode options in the pause playback interface;

[0026] After receiving a trigger instruction from the user for any of the content play mode options, the current media content continues to be played through the display.

[0027] Based on this embodiment, the display device can pause content playback when receiving a user's playback mode setting instruction, preventing the user from missing key content in the media content during the mode switching operation, which is conducive to ensuring the user's experience in viewing media content.

[0028] In one embodiment, the controller is specifically configured to:

[0029] In response to the play mode setting instruction, when the current media content is paused through the display, saving the play progress of the current media content when the play is paused as the target play position; and

[0030] In response to a trigger instruction for the second mode option, based on the saved playback progress, the content picture and audio corresponding to the target playback position are obtained, the current media content is continued to be played from the target playback position in the second mode through the display, the target content picture of the current media content at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device.

[0031] Based on this embodiment, when the display device pauses the playback of media content, it saves the playback progress information of the media content in a timely manner, so that after the user selects a new playback mode, the playback can be continued in the new playback mode based on the saved content playback progress information. After switching, the user does not need to operate the playback progress, thereby reducing the operation process for the viewing user.

[0032] In one embodiment, the current media content includes multiple pages of content images and multiple audios, wherein a first correspondence exists between a single page of content images and a playback time interval, and a second correspondence exists between a single page of content images and a single audio.

[0033] The controller is also configured to:

[0034] Based on the first corresponding relationship, determining a playback time interval corresponding to the playback progress, and obtaining a single-page content image corresponding to the playback time interval as a target content image at the target playback position; and

[0035] Based on the second corresponding relationship, the audio corresponding to the target content picture is obtained as the audio at the target playback position.

[0036] Based on this embodiment, the display device can effectively determine which content picture and which audio segment to start playing again in a new playback mode based on the correspondence between a single-page content picture and the playback time, as well as the correspondence between a single-page content picture and the audio, without the need for manual selection by the user, thereby improving the continuity of content playback.

[0037] In one embodiment, the current media content includes multiple pages of content images and multiple audios, and there is a third correspondence between a single page of content images and multiple video frames, and a fourth correspondence between the video frames and the audios;

[0038] When the controller responds to a trigger instruction for the second mode option and acquires the content image and audio corresponding to the target playback position based on the saved playback progress, the controller is further configured to:

[0039] When the first mode is the video mode and the second mode is the flipping book mode, determining a target video frame corresponding to the playback progress;

[0040] Based on the third corresponding relationship, determining a content picture corresponding to the target video frame as the target content picture at the target playback position; and

[0041] Based on the fourth corresponding relationship, the audio corresponding to the target video frame is determined as the audio at the target playback position.

[0042] Based on this embodiment, since the granularity of the corresponding relationship is smaller, it is possible to obtain more accurate content images and audio at the target playback position, further reducing the overlap of content playback.

[0043] In one embodiment, the display device further comprises:

[0044] communication devices;

[0045] The controller is also configured to:

[0046] When the first mode is the video mode and the second mode is the flipping book mode, in response to a trigger instruction for the second mode option, obtaining an animation identifier and a first network address of a flipping book animation corresponding to the current media content, and obtaining an audio identifier and a second network address of a flipping book audio corresponding to the current media content; and

[0047] The communication device is controlled to access the first network address to obtain the book-turning animation corresponding to the animation identifier and to transmit it to the local display device. The communication device is controlled to access the second network address to obtain the book-turning audio corresponding to the audio identifier and to transmit it to the local display device.

[0048] Based on this embodiment, the display device obtains the book-turning animation and book-turning audio associated with the current media content locally through the animation identifier and network address of the book-turning animation and the audio identifier and network address of the book-turning audio, so as to play the current media content in book-turning mode.

[0049] In one embodiment, the controller is further configured to:

[0050] After receiving a trigger instruction from the user for any of the content play mode options, the content play mode option is hidden through the display.

[0051] Based on this embodiment, after the user selects the play mode through the display device, the content play mode option can be canceled from the display device to avoid affecting the user's viewing of media content.

[0052] In a second aspect, the present application provides a content playback mode switching method, which is applied to a display device, wherein the display device includes a display, an audio output device, and a controller;

[0053] The method includes:

[0054] During playback of media content in a target application in a first mode, in response to a playback mode setting instruction, displaying content playback mode options in a current playback interface via a display, the content playback mode options including at least a first mode option and a second mode option, one of the first mode and the second mode being a flipping book mode; and

[0055] In response to a trigger instruction for the second mode option, the current media content continues to be played from the target playback position in the second mode through the display, the target content picture at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device; and the duration for which the display displays the target content picture corresponds to the broadcast duration of the audio corresponding to the target content picture; wherein, the target playback position corresponds to the playback progress of the current media content when the user triggers the second mode option.

[0056] Based on this embodiment, when the target application is running on the display device, multiple content playback modes can be provided to the viewing user during the playback of the current media content in the first mode. The user can choose to switch the content playback mode while watching the media content. The display device switches the currently playing media content to the new playback mode based on the user's playback mode selection operation, while maintaining the current media content playback progress. This takes into account the habits of different viewing users while maintaining the continuity of content playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0059] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0060] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0061] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0062] Figure 5 A schematic diagram showing picture book playback mode options in one embodiment;

[0063] Figure 6 A schematic diagram showing picture book playback mode options in another embodiment;

[0064] Figure 7 A schematic diagram showing picture book playback mode options in yet another embodiment;

[0065] Figure 8 A schematic diagram of an interface for playing a picture book in flipping mode in one embodiment;

[0066] Figure 9 This is a flowchart of mode switching control during picture book playback in one embodiment;

[0067] Figure 10 This is a timing diagram of mode switching control during picture book playback in one embodiment;

[0068] Figure 11 is a flowchart of a method for switching content playback modes in one embodiment;

[0069] Figure 12 FIG. 4 is a structural block diagram of a content play mode switching device in an embodiment. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0071] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0072] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0073] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0074] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0075] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0076] In the embodiments of the present application, the display device 200 generally refers to a device capable of displaying images and processing data. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0077] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG, a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0078] The mobile terminal 300 can function as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, enabling connection and communication via a network communication protocol, enabling one-to-one control operations and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 for synchronized display.

[0079] like Figure 1 As shown in FIG, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0080] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart TV, Internet Protocol television (IPTV), etc.

[0081] Figure 2 Some embodiments of this application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0082] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0083] In some embodiments, detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 may include a light receiver, such as a sensor for collecting ambient light intensity; or an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or a sound collector, such as a microphone, for receiving external sounds.

[0084] In some embodiments, the display 260 includes a display component for displaying images and a driver component for driving image display. The display 260 is configured to receive image signals output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces.

[0085] In some embodiments, a touch component connected to the display 260 is further included. The touch component is used to receive a user's touch operation, and the display 260 is also used to display a display effect corresponding to the user's touch operation.

[0086] In some embodiments, the communication device 220 is a component used to communicate with an external device or server 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 depending on the supported communication methods. For example, if the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including WiFi functionality. If the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including Bluetooth functionality.

[0087] The communication device 220 can establish a communication connection between the display device 200 and an external device or server 400 via a wireless or wired connection. A wired connection can connect the display device 200 to an external device via a data cable, an interface, or other components. A wireless connection can connect the display device 200 to an external device via a wireless signal or wireless network. The display device 200 can establish a connection with an external device directly or indirectly through a gateway, router, or connection device.

[0088] In some embodiments, the controller 250 may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processor, and a power processor, and first to nth interfaces for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in a memory. The controller 250 controls the overall operation of the display device 200.

[0089] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0090] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0091] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may further be provided with an external audio output terminal, through which the audio output device may be connected to the display device 200 to output the sound of the display device 200.

[0092] In some embodiments, the user input interface 280 may be configured to receive instructions from a user.

[0093] Figure 3 Some embodiments of this application provide Figure 1 The hardware configuration diagram of the control device in the figure is as follows. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0094] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0095] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0096] In some embodiments, as Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0097] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and external and internal data processing functions.

[0098] Under the control of the controller 110, the communication interface 130 communicates control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, or other near field communication modules.

[0099] The user input / output interface 140 includes at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 and other input interfaces.

[0100] In some embodiments, the buttons 144 configured on the control device 100 may include basic buttons, such as a power button, number keys, direction keys, volume keys, home key, back key, etc. The user can control the display device 200 to implement corresponding functions based on these basic buttons. The buttons 144 configured on the control device 100 may also include special buttons, such as color keys, etc. The user can control the display device 200 to implement specific functions in different scenarios based on these special buttons.

[0101] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the display device 200.

[0102] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0103] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0104] To facilitate user interaction, in some embodiments, display device 200 may run an operating system. An operating system is a computer program used to manage and control the hardware and software resources of display device 200. The operating system can provide a user interface (control the display device), allow users to interact with display device 200, and support the running of various application programs.

[0105] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for display devices.

[0106] The operating system can be divided into different modules or layers according to the functions implemented, e.g. Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the application layer (abbreviated as "application layer"), the application framework layer (abbreviated as "framework layer"), the system library layer and the kernel layer.

[0107] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on these applications. The application layer can host at least one application, which can include built-in window programs, system settings programs, clock programs, and other applications provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0108] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0109] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to the system location service; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0110] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling application exit, opening, and back. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling changes in display windows, such as shrinking, shaking, or distorting the display window.

[0111] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0112] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200, and the kernel layer can implement functions such as hardware abstraction, multi-tasking, and memory management. Figure 4 As shown, the kernel layer can be configured with hardware drivers, and the drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0113] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0114] Display devices are typically used as audio and video playback devices, such as for watching online TV or Internet Protocol Television (IPTV). Their value proposition needs to be further developed. On the other hand, creating picture books previously required professional drawing skills and literary literacy, which could be a challenge for children and parents.

[0115] In view of this, the embodiments of the present application propose a display device with a pre-set target application. Users can create their own media content or view existing media content in the target application by simply interacting with the large TV screen. After entering the target application through the display device application homepage, the user moves the focus to select the media content they want to watch from the media content list. Then, based on voice commands or confirmation operations on the control device 100, the selected media content can be played on the display device.

[0116] In an embodiment of the present application, the target application includes preset media content, and the pictures and audio contained in each media content are related to the content text. When the media content is played, the playback time of the pictures and content text therein corresponds to the broadcast time of the audio content corresponding to the picture.

[0117] Typically, a display device plays media content in a fixed format. For example, it defaults to playing media content in video mode. During video playback, content images are played as video frames, allowing a single content image to appear dynamic. Simultaneously, the audio output device of the display device outputs a voice announcement corresponding to the currently displayed content image.

[0118] As a specific example, the target application may be a picture book application for creating and playing picture books. Accordingly, the application may have the following features:

[0119] The Picture Book app comes with multiple pre-installed picture books and allows users to create new ones. Each picture book includes multiple pictures, and the characters in these pictures are consistent. Each picture book picture is based on a section of the story text in the picture book. Each picture book picture has an audio broadcast based on the story text corresponding to the picture book picture.

[0120] Select a picture book in the picture book app and confirm playback. The selected picture book can be played in video mode. During picture book video playback, the picture book image can be displayed statically or with sliding effects. The story text corresponding to the picture on the page can also be displayed in the form of subtitles on the picture book image, and the display duration of each picture book image corresponds to the duration of the audio broadcast of the picture book image. As an example, the switching display of two adjacent picture book images can be continuous, but the audio broadcast of the two adjacent picture book images is played at a certain interval (for example, 1 second). In addition, there can be background music during picture book playback, and the background music matches the story content of the picture book.

[0121] However, using picture book apps as an example, a fixed way of playing picture books makes it difficult to cater to the needs of different users in different scenarios. Therefore, optimizing the playback process of picture books and other content on display devices is an important factor in improving the user experience.

[0122] In the embodiments of the present application, a scenario in which a picture book application plays a picture book in a display device is used as an example for specific description. It can be understood that the picture book application in the following embodiments can also be other applications with similar characteristics.

[0123] In one embodiment of the present application, a display device includes:

[0124] The display is configured to display information, wherein the information at least includes content images of the media content being played in the target application; the target application includes at least one item of media content; and the audio output device is configured to output audio, wherein the audio at least includes the audio of the media content being played.

[0125] The display device further includes a controller, which is coupled to the display and the audio output device respectively and is configured to:

[0126] During playback of current media content in a first mode, in response to a playback mode setting instruction, controlling the display to display content playback mode options in a current playback interface, the content playback mode options including at least options for the first mode and options for the second mode;

[0127] In response to a trigger instruction for the second mode option, the current media content is played back in the second mode from the target playback position via the display, the target content image at the target playback position is displayed, and the audio corresponding to the target content image is output via the audio output device; wherein the duration of the target content image displayed on the display corresponds to the duration of the audio corresponding to the target content image. The target playback position corresponds to the playback progress of the current media content when the user triggers the second mode option.

[0128] The first mode and the second mode are different content playback modes, and different content playback modes correspond to different content playback forms and present different playback effects. Specifically, the content playback modes include video mode, book flipping mode, etc.

[0129] In one embodiment, the effect of displaying the picture book play mode option in the current play interface can be as follows: Figure 5 As shown. The options of the first mode and the options of the second mode are displayed on one side of the current playback interface, such as on the right side in the example in the figure. It is understandable that the display method and display effect of the options of the first mode and the second mode can also be other forms, such as being displayed side by side at the bottom of the current playback interface.

[0130] In one example, when the picture book play mode options are displayed in the current play interface, the options of the first mode currently used can be specially marked. Figure 6 As shown, the option of the first mode currently used is marked with a five-pointed star to let the user know the playback mode currently used by the picture book, so that the user can choose to switch to a new playback mode.

[0131] Among them, the triggering instruction of the content playback mode option can be broadly understood, including moving the focus to the position of a certain content playback mode option (that is, selecting a certain mode option), and issuing a confirmation instruction based on the mode option where the current focus is located (that is, triggering a certain content playback mode option).

[0132] In one embodiment, continuing to take the switching of the picture book play mode as an example, when an option in the picture book play mode is selected, its display state can be different from the state when it is not selected. Figure 7As shown, the option in the second mode is selected, and the display effect of this option is more prominent than the unselected option in the first mode.

[0133] In one embodiment, continuing with the example of switching picture book playback modes, playback mode setting instructions / triggering instructions for picture book playback mode options can be issued based on the control device 100. For example, the user operates the setting button on the control device 100 to call up the playback settings menu from the picture book playback interface, enters the playback mode settings submenu through the playback settings menu, and continues to operate the corresponding button on the control device 100 to enter the playback mode settings submenu, thereby displaying at least two picture book playback mode options on the current playback interface. By operating the volume button on the control device 100 to move the focus to one of the at least two picture book playback mode options, and clicking the confirmation button on the control device 100 to issue a triggering instruction for the picture book playback mode option.

[0134] In another embodiment, continuing with the example of switching the picture book playback mode, the playback mode setting instruction / trigger instruction for the picture book playback mode option can be a voice instruction. For example, the display device is turned on and the voice service is turned on. By pressing the voice button on the remote control to activate the voice service, you can call up the playback settings menu from the picture book playback interface by saying "settings" or "play settings" to the remote control. Press the voice button on the remote control to activate the voice service, and say "switch playback mode" or "set playback mode" to the remote control to enter the submenu of the playback mode settings. Continue to say "select first mode" or "select second mode" to the remote control to issue a trigger instruction for the picture book playback mode option.

[0135] In the embodiment of the present application, the display device 200 can obtain the user's spoken audio through the sound collector and perform voice recognition on the user's audio to determine the degree of match between the voice recognition and the preset picture book playback control instruction. When the match is greater than or equal to a preset match threshold, the corresponding picture book playback control instruction is responded to.

[0136] In another embodiment, continuing with the example of switching picture book playback modes, the playback mode setting instruction / triggering instruction for a picture book playback mode option can be a touch instruction. For example, when the display of the display device includes a touch screen and the touch function of the touch screen is turned on, the user touches the touch screen to call up a playback settings menu from the picture book playback interface, enters a submenu of the playback mode settings through a touch operation on the touch screen, and displays at least two picture book playback mode options in the current playback interface. By touching one of the at least two picture book playback mode options on the touch screen, a triggering instruction for the picture book playback mode option can be issued.

[0137] In another embodiment, continuing with the example of switching picture book playback modes, the playback mode setting instruction / the triggering instruction for the picture book playback mode option can be an air gesture instruction. For example, if the display device is provided with a detector, such as a camera, and the gesture operation function of the display device is turned on, the user can use a pre-set gesture to issue the playback mode setting instruction and / or the triggering instruction for the picture book playback mode option through the air. Compared with the control device 100 and touch screen operation, the air gesture operation is more convenient to operate.

[0138] The target playback position is the playback progress position of the current media content when the user triggers the second mode option. Therefore, the target playback position can be any position during content playback. Continuing with the picture book playback mode switching example, if the current picture book contains 10 pages of picture books, the target playback position can be the playback position corresponding to the picture on page 1 or page 10, or it can also be the playback position corresponding to a picture on a page in the middle. There is no restriction on this.

[0139] A picture book image can be played with multiple picture book audio announcements. Therefore, the picture book audio at the target playback position can be a single sentence from the multiple picture book audio announcements for the current picture book image. The current playback progress can be recorded in real time during picture book playback. The current playback progress can include the current picture book image ID and playback time progress. Based on the recorded current picture book image ID and playback time progress, the picture book image and picture book audio at the target playback position can be determined.

[0140] In other examples, upon receiving a playback mode setting instruction, the current picture book playback can be paused and the current playback progress (including the current picture book image ID and playback time progress) can be obtained. Based on the obtained current picture book image ID and playback time progress, the picture book image and picture book audio at the target playback position can be determined. Compared with the real-time recording method, this method can further reduce background resource overhead.

[0141] In some cases, the content playback mode switching process takes a certain amount of time. For the viewing user, their experience may be that there is a waiting time after the playback mode selection instruction is issued before the current media content resumes playing in the selected new playback mode. To address this, in one embodiment, in response to the triggering instruction for the content playback mode option, before the current media content is played in the second mode, a prompt window can also be displayed on the display to remind the viewing user that the content playback mode switching process is currently in progress. This improves the user experience during the content playback mode switching process and prevents users from mistakenly believing that the current playback is stuck and triggering other operations.

[0142] The above-mentioned display device, when the target application is running on the display device and the current media content is being played, can respond to a playback mode setting instruction and provide the viewing user with multiple content playback mode options, and the user can select any content playback mode option. Based on the user's triggering operation, the display device can switch the currently playing media content to a new playback mode while maintaining the current media content playback progress. This takes into account the habits of different viewing users and the continuity of content playback.

[0143] In one embodiment, one of the first mode and the second mode is a flip book mode, which is a playback mode that includes a flip book animation and a flip book audio during content playback. The controller is further configured to:

[0144] When the display plays the current media content in flipping mode, when the playback progress reaches a preset flipping animation time node or in response to a user's flipping instruction, the display displays a flipping animation and outputs flipping audio through the audio output device.

[0145] Let’s take the above picture book application as an example:

[0146] When playing a picture book in flip book mode, a flipping animation is added to the display effect. The flipping animation uses the current and next frames to create the page-turning animation effect. A corresponding audio broadcast of the page turning is also added to the picture book audio. The corresponding visual effect of playing a picture book in flip book mode is: first frame picture book image + flipping animation + second frame picture book image. The audio effect is: first frame broadcast audio + flipping sound + second frame broadcast audio.

[0147] When playing a picture book in flip book mode, if the flip book animation is embedded in the picture book video as video frames, it can be played in flip book mode through a video player. If the flip book animation is added in a separate Flash format, it can also be played in flip book mode without a video player, for example, using a tool that can play images and Flash animations.

[0148] Among them, the page turning control of the picture book in the flipping book mode can automatically realize the flipping book animation based on the pre-set time node, that is, when the playback progress reaches the preset flipping book animation time node, the flipping book animation can be played and the flipping book sound is output; for example, when the picture book is played in the flipping book mode through a video player, the flipping book animation can be automatically realized based on the pre-set time node. It is also possible to turn the page based on the user's flipping book instruction, and the flipping book animation is played and the flipping book sound is output accordingly. For example, when the picture book is played in the flipping book mode with other tools, the user can control the flipping of the book. In the case of turning the page based on the user's flipping book instruction, the flipping book instruction can be implemented by the control device 100, voice or air gesture.

[0149] It should be noted that, in the embodiment of the present application, the book turning mode is also referred to as the page turning mode. Correspondingly, the description of turning the book content is sometimes also described as turning pages.

[0150] In one embodiment, the visual effects corresponding to the picture book can be played in the flip book mode. Figure 8 As shown, the book turning effect is controlled by the book turning angle and page turning length.

[0151] In this embodiment, when the display device plays a picture book in flipping mode, it introduces visual animation effects and sound effects of flipping books, and can also introduce more user page-turning interactive operations, which can simulate the reading experience of traditional paper picture books and meet the playback needs of different users.

[0152] In one embodiment, the other of the first mode and the second mode is a video mode; the corresponding controller is further configured to:

[0153] In response to a start play instruction for the current media content, the current media content is played through the display in a preset default play mode; the default play mode is a video mode or a book flipping mode.

[0154] The default playback mode may be one of the content playback mode options in the above-mentioned embodiments. For example, if a display device has three content playback modes, the default playback mode may be one of the three, and the content playback mode option provides options for the three modes. The default playback mode may also be a mode other than the content playback mode options in the above-mentioned embodiments. For example, if a display device has three content playback modes, the default playback mode may be one of the three, and the content playback mode option provides options for the other two of the three.

[0155] In one example, using the aforementioned picture book app as an example, the default playback mode can be set based on the picture book app's playback settings function. For example, during the picture book app's SDK development, the default playback mode is set based on the playback settings function and is updated when the picture book app is updated. In another example, the default playback mode can be exposed to the user through a setting interface. After entering the picture book app, the user can set the default playback mode through the playback settings function.

[0156] In this embodiment, after the display device enters the target application, when the user selects a media content in the media content list and instructs to play the picture book, the current media content will be played in the default playback mode. This eliminates the need for the user to select a content playback mode each time they instruct to play the content, which helps simplify user operations.

[0157] In an optional embodiment, if a user switches from the default playback mode to another playback mode using the content playback mode option during content playback on the display device, the display device can also update the default playback mode of the target application with the user-selected playback mode. The next time content playback is instructed, the media content will be played in the user-selected playback mode, better matching the user's picture book viewing habits.

[0158] In one embodiment, the controller is specifically configured to:

[0159] When the default playback mode is video mode, in response to a start playback instruction, the video player provided by the display device is controlled to start, the media resources corresponding to the current media content are loaded into the video player, and the current media content is played through the display based on the video player.

[0160] Let’s take the above picture book application as an example:

[0161] When the default playback mode is flipbook mode, determine whether the picture book animation of the current picture book is compatible with the display device's built-in video player. If compatible, control the display device's built-in video player to start, load the media resources corresponding to the current picture book into the video player, and play the current picture book through the display based on the video player. If not compatible, control the display device's built-in video player to start, and play the current picture book through the display device's built-in image player.

[0162] In order to achieve the mutual switching of playback modes and ensure the continuity of playback progress to the greatest extent. As an example, the target data format corresponding to each picture book can be pre-defined, and the target data format can specifically include the picture book identifier, picture book storage address information, picture book title information and picture book picture list information. Among them, the picture book picture list information can specifically include the picture identifier of each page of the picture book, the storage address of each page of the picture book, and the storage address of the audio file corresponding to each page of the picture book. If it is a flipping picture book, the picture book picture list information can also include the time node of the flipping animation, the identifier of the flipping animation, the network address of the flipping animation, and the identifier and network address of the flipping audio.

[0163] Correspondingly, loading the media resource corresponding to the current picture book into the video player may be loading information of the target data format of the current picture book into the video player.

[0164] In this embodiment, when the default playback mode of the target application is video mode, when the user instructs content playback, the pre-installed video player in the display device is automatically started, and the content playback state can be quickly entered without installing other players.

[0165] In one embodiment, the controller is specifically configured to:

[0166] During playback of the current media content in the first mode, in response to a playback mode setting instruction, the current media content is paused through the display, and content playback mode options are displayed in the currently paused playback interface; after receiving a user trigger instruction for any of the content playback mode options, the current media content continues to be played through the display.

[0167] It is understandable that in other embodiments, when the current media content is played in the first mode, in response to the playback mode setting instruction, the current media content is not paused through the display, and the content playback mode option is displayed in the playback interface of the current media content.

[0168] Through this embodiment, the display device can pause content playback when receiving a playback mode setting instruction from the user, thereby preventing the user from missing key content during the mode switching operation, which is conducive to ensuring the user's viewing experience.

[0169] In one embodiment, the controller is specifically configured to:

[0170] In response to the play mode setting instruction, when the current media content is paused through the display, saving the play progress of the current media content when the play is paused; and

[0171] In response to a trigger instruction for the second mode option, the content picture and audio corresponding to the target playback position are obtained based on the saved playback progress; then, the current media content continues to be played from the target playback position in the second mode through the display, the target content picture of the current media content when it is at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device.

[0172] Take the above-mentioned picture book application as an example: the playback progress can be understood as the playback time of the current picture book. For example, the total length of the current picture book is 10 minutes, and it has been played to the 3rd minute when it is paused. Then the saved playback progress is 3 minutes, and the target playback position is also the 3rd minute position.

[0173] As an example, in the target data format defined for the current picture book, the picture book image list information includes the image ID, storage address, and storage address of the corresponding audio file for each picture page in the picture book. In addition, it can also include the picture book video frame information corresponding to each picture page, that is, the correspondence between a single picture page and the playback time interval. Based on the playback progress, it can determine which picture book image and which picture book audio to continue playing the current picture book in the new mode.

[0174] Therefore, when switching playback modes, the current media content is paused to avoid missing content due to switching playback modes, which could affect the user's viewing experience. By saving the current playback progress, playback can be resumed from the corresponding position after the user selects a new playback mode, ensuring the continuity of content playback.

[0175] In one embodiment, the current media content includes multiple pages of content images and multiple audio tracks, wherein a first correspondence exists between a single page of content images and a playback time interval, and a second correspondence exists between a single page of content images and a single audio track. When executing the saved playback progress and obtaining the content images and audio tracks corresponding to the target playback position, the controller is further configured to:

[0176] Determining the playback time interval corresponding to the playback progress based on a first correspondence between the single-page content image of the current media content and the playback time interval, and obtaining the single-page content image corresponding to the playback time interval as the target content image at the target playback position; and

[0177] Based on the second correspondence between the single-page content image of the current media content and the audio, the audio corresponding to the target content image is obtained as the audio at the target playback position.

[0178] Let's take the picture book application described above as an example: The first correspondence between a single-page content image and a playback time interval, for example, the first picture in a picture book, corresponds to a playback time interval of 0 to 1 minute. The second correspondence between a single-page content image and audio can be understood as generating text content for a single-page picture book image and converting it through TTS to obtain a picture book audio segment. Therefore, a single-page picture book image corresponds to a picture book audio segment. The playback time of this picture book audio segment can correspond to the playback time of the single-page picture book image. For example, if the first picture book image in a picture book is generated based on the first section of text content in the picture book story, the picture book audio obtained through TTS conversion of the first section of text content is the picture book audio corresponding to the first picture book image, and the corresponding playback time period of this picture book audio segment is also 0 to 1 minute. It is understandable that the playback time period corresponding to the picture book audio segment can also be slightly shorter or slightly longer than the playback time period of the corresponding picture book image.

[0179] In this embodiment, when playback of the current media content is paused, the video and audio corresponding to the single-page content image may have been partially played. In this case, the first correspondence between the single-page content image and the playback time interval, and the second correspondence between the single-page content image and the audio, determine the target content image and its corresponding audio. When resuming playback of the current media content in the second mode, playback can be resumed from the beginning of the playback time interval corresponding to the target content image.

[0180] In other words, in this case, the audio corresponding to the target content image will partially overlap before and after the playback mode is switched. However, since the playback time corresponding to a single page of content images is generally short, this overlap has little impact on user viewing.

[0181] Through this embodiment, the display device can effectively determine which content picture and which audio segment to start playing again in a new playback mode based on the correspondence between a single-page content picture and the playback time, as well as the correspondence between a single-page content picture and the audio, without the need for manual selection by the user, thereby reducing the inconvenience caused by mode switching to the user.

[0182] In another embodiment, the current media content includes multiple pages of content images and multiple audios, and there is a third correspondence between a single page of content images and multiple video frames, and a fourth correspondence between the video frames and the audios;

[0183] When the controller responds to a trigger instruction for the second mode option and acquires the content image and audio corresponding to the target playback position based on the saved playback progress, the controller is further configured to:

[0184] When the first mode is the video mode and the second mode is the flipping book mode, determining a target video frame corresponding to the playback progress;

[0185] Based on the third corresponding relationship, determining a content picture corresponding to the target video frame as the target content picture at the target playback position; and

[0186] Based on the fourth corresponding relationship, the audio corresponding to the target video frame is determined as the audio at the target playback position.

[0187] For example, in the picture book application described above, there is a third correspondence between a single-page image and multiple video frames. This can be understood as interpolation performed when generating the video picture book, resulting in multiple picture book video frames corresponding to the single-page image. This third correspondence reflects this relationship.

[0188] Among them, there is a fourth corresponding relationship between video frames and audio. It can be understood that when generating a video picture book, a section of picture book audio corresponding to a single-page picture book image is further segmented and marked, so that each picture book video frame corresponding to the single-page picture book image has a corresponding picture book audio.

[0189] Based on this embodiment, compared with the previous embodiment, since the granularity of the correspondence between a single-page content picture and a video frame, and a video frame and audio is smaller, when the display device responds to a trigger instruction for the second mode option and obtains the content picture and audio corresponding to the target playback position based on the saved playback progress, it can obtain more accurate content pictures and audio, continue to play the determined content pictures and audio, and further reduce the content playback overlap.

[0190] The following will be combined with the Figure 9 and attached Figure 10 Taking the picture book application on a smart TV as an example, a specific example of the picture book playback mode switching process between video mode and flipping mode is provided.

[0191] As an example, for each picture book in the picture book application, the corresponding target data format is pre-defined when generating the picture book as follows:

[0192] aibook{

[0193] videoURL: URL address of the corresponding picture book

[0194] title picture book title

[0195] AibookPageInfo list[]

[0196] }

[0197] AibookPageInfo{

[0198] ImageUrl represents the image of each page in the picture book

[0199] AudioURL represents an audio file

[0200] / / Video time playback correspondence

[0201] Int timeStart The start time of the video playback segment

[0202] Int timeEnd The end time of the video playback segment

[0203] / / Picture book fragment content

[0204] Sting content

[0205] }

[0206] Furthermore, based on the start and end times of the video playback segments and the number of picture book images in the aibook, the playback time information corresponding to each picture book page can be determined. The playback time information corresponding to each picture book page can be pre-calculated and saved in the picture book image list information of the picture book.

[0207] When generating picture books, the cloud also defines the correspondence between single-page picture book images and playback time intervals, and between single-page picture book images and picture book audio. In addition, for flipbook picture books, the corresponding flipping animation and flipping audio are also defined.

[0208] The smart TV downloads the picture book resource information generated in the cloud to the local computer, and can download the picture book according to the target data format of the picture book in the above embodiment, thereby obtaining the media resource information of the picture book.

[0209] Smart TVs play picture books according to the default playback mode configured in the picture book app.

[0210] If the picture book application is configured with video mode by default, the platform's built-in player (such as mediaPlayer) is started, and the media resource information of the picture book is loaded into the video player, thereby playing the picture book in video mode.

[0211] On the picture book playback interface of a smart TV, users can switch from video mode to flip book mode by operating the video settings menu. Specifically:

[0212] (1) When the user switches from video mode to flip book mode

[0213] First, determine whether the current playback mode is flip book mode. If so, do not switch and continue playing the current picture book.

[0214] If not, the target picture book image is determined based on the corresponding relationship between the playback time and the single-page picture book image according to the target playback position of the current picture book. The smart TV displays the determined target picture book image and simultaneously reads and plays the picture book audio corresponding to the target picture book image.

[0215] Before playing audio, you need to initialize the audio playback interface. For example, you can load the AudioTrack class on the Android platform to read and play the picture book audio corresponding to the target picture book image.

[0216] When playing the current picture book in flip book mode, when the predefined flip book animation node is reached, the flip book animation is displayed and the flip book audio is output.

[0217] (2) When the user switches from flip book mode to video mode:

[0218] First, determine whether the current playback mode is video mode. If so, do not switch and continue playing the current picture book.

[0219] If not, for example, the current playback mode is the flip book mode, then the corresponding video playback position in the video mode is located according to the information of the picture book in the current flip book mode.

[0220] If the video player is not started in the flip book mode, the platform's built-in video player is started first, and the video playback position determined in the previous step is jumped to. The smart TV displays the picture book video frame corresponding to the video playback position and outputs the corresponding picture book audio.

[0221] In video mode, when the video frame corresponding to the next drawing picture is reached, the audio broadcast of the picture book corresponding to the next drawing picture is output.

[0222] As an example, the switching process between the video mode and the flip book mode of the current picture book can be implemented according to the data correspondence in the picture book image list of the current picture book. The switching strategy is as follows:

[0223] When the display pauses playing the current picture book, obtain the current video playback status, such as obtaining and recording the current playback progress through the parameter CurrentPlayPosition;

[0224] Based on the current playback progress, the program sequentially traverses the picture book image list AibookPageInfo list to find the picture book image identifier AibookPageInfo corresponding to the playback progress. Based on the obtained picture book image identifier aibookPageInfo, the program uses the corresponding address information aibookPageInfo.ImageUrl to obtain the corresponding picture book page image and the corresponding audio address aibookPageInfo.AudioURL to obtain the audio data corresponding to the current playback progress.

[0225] Thus, when the display device continues to play the current picture book in the new mode, it can jump to the picture book image corresponding to the current playback progress and output the corresponding picture book audio.

[0226] In addition to the above implementations, the playback time information corresponding to each picture book page in a picture book can also be calculated in real time. For example, upon receiving a playback mode switch instruction, the playback time information corresponding to each picture book page is calculated, and then the playback time interval corresponding to the current playback progress is determined, and then the picture book image corresponding to the current playback progress is determined. Although this method is real-time calculation, the calculation method is simple and still meets the timeliness requirements of picture book playback.

[0227] In other embodiments, for the same picture book content, the server can pre-generate a video picture book and a flip book picture book. The main difference lies in the playback format, that is, the flip book picture book also includes flip animation and flip audio. Therefore, the video picture book can be generated first, and then the corresponding flip book picture book can be generated based on the video picture book.

[0228] As an example, for a picture book content, after generating a video picture book, the playback time of the picture book is obtained based on the start time and the end time of the video playback segment of the picture book. The playback time of a single-page picture book image is obtained based on the playback time and the number of picture book images. A flipping animation is added based on the playback time of the single-page picture book image. One case is to add the flipping animation while keeping the total playback time of the picture book unchanged. Another case is to increase the time for displaying the flipping animation based on the total playback time of the video picture book. Optionally, the duration of the flipping animation can be determined based on the playback time of the single-page picture book image, for example, not exceeding the playback time of the single-page picture book image.

[0229] On this basis, continue to refer to Figure 10 , Figure 10 This is a timing diagram corresponding to the picture book playback mode switching process. The process involved in the playback mode switching process of this application includes:

[0230] 1. The picture book app homepage on the smart TV loads the media resources corresponding to the picture book. The corresponding process is as follows: 2. The homepage requests the picture book resources from the cloud media resource library. 3. The cloud media resource library returns the corresponding picture book media resources to the picture book app homepage.

[0231] 4. The picture book application homepage determines the current default playback mode. 5. Correspondingly, the homepage requests the playback settings module to obtain the default playback mode settings. 6. The playback settings module returns the default playback mode settings.

[0232] 7. If the default playback mode is video mode, the playback page will be launched. 8. The playback page will get the current video playback position (if it is starting playback, the playback position will be the start position). 9. The platform video player will play the video based on the relevant information loaded by the playback page.

[0233] 10. The user issues a playback mode switching instruction based on the playback setting function module and chooses to switch to the page-turning mode (i.e., the book-turning mode mentioned above); correspondingly, 11. The playback page obtains information on switching to the book-turning mode from the playback setting function module; 12. Based on the current video playback position, the picture book video frame index (index is a serial number or index information) is obtained, and the target picture book image corresponding to the picture book video frame index is determined; 13. The corresponding picture book audio is obtained through the target picture book image; or the corresponding picture book audio is obtained through the picture book video frame index.

[0234] If the current video playback position is not the playback end position, continue with the subsequent process.

[0235] 14. The playback page stops playing the current video; this step can also be performed after 11 and before 12.

[0236] 15. The playback page directly displays the target picture book image. 16. At the same time, the playback page requests the picture book audio data from the PCM audio playback module. PCM (Pulse Code Modulation) refers to the process of encoding and playing audio signals in digital form. 17. The PCM audio playback module registers the playback status. Before PCM audio can play, this registration typically involves setting audio stream parameters such as sampling rate, number of channels, and format. 18. The PCM audio playback module returns a callback message indicating the picture book audio playback status, which carries the configured parameters.

[0237] 19. If the audio playback of the current picture book image is completed, get the picture of the next page of the picture book, and display the flipping animation and audio at the same time.

[0238] 20. If there is a picture on the next page, repeat steps 15 to 19.

[0239] 21. In another case, if the user selects to switch from the flipping book mode to the video mode based on the playback setting function module.

[0240] 22. The playback page obtains the switching information from the playback setting function module. After that, the playback time information corresponding to the current picture book image identifier can be obtained based on the current picture book image identifier, thereby obtaining the video playback position corresponding to the current picture book image identifier, thereby ensuring that the video jumps to the corresponding video when the video starts.

[0241] Based on the above example, on the one hand, users can quickly enter the picture book playback state based on the smart TV, and on the other hand, they can quickly realize the picture book data processing from video mode to book flipping mode, thereby improving the efficiency of switching playback modes.

[0242] In one embodiment, the controller is further configured to:

[0243] After receiving a trigger instruction from the user for the first mode option or the second mode option, the picture book play mode option is hidden through the display.

[0244] Through the method of this embodiment, after the user selects the play mode through the display device, the picture book play mode option can be canceled from the display device to avoid affecting the user's viewing of the picture book.

[0245] In one embodiment, the controller is further configured to: in response to a trigger instruction for the option of the first mode, continue to play the current picture book in the first mode through the display.

[0246] As an example, in response to the selection instruction for the picture book playback mode option, the controller continues to play the current picture book in the first mode through the display when the current picture book playback mode is the first mode and the user selects the option of the first mode; or

[0247] In response to the selection instruction for the picture book play mode option, the controller continues to play the current picture book in the second mode through the display when the current picture book play mode is the second mode and the user selects the option of the second mode.

[0248] In one embodiment, during the process of generating the current picture book playback, if hidden elements or multiple story branches are pre-set for the current picture book, the hidden elements may be character dialogues, item interactions, scene changes, etc. During the picture book playback process, the user may also perform other interactive operations based on the picture book playback page. Correspondingly, the picture book playback page transmits the user's interactive information to the server, that is, transmits to the server which interactive element the user has currently triggered.

[0249] The server generates corresponding feedback based on the user's interactive information, including the character's reaction, the story's progress, the appearance of new interactive elements, etc., and returns this feedback information to the picture book playback page. The picture book playback page updates the picture book content and the currently playing picture book screen synchronously.

[0250] In an exemplary embodiment, Figure 11 As shown, a content playback mode switching method is provided, which is applied to Figure 1 The display device 200 in FIG. 1 is taken as an example to illustrate the method, which includes the following steps S202 to S204. In which:

[0251] S202: In response to a playback mode setting instruction, during playback of media content in a target application in a first mode, displaying content playback mode options in a current playback interface via a display, the content playback mode options including at least a first mode option and a second mode option, the first mode and the second mode corresponding to different content playback formats; and

[0252] S204: In response to a trigger instruction for selecting the second mode, continue playing the current media content from the target playback position in the second mode via the display, display the target content image at the target playback position, and output audio corresponding to the target content image via the audio output device;

[0253] The duration for which the target content image is displayed on the display corresponds to the duration for which the audio corresponding to the target content image is played;

[0254] The target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

[0255] The above-mentioned content playback mode switching method, when the target application is running in the display device, the current media content is being played, in response to the playback mode setting instruction, controls the display of the display device to display the content playback mode options in the current playback interface, and the content playback mode options include at least the first mode option and the second mode option; in response to the user's triggering instruction for the content playback mode option, when the current media content playback mode is the first mode and the user selects the second mode option, the current media content is played in the second mode through the display, and jumps to the content image showing the current media content at the target playback position, and controls the audio output device of the display device to output the audio at the target playback position; wherein the target playback position corresponds to the playback progress of the current media content when the user triggers the second mode option. Through the present application scheme, multiple content playback modes can be provided to the viewing user, and the user can select any content playback mode during the viewing of the media content. The display device switches the currently playing media content to the new playback mode based on the user's playback mode selection operation, while keeping the current media content playback progress unchanged. In this way, the habits of different viewing users are taken into account while maintaining the continuity of content playback.

[0256] In one embodiment, one of the first mode and the second mode is a flipping book mode, which is a playback mode including a flipping book animation and a flipping book audio during content playback;

[0257] The method further includes: when the display plays the current media content in a flipping book mode, when the playback progress reaches a preset flipping book animation time node or in response to a user's flipping book instruction, displaying a flipping book animation through the display and outputting flipping book audio through an audio output device.

[0258] In one embodiment, the other of the first mode and the second mode is a video mode;

[0259] The method further includes: in response to a start playing instruction for the current media content, playing the current media content in a preset default playing mode via the display; the default playing mode is a video mode or a book flipping mode.

[0260] In one embodiment, the method further includes: in response to the play mode setting instruction, pausing the current media content via the display and displaying a content play mode option in the pause play interface;

[0261] After receiving a trigger instruction from the user for any of the content play mode options, the current media content continues to be played through the display.

[0262] In one embodiment, the method further comprises:

[0263] In response to the play mode setting instruction, when the current media content is paused through the display, saving the play progress of the current media content when the play is paused as the target play position; and

[0264] In response to a trigger instruction for the second mode option, based on the saved playback progress, the content picture and audio corresponding to the target playback position are obtained, and the current media content is continued to be played from the target playback position in the second mode through the display to display the target content picture of the current media content at the target playback position, and the audio corresponding to the target content picture is output through the audio output device.

[0265] In one embodiment, the method further includes: determining a playback time interval corresponding to the playback progress based on a first correspondence between a single-page content image of the current media content and a playback time interval, and obtaining a single-page content image corresponding to the playback time interval as a target content image at the target playback position; and

[0266] Based on the second correspondence between the single-page content image of the current media content and the audio, the audio corresponding to the target content image is obtained as the audio at the target playback position.

[0267] In one embodiment, the current media content includes multiple pages of content images and multiple audios, and a third correspondence exists between a single page of content images and multiple video frames, and a fourth correspondence exists between the video frames and the audios. In response to a trigger instruction for the second mode option, when obtaining content images and audios corresponding to a target playback position based on the saved playback progress, the method further includes:

[0268] When the first mode is the video mode and the second mode is the book-turning mode, the target video frame corresponding to the playback progress is determined; based on the third correspondence, the content picture corresponding to the target video frame is determined as the target content picture at the target playback position; and based on the fourth correspondence, the audio corresponding to the target video frame is determined as the audio at the target playback position.

[0269] In one embodiment, the method further includes: when the first mode is the video mode and the second mode is the flipping book mode, in response to a trigger instruction for the second mode option, obtaining an animation identifier and a first network address of a flipping book animation corresponding to the current media content, and obtaining an audio identifier and a second network address of a flipping book audio corresponding to the current media content; and

[0270] The communication device is controlled to access the first network address to obtain the book-turning animation corresponding to the animation identifier and to transmit it to the local display device. The communication device is controlled to access the second network address to obtain the book-turning audio corresponding to the audio identifier and to transmit it to the local display device.

[0271] In one embodiment, the method further includes: after receiving a trigger instruction from the user for any of the content play mode options, hiding the content play mode options through a display.

[0272] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0273] Based on the same inventive concept, embodiments of the present application also provide a content playback mode switching device. The solution provided by this device is similar to the solution described in the aforementioned content playback mode switching method. Therefore, the specific limitations of one or more of the following embodiments of the content playback mode switching device can be found in the aforementioned limitations of the content playback mode switching method and will not be further elaborated here.

[0274] In an exemplary embodiment, Figure 12 As shown, a content play mode switching device 1200 is provided, which is applied to a display device, and the display device includes a display, an audio output device and a controller;

[0275] The switching device 1200 includes:

[0276] a mode setting unit, configured to, in response to a playback mode setting instruction, display content playback mode options in a current playback interface via a display during playback of media content in a target application in a first mode, wherein the content playback mode options include at least a first mode option and a second mode option, wherein the first mode and the second mode correspond to different content playback formats; and

[0277] a switching control unit, configured to, in response to a trigger instruction for selecting the second mode, continue playing the current media content from the target playback position in the second mode via the display, display the target content image at the target playback position, and output audio corresponding to the target content image via the audio output device;

[0278] The duration for which the target content image is displayed on the display corresponds to the duration for which the audio corresponding to the target content image is played;

[0279] The target application includes at least one media content; the target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

[0280] Each module in the above-mentioned content playback mode switching device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in the processor of the display device in hardware form, or stored in the memory of the display device in software form, so that the processor can call and execute the corresponding operations of each module.

[0281] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned content playback mode switching method embodiments are implemented.

[0282] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above-mentioned content playback mode switching method embodiments are implemented.

[0283] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0284] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0285] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0286] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A display device, characterized in that: include: A display configured to display information, wherein the information at least includes a content image included in the media content played in the target application; the media content played is picture book content; an audio output device configured to output audio, the audio including at least audio of the media content being played; A controller is coupled to the display and the audio output device respectively, and is configured to: During playback of current media content in a first mode, in response to a playback mode setting instruction, pausing playback of the current media content via a display, saving the playback progress of the current media content when playback was paused as a target playback position, and displaying content playback mode options on a current playback interface via the display, the content playback mode options including at least a first mode option and a second mode option, the first mode and the second mode corresponding to different content playback formats; One of the first mode and the second mode is a flipping book mode, and the other is a video mode. The flipping book mode is a playback mode that includes flipping book animation and flipping book audio during the playback of media content. The video mode plays content images in the form of video frames, so that a single content image presents a certain dynamic effect. At the same time, the voice broadcast output by the audio output device of the display device corresponds to the currently displayed content image; and In response to a trigger instruction for the option of the second mode, the current media content continues to be played from the target playback position in the second mode through the display to display the target content picture at the target playback position, and the audio corresponding to the target content picture is output through the audio output device, and the duration for which the display displays the target content picture corresponds to the broadcast duration of the audio corresponding to the target content picture.

2. The display device according to claim 1, wherein The controller is further configured to: When the display plays the current media content in a flipping book mode, when the playback progress reaches a preset flipping book animation time node or in response to a user's flipping book instruction, the display displays a flipping book animation and outputs a flipping book audio through the audio output device.

3. The display device according to claim 2, wherein The controller is further configured to: In response to a start playback instruction for the current media content, reading a default playback mode of the target application, and playing the current media content in the default playback mode through the display; The default playback mode is the video mode or the book flipping mode.

4. The display device according to claim 3, wherein The controller is further configured to: After responding to a trigger instruction for the option of the second mode, if the second mode is different from the default play mode, the default play mode of the target application is updated according to the second mode.

5. The display device according to claim 2, wherein: The controller is further configured to: In response to the playback mode setting instruction, pausing playback of the current media content via the display, and displaying the content playback mode option in a pause playback interface; After receiving a trigger instruction from the user for any of the content play mode options, the current media content continues to be played through the display.

6. The display device according to claim 5, wherein: When the controller pauses playing the current media content via the display in response to the play mode setting instruction, the controller is further configured to: In response to a trigger instruction for the option of the second mode, obtaining content images and audio corresponding to the target playback position based on the saved playback progress; The current media content is continued to be played from the target playback position in a second mode through the display to display a target content picture of the current media content at the target playback position, and audio corresponding to the target content picture is output through the audio output device.

7. The display device according to claim 6, wherein: The current media content includes multiple pages of content images and multiple audios, and there is a first correspondence between a single page of content images and a playback time interval, and a second correspondence between a single page of content images and a single audio; When the controller, in response to a trigger instruction for the option of the second mode, acquires the content image and audio corresponding to the target playback position based on the saved playback progress, it is further configured to: Determining a target playback time interval corresponding to the playback progress; Based on the first corresponding relationship, obtaining a single-page content picture corresponding to the target playback time interval as a target content picture at the target playback position; as well as, Based on the second corresponding relationship, the audio corresponding to the target content picture is obtained as the audio at the target playback position.

8. The display device according to claim 6, wherein: The current media content includes multiple pages of content images and multiple audios, and there is a third correspondence between a single page of content images and multiple video frames, and there is a fourth correspondence between the video frames and the audios; When the controller, in response to a trigger instruction for the option of the second mode, acquires the content image and audio corresponding to the target playback position based on the saved playback progress, it is further configured to: When the first mode is a video mode and the second mode is a flipping book mode, determining a target video frame corresponding to the playback progress; Based on the third corresponding relationship, determining a content picture corresponding to the target video frame as the target content picture at the target playback position; and Based on the fourth corresponding relationship, the audio corresponding to the target video frame is determined as the audio at the target playback position.

9. The display device according to claim 2, wherein: Also includes: communication devices; The controller is further configured to: When the first mode is a video mode and the second mode is a flipping book mode, in response to a trigger instruction for selecting the second mode, obtaining an animation identifier and a first network address of a flipping book animation corresponding to the current media content, and an audio identifier and a second network address of a flipping book audio corresponding to the current media content; as well as The communication device is controlled to access the first network address to obtain the book-turning animation corresponding to the animation identifier and to the local display device, and to access the second network address to obtain the book-turning audio corresponding to the audio identifier and to the local display device.

10. The display device according to any one of claims 1 to 9, characterized in that: The controller is further configured to: After receiving a trigger instruction for any one of the content play mode options, hiding the content play mode option through the display.

11. A method for switching content playback mode, characterized in that: Applicable to a display device, the display device comprising a display, an audio output device and a controller; The method comprises: During playback of media content in a target application in a first mode, in response to a playback mode setting instruction, pausing playback of the current media content via a display, saving the playback progress of the current media content when playback was paused as a target playback position, and displaying content playback mode options in a current playback interface via the display, the content playback mode options including at least a first mode option and a second mode option, the first mode and the second mode corresponding to different content playback formats; and In response to a trigger instruction for the second mode option, the display continues to play the current media content from the target playback position in the second mode, displays the target content image at the target playback position, and outputs the audio corresponding to the target content image through the audio output device, and the duration for which the target content image is displayed on the display corresponds to the duration for which the audio corresponding to the target content image is played; Among them, the media content played is picture book content; one of the first mode and the second mode is a flipping book mode, and the other is a video mode. The flipping book mode is a playback mode that includes flipping book animation and flipping book audio during the playback of media content; the video mode plays content pictures in the form of video frames, so that a single content picture presents a certain dynamic effect. At the same time, the voice broadcast output by the audio output device of the display device corresponds to the currently displayed content picture.

Images