Display device and content playing mode switching method

Abstract

The invention relates to display equipment and a content playing mode switching method, and relates to the technical field of display equipment. The display device comprises a displayer, an audio output device and a controller, and the controller is configured to control the displayer to at least display an option of a first mode and an option of a second mode in a current playing interface in response to a playing mode setting instruction in the process that media content is played in the first mode in the display device; in response to a selection instruction of the user, controlling the display to continuously play the current media content in a second mode so as to display a content picture of the current media content at a target playing position, and controlling the audio output device to output audio at the target playing position; wherein the target playing position is the playing progress position of the current media content when the user triggers the option of the second mode. According to the scheme, multiple content playing modes can be provided for the user, habits of different watching users are considered, and meanwhile the content playing continuity is kept.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a content playback mode switching method. Background Art

[0002] Display devices are devices with image display and data processing capabilities, such as smart TVs. Smart TVs are TV products based on Internet application technology, with an open application platform, integrating multiple functions such as audio and video, entertainment, and data.

[0003] Display devices are usually used as video and audio playback devices in the past. In order to more fully develop the use value of display devices, more applications can be configured for display devices, such as configuring an application for viewing picture books in a smart TV.

[0004] The application program in the display device can play predefined picture book contents, but the playback mode is relatively fixed and cannot take into account the habits of different viewing users. Summary of the invention

[0005] Based on this, it is necessary to provide a display device and content playback mode switching method that can take into account the different viewing habits of users in order to address the above technical issues.

[0006] In a first aspect, the present application provides a display device, comprising:

[0007] A display, configured to display information, the information at least including a content image included in the media content played in the target application; the target application includes at least one media content;

[0008] an audio output device configured to output audio, the audio including at least audio of the media content played in the target application;

[0009] The controller is coupled to the display and the audio output device respectively, and is configured to:

[0010] During the playback of the current media content in the first mode, in response to the playback mode setting instruction, displaying content playback mode options in the current playback interface through the display, the content playback mode options at least including options of the first mode and options of the second mode, the first mode and the second mode corresponding to different content playback forms; and

[0011] In response to a trigger instruction for the option of the second mode, the current media content continues to be played from the target playback position in the second mode through the display, the target content picture at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device, and the duration of the target content picture displayed on the display corresponds to the broadcast duration of the audio corresponding to the target content picture; wherein the target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

[0012] Based on this embodiment, the display device can provide multiple content playback modes for viewing users, and users can choose to switch content playback modes while viewing picture books. The display device switches the currently playing media content to a new playback mode based on the user's playback mode selection operation, while keeping the current media content playback progress unchanged. In this way, the habits of different viewing users are taken into account while maintaining the continuity of content playback.

[0013] In one embodiment, one of the first mode and the second mode is a flipping book mode, and the flipping book mode is a playback mode including a flipping book animation and a flipping book audio during content playback;

[0014] The controller is also configured to:

[0015] When the display plays the current media content in book flipping mode, when the playback progress reaches a preset book flipping animation time node or in response to a user's book flipping instruction, the display displays a book flipping animation and outputs book flipping audio through an audio output device.

[0016] Based on this embodiment, the display device can play media content in a book flipping mode, and play book flipping animation and book flipping audio automatically or based on user instructions, which is conducive to enhancing the interactivity with the user during content playback and improving the content playback effect.

[0017] In one embodiment, the other of the first mode and the second mode is a video mode;

[0018] The controller is also configured to:

[0019] In response to a start play instruction for the current media content, the current media content is played through the display in a preset default play mode; the default play mode is a video mode or a book flipping mode.

[0020] Based on this embodiment, the target application in the display device can pre-set a default playback mode, and start playing the media content in the default playback mode. The user does not need to set the playback mode every time the media content is played, which simplifies the user's setting operation and improves the continuity of content playback.

[0021] In one embodiment, the controller is specifically configured to:

[0022] After responding to a trigger instruction for the option of the second mode, if the second mode is different from the default play mode, the default play mode of the target application is updated with the second mode.

[0023] Based on this embodiment, based on the mode switching operation of the user, the default play mode of the target application is dynamically updated, which can better meet the media content viewing habits of different users and reduce the operation of switching the play mode by the user.

[0024] In one embodiment, the controller is specifically configured to:

[0025] In response to the play mode setting instruction, pausing the current media content through the display, and displaying the content play mode options in the pause play interface;

[0026] After receiving a trigger instruction from the user for any of the content play mode options, the current media content continues to be played through the display.

[0027] Based on this embodiment, when the display device receives a playback mode setting instruction from the user, it can pause content playback to prevent the user from missing key content in the media content during the mode switching operation, which is beneficial to ensuring the user's experience in viewing media content.

[0028] In one embodiment, the controller is specifically configured to:

[0029] In response to the play mode setting instruction, when the current media content is paused through the display, the play progress of the current media content when the play is paused is saved as the target play position; and

[0030] In response to a trigger instruction for the option of the second mode, based on the saved playback progress, the content picture and audio corresponding to the target playback position are obtained, the current media content continues to be played from the target playback position in the second mode through the display, the target content picture of the current media content at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device.

[0031] Based on this embodiment, when the display device pauses the playback of media content, it saves the playback progress information of the media content in a timely manner, so that after the user selects a new playback mode, the playback can be continued through the new playback mode based on the saved content playback progress information. After switching, the user does not need to operate the playback progress, thereby reducing the operation process for the viewing user.

[0032] In one embodiment, the current media content includes multiple pages of content pictures and multiple audios, wherein there is a first corresponding relationship between a single page of content pictures and a playback time interval, and there is a second corresponding relationship between a single page of content pictures and a single audio;

[0033] The controller is also configured to:

[0034] Based on the first corresponding relationship, determining a playback time interval corresponding to the playback progress, and obtaining a single-page content picture corresponding to the playback time interval as a target content picture at a target playback position; and

[0035] Based on the second corresponding relationship, the audio corresponding to the target content picture is obtained as the audio at the target playback position.

[0036] Based on this embodiment, the display device can effectively determine which content picture and which audio segment to start playing again in a new playback mode based on the correspondence between a single-page content picture and the playback time, and the correspondence between a single-page content picture and the audio, without the need for manual selection by the user, thereby improving the continuity of content playback.

[0037] In one embodiment, the current media content includes multiple pages of content pictures and multiple audios, and there is a third corresponding relationship between the single page of content pictures and the multiple video frames, and there is a fourth corresponding relationship between the video frames and the audios;

[0038] When the controller, in response to a trigger instruction for the option of the second mode, acquires the content picture and audio corresponding to the target playback position based on the saved playback progress, the controller is further configured to:

[0039] When the first mode is the video mode and the second mode is the book flipping mode, determining a target video frame corresponding to the playback progress;

[0040] Based on the third corresponding relationship, determining a content picture corresponding to the target video frame as the target content picture at the target playback position; and

[0041] Based on the fourth corresponding relationship, the audio corresponding to the target video frame is determined as the audio at the target playback position.

[0042] Based on this embodiment, since the granularity of the corresponding relationship is smaller, a more accurate content image and audio at the target playback position can be obtained, further reducing the overlap of content playback.

[0043] In one embodiment, the display device further includes:

[0044] communication devices;

[0045] The controller is also configured to:

[0046] When the first mode is the video mode and the second mode is the book flipping mode, in response to a trigger instruction for the option of the second mode, obtaining an animation identifier and a first network address of a book flipping animation corresponding to the current media content, and obtaining an audio identifier and a second network address of a book flipping audio corresponding to the current media content; and

[0047] The communication device is controlled to access the first network address to obtain the book-turning animation corresponding to the animation identifier and to the local display device, and the communication device is controlled to access the second network address to obtain the book-turning audio corresponding to the audio identifier and to the local display device.

[0048] Based on this embodiment, the display device obtains the book-turning animation and the book-turning audio associated with the current media content locally through the animation identifier and network address of the book-turning animation and the audio identifier and network address of the book-turning audio, so as to play the current media content in book-turning mode.

[0049] In one embodiment, the controller is further configured to:

[0050] After receiving a trigger instruction from the user for any option in the content play mode options, the content play mode option is hidden through the display.

[0051] Based on this embodiment, after the user selects a play mode through the display device, the display content play mode option can be canceled from the display device to avoid affecting the user's viewing of media content.

[0052] In a second aspect, the present application provides a content playback mode switching method, which is applied to a display device, the display device including a display, an audio output device and a controller;

[0053] The method includes:

[0054] In the process of playing the media content in the target application in the first mode, in response to the play mode setting instruction, displaying content play mode options in the current play interface through the display, the content play mode options at least including options of the first mode and options of the second mode, one of the first mode and the second mode being a flipping book mode; and

[0055] In response to a trigger instruction for the option of the second mode, the current media content continues to be played from the target playback position in the second mode through the display, the target content picture at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device; and the duration for which the display displays the target content picture corresponds to the broadcast duration of the audio corresponding to the target content picture; wherein the target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

[0056] Based on this embodiment, when the target application is running in the display device, during the first mode playback of the current media content, multiple content playback modes can be provided to the viewing user, and the user can choose to switch the content playback mode during the viewing of the media content. The display device switches the currently playing media content to a new playback mode based on the user's playback mode selection operation, while keeping the current media content playback progress unchanged. In this way, the habits of different viewing users are taken into account, while maintaining the continuity of content playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] FIG1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0059] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0060] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0061] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0062] FIG5 is a schematic diagram showing picture book playback mode options in one embodiment;

[0063] Figure 6 A schematic diagram showing picture book playback mode options in another embodiment;

[0064] Figure 7 A schematic diagram showing picture book playback mode options in yet another embodiment;

[0065] Figure 8 A schematic diagram of an interface for playing a picture book in a flipping book mode in an embodiment;

[0066] Fig. 9 It is a flowchart of mode switching control during picture book playback in one embodiment;

[0067] Fig.10 is a timing diagram of mode switching control during picture book playback in one embodiment;

[0068] FIG11 is a flow chart of a method for switching content playback modes in one embodiment;

[0069] FIG. 12 is a structural block diagram of a device for switching content playback modes in one embodiment. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0071] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0072] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0073] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0074] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0075] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0076] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0077] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. As shown in Figure 1, a user can operate a display device 200 through touch operation, a mobile terminal 300, and a control device 100. For example, the control device 100 can be a remote controller, a stylus, a handle, etc.

[0078] The mobile terminal 300 can be used as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0079] 1 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks.

[0080] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function of a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0081] Figure 2 FIG. 1 is a block diagram of a hardware configuration of a display device 200 provided for some embodiments of the present application.

[0082] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0083] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0084] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0085] In some embodiments, a touch control component connected to the display 260 is further included, and the touch control component is used to receive a user's touch control operation. The display 260 is also used to display a display effect corresponding to the user's touch control operation.

[0086] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0087] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0088] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0089] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0090] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0091] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0092] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0093] Figure 3 The hardware configuration block diagram of the control device in FIG1 is provided for some embodiments of the present application. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0094] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0095] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0096] In some embodiments, as shown in FIG. 1 , the mobile terminal 300 or other intelligent electronic device can perform similar functions as the control device 100 after installing an application for operating the display device 200 .

[0097] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0098] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0099] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0100] In some embodiments, the buttons 144 configured by the control device 100 may include basic buttons, such as a power button, a number button, a direction button, a volume button, a home button, a back button, etc. The user can control the display device 200 to implement corresponding functions based on these basic buttons. The buttons 144 configured by the control device 100 may also include special buttons, such as a color button, etc. The user can control the display device 200 to implement specific functions in different scenarios based on these special buttons.

[0101] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0102] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0103] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0104] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may (control the display device) provide a user interface, allow the user to interact with the display device 200, and support the running of various application programs.

[0105] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0106] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications layer) (referred to as "application layer"), the application framework layer (Application Framework layer) (referred to as "framework layer"), the system library layer and the kernel layer.

[0107] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0108] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions of applications in the application layer. Applications can access system resources and obtain system services during execution through the API interface.

[0109] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager is used to interact with all activities running in the system; a location manager is used to provide system services or applications with access to system location services; a package manager is used to retrieve various information related to the application package currently installed on the device; a notification manager is used to control the display and clearing of notification messages; and a window manager is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0110] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0111] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0112] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200, and the kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0113] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0114] Usually, display devices are used as audio and video playback devices, such as watching online TV or Internet Protocol TV (IPTV), etc. The use value of display devices needs to be further developed. On the other hand, the creation of picture books previously required professional painting skills and literary literacy, which may be a challenge for children and parents.

[0115] In view of this, the embodiment of the present application proposes a display device based on which a target application is pre-set, and the user can create his own media content or watch the existing media content in the target application by simply interacting with the large TV screen. After the user enters the target application through the display device application homepage, the user moves the focus to select the media content he wants to watch from the media content list, and then the selected media content can be played on the display device based on voice commands or confirmation operations on the control device 100.

[0116] In an embodiment of the present application, the target application includes preset media content, and the pictures and audio contained in each media content are related to the content text. When playing the media content, the playing time of the pictures and content text therein corresponds to the broadcasting time of the audio content corresponding to the picture.

[0117] Usually, the display device plays media content in a fixed manner, for example, the media content is played in video mode by default, and during the media content video playback mode, the content picture is played in the form of video frames, so that a single content picture can present a certain dynamic effect. At the same time, the voice broadcast output by the audio output device of the display device also corresponds to the currently displayed content picture.

[0118] As a specific example, the target application may be a picture book application for creating and playing picture books. Correspondingly, the application may have the following features:

[0119] The picture book application has multiple picture books preset, and supports users to create new picture books. Each picture book includes multiple picture book pictures, and the character images in multiple picture book pictures are unified; each picture book picture is generated based on a story text in the picture book story, and each picture book picture corresponds to a broadcast audio, which is obtained based on the story text corresponding to the picture book picture.

[0120] Select a picture book in the picture book application and confirm the play. The selected picture book can be played in video mode. During the picture book video playback, the picture book picture can be displayed statically or with sliding effects. The story text corresponding to the page picture can also be displayed on the picture book picture in the form of subtitles, and the display duration of each picture book picture corresponds to the broadcast audio duration of the picture book picture. As an example, the switching display of two adjacent picture book pictures can be continuous, but the broadcast audio of two adjacent picture book pictures is played at a certain interval (for example, 1s). In addition, there can be background music during the picture book playback, and the background music matches the story content of the picture book.

[0121] However, taking the picture book application as an example, when playing picture books in a fixed way, it is difficult to take into account the needs of different viewing users in different scenarios. Therefore, optimizing the playback process of picture books and other content in the display device is an important factor in improving the user experience.

[0122] In the embodiment of the present application, a scene in which a picture book application in a display device plays a picture book is taken as an example for specific description. It can be understood that the picture book application in the following embodiment can also be other applications with similar characteristics.

[0123] In one embodiment of the present application, the display device includes:

[0124] The display is configured to display information, wherein the information at least includes content images contained in the media content played in the target application; the target application includes at least one media content; and the audio output device is configured to output audio, wherein the audio at least includes the audio of the media content played.

[0125] The display device further includes a controller, which is coupled to the display and the audio output device respectively and is configured as follows:

[0126] During the playback of the current media content in the first mode, in response to the playback mode setting instruction, controlling the display to display content playback mode options in the current playback interface, the content playback mode options including at least options of the first mode and options of the second mode;

[0127] In response to a trigger instruction for the second mode option, the current media content is played continuously from the target playback position in the second mode through the display, the target content picture at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device; wherein the duration of the display displaying the target content picture corresponds to the broadcast duration of the audio corresponding to the target content picture. wherein the target playback position corresponds to the playback progress of the current media content when the user triggers the second mode option.

[0128] The first mode and the second mode are different content playback modes, and different content playback modes correspond to different content playback forms and present different playback effects. Specifically, the content playback modes include video mode, book flipping mode, and the like.

[0129] In one embodiment, the effect of the display displaying the picture book play mode options in the current play interface may be as shown in FIG5. The options of the first mode and the options of the second mode are displayed on one side of the current play interface, such as on the right side in the example in the figure. It can be understood that the display method and display effect of the options of the first mode and the options of the second mode may also be in other forms, such as being displayed side by side at the bottom of the current play interface.

[0130] In one example, when the picture book playback mode options are displayed in the current playback interface, the options of the first mode currently used can be specially marked. Figure 6 As shown, the option of the first mode currently used is marked with a five-pointed star to let the user know the playback mode currently used by the picture book, so that the user can choose to switch to a new playback mode.

[0131] Among them, the triggering instruction of the content playback mode option can be broadly understood, including moving the focus to the position of a certain content playback mode option (i.e. selecting a certain mode option), and issuing a confirmation instruction based on the mode option where the current focus is located (i.e. triggering a certain content playback mode option).

[0132] In one embodiment, continuing to take the switching of the picture book play mode as an example, when a certain option in the picture book play mode option is selected, its display state can be different from the state when it is not selected. Figure 7As shown, the option of the second mode is in a selected state, and the display effect of this option is more prominent than the unselected option of the first mode.

[0133] In one embodiment, continuing to take the picture book play mode switching as an example, the play mode setting instruction / trigger instruction for the picture book play mode option can be issued based on the control device 100. For example, the user calls up the play setting menu from the picture book play interface by operating the setting button on the control device 100, enters the play mode setting submenu through the play setting menu, and continues to operate the corresponding button of the control device 100 to enter the play mode setting submenu, and then displays at least two picture book play mode options in the current play interface. Move the focus to one of the at least two picture book play mode options by operating the volume button on the control device 100, and click the confirmation case on the control device 100 to issue a trigger instruction for the picture book play mode option.

[0134] In another embodiment, continuing with the example of switching the picture book playback mode, the playback mode setting instruction / trigger instruction for the picture book playback mode option can be a voice instruction. For example, the display device is turned on and the voice service is turned on. By pressing the voice button on the remote control to call up the voice service, you can call up the playback settings menu from the picture book playback interface by saying "settings" or "play settings" to the remote control. Press the voice button on the remote control to call up the voice service, and say "switch playback mode" or "set playback mode" to the remote control to enter the submenu of playback mode settings. Continue to say "select the first mode" or "select the second mode" to the remote control to issue a trigger instruction for the picture book playback mode option.

[0135] In the embodiment of the present application, the display device 200 can obtain the audio spoken by the user through the sound collector, and perform voice recognition on the user audio to determine the matching degree between the voice recognition and the preset picture book playback control instruction. When the matching degree is greater than or equal to the preset matching degree threshold, the corresponding picture book playback control instruction is responded.

[0136] In another embodiment, continuing to take the picture book play mode switching as an example, the play mode setting instruction / trigger instruction for the picture book play mode option can be a touch instruction. For example, when the display of the display device includes a touch screen and the touch function of the touch screen is turned on, the user calls up the play setting menu from the picture book play interface by touching the touch screen, enters the play mode setting submenu by touching the touch screen, and displays at least two picture book play mode options in the current play interface. By touching one of the at least two picture book play mode options through the touch screen, a trigger instruction for the picture book play mode option can be issued.

[0137] In another embodiment, continuing to take the picture book play mode switching as an example, the play mode setting instruction / trigger instruction for the picture book play mode option can be a gesture instruction in the air. For example, when a detector is provided on the display device, such as a camera and the gesture operation function of the display device is turned on, the user can issue a play mode setting instruction and / or a trigger instruction for the picture book play mode option in the air through a pre-set gesture. Compared with the control device 100 and the touch screen operation, the gesture operation in the air is more convenient to operate.

[0138] The target playback position is the playback progress position of the current media content when the user triggers the second mode option, so the target playback position can be any position during content playback. Continuing with the example of switching the picture book playback mode, for example, if the current picture book contains 10 pages of picture book pictures, then the target playback position can be the playback position corresponding to the first or tenth page of the picture book picture, or it can be the playback position corresponding to a certain page of the picture book picture in the middle. There is no restriction on this.

[0139] Among them, when a picture book picture is played, there may be multiple picture book voice broadcasts corresponding to it, so the picture book audio at the target playback position may be one of the multiple picture book voice broadcasts of the current picture book picture. The current playback progress may be recorded in real time during the picture book playback process, and the current playback progress may include the current picture book picture identifier and the playback time progress. Based on the recorded current picture book picture identifier and the playback time progress, the picture book picture and picture book audio at the target playback position may be determined.

[0140] In other examples, when receiving the playback mode setting instruction, the current picture book playback can be paused and the current playback progress (including the current picture book image identifier and playback time progress) can be obtained. Based on the obtained current picture book image identifier and playback time progress, the picture book image and picture book audio at the target playback position can be determined. Compared with the real-time recording method, this method can further reduce the background resource overhead.

[0141] In some cases, the content playback mode takes a certain amount of time to switch. For the viewing user, the experience may be that after issuing the playback mode selection instruction, there is a waiting time before the current media content continues to play in the selected new playback mode. In this regard, in one embodiment, in response to the trigger instruction for the content playback mode option, before the current media content is played in the second mode through the display, a prompt window can also be displayed through the display to remind the viewing user that the content playback mode switching process is currently in progress. This improves the user experience during the content playback mode switching process and avoids the user mistakenly thinking that the current playback freeze triggers other operations.

[0142] The above-mentioned display device, when the target application in the display device is running and the current media content is being played, responds to the playback mode setting instruction and the display device can provide the viewing user with multiple content playback mode options, and the user can select any content playback mode option. Based on the user's trigger operation, the display device can switch the currently playing media content to a new playback mode for playback, while keeping the current media content playback progress unchanged. In this way, the habits of different viewing users and the continuity of content playback are taken into account.

[0143] In one embodiment, one of the first mode and the second mode is a flip book mode, which is a playback mode that includes a flip book animation and a flip book audio during content playback. The controller is further configured to:

[0144] When the display plays the current media content in book flipping mode, when the playback progress reaches a preset book flipping animation time node or in response to a user's book flipping instruction, the display displays a book flipping animation and outputs book flipping audio through an audio output device.

[0145] Take the above picture book application as an example:

[0146] When the picture book is played in flip book mode, a flip book animation is added to the display effect. The flip book animation uses the current frame and the next frame to create the page-turning animation effect; in the picture book audio, a corresponding page-turning audio broadcast is also added. The corresponding visual effect of playing the picture book in flip book mode is: the first frame picture book picture + flip book animation + the second frame picture book picture, and the audio effect is: the first frame broadcast audio + flip book sound + the second frame broadcast audio.

[0147] When the picture book is played in flip book mode, if the flip book animation is embedded in the picture book video in the form of video frames, it can be played in flip book mode through a video player. If the flip book animation is added in the form of an independent flash, it can also be played without a video player, for example, through a tool that can play pictures and flash animations in flip book mode.

[0148] Among them, the page turning control of the picture book in the flipping book mode can automatically realize the book turning animation based on the preset time node, that is, when the playback progress reaches the preset book turning animation time node, the book turning animation is played and the book turning sound is output; for example, when the picture book is played in the flipping book mode through a video player, the book turning animation can be automatically realized based on the preset time node. It is also possible to turn the page based on the user's book turning instruction, and the book turning animation is played and the book turning sound is output accordingly. For example, when the picture book is played in the flipping book mode with other tools, the book turning can be controlled by the user. In the case of turning the page based on the user's book turning instruction, the book turning instruction can be implemented by the control device 100, voice or air gesture.

[0149] It should be noted that in the embodiment of the present application, the book turning mode is also referred to as the page turning mode. Correspondingly, the description of the content of the book turning is sometimes also described as page turning.

[0150] In one embodiment, the visual effects corresponding to the picture book can be played in the flipping book mode. Figure 8 As shown, the book turning effect is controlled by the book turning angle and page turning length.

[0151] In this embodiment, when the display device plays a picture book in the flipping mode, it introduces book-flipping visual animation effects and book-flipping sound effects, and can also introduce more user page-flipping interactive operations, which can simulate the reading experience of traditional paper picture books and meet the playback needs of different users.

[0152] In one embodiment, the other of the first mode and the second mode is a video mode; the corresponding controller is further configured to:

[0153] In response to a start play instruction for the current media content, the current media content is played through the display in a preset default play mode; the default play mode is a video mode or a book flipping mode.

[0154] The default play mode may be one of the content play mode options in the above embodiment. For example, there are three content play modes for the display device, and the default play mode may be one of the three. The content play mode option provides options for the three modes. The default play mode may also be a mode other than the content play mode option in the above embodiment. For example, there are three content play modes for the display device, and the default play mode may be one of the three. The content play mode option provides options for the other two of the three.

[0155] In one example, the above-mentioned picture book application is used as an example to illustrate: the default play mode can be set based on the play setting function of the picture book application. For example, when the SDK of the picture book application is developed, the default play mode is set based on the play setting function, and is updated when the picture book application is updated. In another example, the default play mode can open a setting interface to the user. After entering the picture book application, the user can set the default play mode through the play setting function.

[0156] In this embodiment, after the display device enters the target application, when the user selects a certain media content in the media content list and instructs to play the picture book, the current media content will be played in the default play mode. The user does not need to select the content play mode every time he instructs to play the content, which is conducive to simplifying the user operation.

[0157] In an optional embodiment, during the content playback process of the display device, if the user switches from the default playback mode to another playback mode through the content playback mode option, the display device can also update the default playback mode of the target application with the other playback mode selected by the user. The next time the content is instructed to be played, the media content will be played in the other playback mode selected by the user, which can better match the user's picture book viewing habits.

[0158] In one embodiment, the controller is specifically configured to:

[0159] When the default playback mode is the video mode, in response to a start playback instruction, the video player provided by the display device is controlled to start, the media resources corresponding to the current media content are loaded into the video player, and the current media content is played through the display based on the video player.

[0160] Take the above picture book application as an example:

[0161] When the default playback mode is the flipping book mode, determine whether the picture book animation of the current picture book is compatible with the video player built into the display device. If compatible, control the video player built into the display device to start, load the media resources corresponding to the current picture book into the video player, and play the current picture book through the display based on the video player. If not compatible, do not control the video player built into the display device to start, and play the current picture book through the picture player built into the display device.

[0162] In order to achieve the mutual switching of playback modes and ensure the continuity of playback progress to the greatest extent. As an example, the target data format corresponding to each picture book can be pre-defined, and the target data format can specifically include the picture book identification, picture book storage address information, picture book title information and picture book picture list information. Among them, the picture book picture list information can specifically include the picture identification of each page of the picture book, the storage address of each page of the picture book, and the storage address of the audio file corresponding to each page of the picture book. If it is a flipping picture book, the picture book picture list information can also include the time node of the flipping animation, the identification of the flipping animation, the network address of the flipping animation, and the identification and network address of the flipping audio.

[0163] Correspondingly, loading the media resource corresponding to the current picture book into the video player may be loading information of the target data format of the current picture book into the video player.

[0164] In this embodiment, when the default play mode of the target application is the video mode, when the user instructs content play, the video player preinstalled in the display device is automatically started, and no other player needs to be installed, and the content play state can be quickly entered.

[0165] In one embodiment, the controller is specifically configured to:

[0166] During the playback of the current media content in the first mode, in response to a playback mode setting instruction, the playback of the current media content is paused through the display, and content playback mode options are displayed in the currently paused playback interface; after receiving a trigger instruction from the user for any option in the content playback mode options, the current media content continues to be played through the display.

[0167] It is understandable that in other embodiments, when the current media content is played in the first mode, in response to the play mode setting instruction, the current media content is not paused through the display, and the content play mode option is displayed in the play interface of the current media content.

[0168] Through this embodiment, when the display device receives a playback mode setting instruction from the user, it can pause content playback to prevent the user from missing key content during the mode switching operation, which is beneficial to ensuring the user's viewing experience.

[0169] In one embodiment, the controller is specifically configured to:

[0170] In response to the play mode setting instruction, when the current media content is paused through the display, saving the play progress of the current media content when the play is paused; and

[0171] In response to a trigger instruction for the second mode option, content images and audio corresponding to the target playback position are obtained based on the saved playback progress; then, the current media content is continued to be played from the target playback position in the second mode through the display, the target content image of the current media content at the target playback position is displayed, and the audio corresponding to the target content image is output through the audio output device.

[0172] Take the above picture book application as an example: the playback progress can be understood as the playback time of the current picture book. For example, the total duration of the current picture book is 10 minutes, and it has been played to the 3rd minute when it is paused. Then the saved playback progress is 3 minutes, and the target playback position is also the 3rd minute position.

[0173] As an example, in the target data format defined by the current picture book, the picture book picture list information contains the picture identifier, storage address, and storage address of the corresponding audio file for each page of the picture book. In addition, it can also include the picture book video frame information corresponding to each page of the picture book, that is, the correspondence between a single page of the picture book and the playback time interval. Based on the playback progress, it can be determined which picture book picture and which picture book audio to continue playing the current picture book in the new mode.

[0174] Based on this, during the playback mode switching process, the playback of the current media content is paused first to avoid missing content due to switching playback modes and affecting the user's viewing experience. By saving the current playback progress, it is convenient to continue playing from the position corresponding to the current playback progress after the user selects a new playback mode, ensuring the continuity of content playback.

[0175] In one embodiment, the current media content includes multiple pages of content images and multiple audios, wherein a single page of content images has a first corresponding relationship with a playback time interval, and a single page of content images has a second corresponding relationship with a single audio. When executing the saved playback progress and obtaining the content images and audios corresponding to the target playback position, the controller is further configured to:

[0176] Based on a first correspondence between a single-page content image of the current media content and a playback time interval, determining a playback time interval corresponding to the playback progress, and acquiring a single-page content image corresponding to the playback time interval as a target content image at a target playback position; and

[0177] Based on the second corresponding relationship between the single-page content picture of the current media content and the audio, the audio corresponding to the target content picture is obtained as the audio at the target playback position.

[0178] Take the above-mentioned picture book application as an example for explanation: Among them, the first correspondence between the single-page content picture and the playback time interval, for example, the first picture book picture in the picture book, corresponds to a playback time interval of 0~1 minute. Among them, the second correspondence between the single-page content picture and the audio can be understood as generating the text content of the single-page picture book picture, and obtaining a picture book audio through TTS conversion. Therefore, a single-page picture book picture corresponds to a picture book audio. The playback time of this picture book audio can correspond to the playback time of the single-page picture book picture. For example: For example, the first picture book picture in the picture book is generated based on the first section of the text content in the picture book story, and the picture book audio obtained by TTS conversion of the first section of the text content is the picture book audio corresponding to the first picture book picture, and the playback time period corresponding to the picture book audio is also 0~1 minute. It can be understood that the playback time period corresponding to the picture book audio can also be slightly shorter than the playback time period of the corresponding picture book picture, or slightly longer than the playback time period of the corresponding picture book picture.

[0179] In this embodiment, when the current media content is paused, the video image and audio corresponding to the single-page content image may have been partially played. At this time, the first corresponding relationship between the single-page content image and the playback time interval and the second corresponding relationship between the single-page content image and the audio determine the target content image and its corresponding audio. When the current media content is continued to be played in the second mode, the playback can be continued from the start position of the playback time period corresponding to the target content image.

[0180] In other words, in this case, the audio corresponding to the target content image will partially overlap before and after the playback mode is switched. However, since the playback time corresponding to a single-page content image is generally short, this overlap has little impact on user viewing.

[0181] Through this embodiment, the display device can effectively determine which content picture and which audio segment to start playing again in a new playback mode based on the correspondence between a single-page content picture and the playback time, and the correspondence between a single-page content picture and the audio, without the need for manual selection by the user, thereby reducing the inconvenience caused by mode switching to the user.

[0182] In another embodiment, the current media content includes multiple pages of content pictures and multiple audios, and there is a third corresponding relationship between the single page of content pictures and the multiple video frames, and there is a fourth corresponding relationship between the video frames and the audios;

[0183] When the controller, in response to a trigger instruction for the option of the second mode, acquires the content picture and audio corresponding to the target playback position based on the saved playback progress, the controller is further configured to:

[0184] When the first mode is the video mode and the second mode is the book flipping mode, determining a target video frame corresponding to the playback progress;

[0185] Based on the third corresponding relationship, determining a content picture corresponding to the target video frame as the target content picture at the target playback position; and

[0186] Based on the fourth corresponding relationship, the audio corresponding to the target video frame is determined as the audio at the target playback position.

[0187] Take the above picture book application as an example: there is a third correspondence between a single page content image and multiple video frames. It can be understood that for a single page picture book image, interpolation processing is performed when generating a video picture book, and multiple picture book video frames corresponding to the single page picture book image are obtained. The third correspondence reflects this correspondence.

[0188] Among them, there is a fourth corresponding relationship between the video frame and the audio, which can be understood as that when generating a video picture book, a section of picture book audio corresponding to a single-page picture book image is further segmented and marked, so that each picture book video frame corresponding to the single-page picture book image has a corresponding picture book audio.

[0189] Based on this embodiment, compared with the previous embodiment, since the granularity of the correspondence between a single-page content picture and video frame, and between a video frame and audio is smaller, when the display device responds to a trigger instruction for the option of the second mode and obtains the content picture and audio corresponding to the target playback position based on the saved playback progress, it can obtain more accurate content pictures and audio, and continue to play the determined content pictures and audio, which can further reduce the content playback overlap.

[0190] The following will be combined with the attached Fig. 9 and attached Fig.10 , taking a picture book application on a smart TV as an example, a specific example of a picture book playback mode switching process between a video mode and a flipping book mode is provided.

[0191] As an example, for each picture book in the picture book application, the corresponding target data format is predefined when generating the picture book as follows:

[0192] aibook{

[0193] videoURL : The URL address of the corresponding picture book

[0194] title picture book title

[0195] AibookPageInfo list[]

[0196] }

[0197] AibookPageInfo{

[0198] ImageUrl represents the image of each page in the picture book

[0199] AudioURL represents an audio file

[0200] / / Video time playback correspondence

[0201] Int timeStart The start time of the video playback segment

[0202] Int timeEnd The end time of the video playback segment

[0203] / / Picture book excerpt content

[0204] Sting content

[0205] }

[0206] Furthermore, based on the start time and end time of the video playback segment and the number of picture book images contained in the aibook, the playback time information corresponding to each page of the picture book image can be determined. The playback time information corresponding to each page of the picture book image can be pre-calculated and saved in the picture book image list information of the picture book.

[0207] When the cloud generates picture books, it also defines the correspondence between single-page picture book images and playback time intervals, and the correspondence between single-page picture book images and picture book audio. In addition, for flipping picture books, the corresponding flipping animations and flipping audio are also defined.

[0208] The smart TV downloads the picture book resource information generated in the cloud to the local computer, and can download the picture book according to the target data format of the picture book in the above embodiment, thereby obtaining the media resource information of the picture book.

[0209] Smart TV plays picture books according to the default playback mode configured in the picture book application.

[0210] If the picture book application is configured with video mode by default, the platform's own player (such as mediaPlayer) is started, and the media resource information of the picture book is loaded into the video player, thereby playing the picture book in video mode.

[0211] On the picture book playback interface of smart TVs, users can switch from video mode to book flipping mode by operating the video setting menu. Specifically:

[0212] (1) When the user chooses to switch from video mode to book flipping mode

[0213] First, determine whether the current playback mode is the flip book mode. If so, do not switch and continue to play the current picture book.

[0214] If not, the corresponding picture book picture is determined according to the target playback position of the current picture book through the corresponding relationship between the playback time and the single-page picture book picture. The smart TV displays the determined target picture book picture and simultaneously reads and broadcasts the picture book audio corresponding to the target picture book picture.

[0215] Before playing audio, you need to initialize the audio playback interface. For example, you can load the AudioTrack class on the Android platform to read and play the picture book audio corresponding to the target picture book image.

[0216] When playing the current picture book in flip book mode, when the predefined flip book animation node is reached, the flip book animation is displayed and the flip book audio is output.

[0217] (2) When the user chooses to switch from flipbook mode to video mode:

[0218] First, determine whether the current playback mode is video mode. If so, do not switch and continue to play the current picture book.

[0219] If not, for example, the current playback mode is the book flipping mode, then the corresponding video playback position in the video mode is located according to the information of the picture book in the current book flipping mode. The specific positioning method can refer to the above embodiment.

[0220] If the video player is not started in the flipping book mode, start the platform's built-in video player first, and jump to the video playback position determined in the previous step. The smart TV displays the picture book video frame corresponding to the video playback position and outputs the corresponding picture book audio.

[0221] In video mode, when the video frame corresponding to the next drawing picture is reached, the picture book audio broadcast corresponding to the next drawing picture is output.

[0222] As an example, the switching process between the video mode and the flipping book mode of the current picture book can be implemented according to the data correspondence in the picture book picture list of the current picture book, and the switching strategy is as follows:

[0223] When the display pauses playing the current picture book, obtain the current video playback status, such as obtaining and recording the current playback progress through the parameter CurrentPlayPosition;

[0224] According to the current playback progress, the picture book image list AibookPageInfo list is traversed in sequence to find the picture book image identifier AibookPageInfo corresponding to the playback progress. According to the obtained picture book image identifier aibookPageInfo, the corresponding page picture book image is obtained through the address information aibookPageInfo. ImageUrl corresponding to the picture book image identifier, and the audio address aibookPageInfo. AudioURL corresponding to the picture book image identifier is obtained to obtain the audio data corresponding to the current playback progress.

[0225] Thus, when the display device continues to play the current picture book in a new mode, it can jump to display the picture book image corresponding to the current playback progress and output the corresponding picture book audio.

[0226] In addition to the above implementation, the play time information corresponding to each page of the picture book in the picture book can also be calculated in real time. For example: when receiving the play mode switching instruction, calculate the play time information corresponding to each page of the picture book, then determine the play time interval corresponding to the current play progress, and then determine the picture book picture corresponding to the current play progress. Although this method is a real-time calculation, the calculation method is simple and can still meet the timeliness requirements of picture book playback.

[0227] In other embodiments, for the same picture book content, the server can pre-generate a video picture book and a flip book picture book. The difference lies mainly in the playback format, that is, the flip book picture book also includes flip book animation and flip book audio. Therefore, the video picture book can be generated first, and then the corresponding flip book picture book can be generated based on the video picture book.

[0228] As an example, for a picture book content, after generating a video picture book, the playing time of the picture book is obtained according to the starting time of the video playing segment of the picture book and the ending time of the video playing segment, and the playing time of a single page of the picture book is obtained based on the playing time and the number of pictures in the picture book. A flip book animation is added based on the playing time of a single page of the picture book. One case is to add the flip book animation while keeping the total playing time of the picture book unchanged, and the other case is to increase the time for displaying the flip book animation based on the total playing time of the video picture book. Optionally, the duration of the flip book animation can be determined based on the playing time of a single page of the picture book, for example, not exceeding the playing time of a single page of the picture book.

[0229] On this basis, continue to refer to Fig.10 , Fig.10 This is a timing diagram corresponding to the picture book playback mode switching process. The process involved in the playback mode switching process of this application includes:

[0230] 1. The picture book application homepage based on the smart TV loads the media resources corresponding to the picture book. The corresponding process is: 2. The homepage requests the cloud media resource library to obtain the picture book resources. 3. The cloud media resource library returns the corresponding picture book media resources to the picture book application homepage.

[0231] 4. The picture book application homepage determines the current default play mode. Correspondingly, 5. The homepage requests the play setting function module to obtain the setting information of the default play mode. 6. The play setting function module returns the setting information of the default play mode.

[0232] 7. The default playback mode is video mode, so the playback page is started. 8. The playback page obtains the playback position of the current video (if it starts playing, the playback position is the start position). 9. The platform video player plays the video based on the relevant information loaded on the playback page.

[0233] 10. The user issues a playback mode switching instruction based on the playback setting function module and chooses to switch to the page turning mode (i.e., the book turning mode mentioned above); correspondingly, 11. The playback page obtains information on switching to the book turning mode from the playback setting function module; 12. According to the current video playback position, the picture book video frame index (index is a serial number or index information) is obtained, and the target picture book image corresponding to the picture book video frame index is determined; 13. The corresponding picture book audio is obtained through the target picture book image; or the corresponding picture book audio is obtained through the picture book video frame index.

[0234] If the current video playback position is not the playback end position, continue with the subsequent process.

[0235] 14. The playback page stops playing the current video. This step can also be performed after 11 and before 12.

[0236] 15. The playback page directly displays the target picture book image; at the same time, 16. The playback page requests the picture book audio data from the PCM audio playback function module; PCM (Pulse Code Modulation) refers to the process of encoding and playing audio signals in digital form; 17. The PCM audio playback function module registers the playback status; before PCM plays audio, it is necessary to register the playback status, which usually involves setting the parameters of the audio stream, such as sampling rate, number of channels, format, etc. 18. The PCM audio playback function module returns the callback information of the picture book audio playback status, carrying the set parameter information.

[0237] 19. If the audio playback of the current picture book image is completed, get the picture of the next page of the picture book, and display the book turning animation and audio at the same time.

[0238] 20. If there is a picture of the next page of picture book, repeat steps 15 to 19.

[0239] 21. In another case, if the user selects to switch from the book flipping mode to the video mode based on the playback setting function module.

[0240] 22. The play page obtains the switching information from the play setting function module, and then obtains the play time information corresponding to the current picture book picture identifier according to the current picture book picture identifier, thereby obtaining the video play position corresponding to the current picture book picture identifier, thereby ensuring that the video jumps to the corresponding video when the video starts.

[0241] Based on the above example, on the one hand, the user can quickly enter the picture book playback state based on the smart TV, and on the other hand, the picture book data processing from the video mode to the book flipping mode can be quickly realized, thereby improving the efficiency of switching the playback mode.

[0242] In one embodiment, the controller is further configured to:

[0243] After receiving a trigger instruction from the user for the first mode option or the second mode option, the picture book play mode option is hidden through the display.

[0244] Through the method of this embodiment, after the user selects the play mode through the display device, the picture book play mode option can be cancelled from the display device to avoid affecting the user's viewing of the picture book.

[0245] In one embodiment, the controller is further configured to: in response to a trigger instruction for an option of the first mode, continue to play the current picture book in the first mode through the display.

[0246] As an example, the controller responds to the selection instruction of the picture book playback mode option, and when the current picture book playback mode is the first mode and the user selects the option of the first mode, the current picture book continues to be played in the first mode through the display; or,

[0247] In response to the selection instruction for the picture book playing mode option, the controller continues to play the current picture book in the second mode through the display when the current picture book playing mode is the second mode and the user selects the option of the second mode.

[0248] In one embodiment, in the process of generating the current picture book playback, if the current picture book has pre-set hidden elements or multiple story branches, the hidden elements may be character dialogues, item interactions, scene changes, etc. During the picture book playback process, the user may also perform other interactive operations based on the picture book playback page, and correspondingly, the picture book playback page transmits the user's interactive information to the server, that is, transmits to the server which interactive element the user has currently triggered.

[0249] The server generates corresponding feedback based on the user's interactive information, including the character's reaction, the progress of the story, the appearance of new interactive elements, etc., and returns the feedback information to the picture book playback page. The picture book playback page updates the picture book content and synchronously updates the currently playing picture book screen.

[0250] In an exemplary embodiment, as shown in FIG. 11 , a method for switching content playback modes is provided, which is described by taking the method applied to the display device 200 in FIG. 1 as an example, and includes the following steps S202 to S204 . Among them:

[0251] S202, in the process of playing the media content in the target application in the first mode, in response to the play mode setting instruction, displaying content play mode options in the current play interface through the display, the content play mode options at least including the first mode option and the second mode option, the first mode and the second mode corresponding to different content play forms; and

[0252] S204, in response to a trigger instruction for the option of the second mode, continue to play the current media content from the target playback position in the second mode through the display, display the target content picture at the target playback position, and output the audio corresponding to the target content picture through the audio output device;

[0253] The duration of the target content image displayed on the display corresponds to the duration of the audio corresponding to the target content image;

[0254] The target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

[0255] The above-mentioned content playback mode switching method, in the target application running state in the display device, during the current media content being played, in response to the playback mode setting instruction, controls the display of the display device to display the content playback mode options in the current playback interface, and the content playback mode options include at least the first mode option and the second mode option; in response to the user's triggering instruction for the content playback mode option, when the current media content playback mode is the first mode and the user selects the second mode option, the current media content is played in the second mode through the display, and jumps to the content picture showing the current media content at the target playback position, and controls the audio output device of the display device to output the audio at the target playback position; wherein the target playback position corresponds to the playback progress of the current media content when the user triggers the second mode option. Through the present application scheme, a variety of content playback modes can be provided to viewing users, and users can select any content playback mode during the viewing of media content. The display device switches the currently playing media content to a new playback mode based on the user's playback mode selection operation, while keeping the current media content playback progress unchanged. In this way, the habits of different viewing users are taken into account, while maintaining the continuity of content playback.

[0256] In one embodiment, one of the first mode and the second mode is a flipping book mode, and the flipping book mode is a playback mode including a flipping book animation and a flipping book audio during content playback;

[0257] The method also includes: when the display plays the current media content in a book flipping mode, when the playback progress reaches a preset book flipping animation time node or in response to a user's book flipping instruction, displaying a book flipping animation through the display and outputting book flipping audio through an audio output device.

[0258] In one embodiment, the other of the first mode and the second mode is a video mode;

[0259] The method further includes: in response to a start playing instruction for the current media content, playing the current media content in a preset default playing mode through the display; the default playing mode is a video mode or a book flipping mode.

[0260] In one embodiment, the method further includes: in response to the play mode setting instruction, pausing the play of the current media content through the display, and displaying the content play mode option in the pause play interface;

[0261] After receiving a trigger instruction from the user for any of the content play mode options, the current media content continues to be played through the display.

[0262] In one embodiment, the method further comprises:

[0263] In response to the play mode setting instruction, when the current media content is paused through the display, the play progress of the current media content when the play is paused is saved as the target play position; and

[0264] In response to a trigger instruction for the option of the second mode, based on the saved playback progress, the content picture and audio corresponding to the target playback position are obtained, and the current media content is continued to be played from the target playback position in the second mode through the display to display the target content picture of the current media content at the target playback position, and the audio corresponding to the target content picture is output through the audio output device.

[0265] In one embodiment, the method further includes: determining the playback time interval corresponding to the playback progress based on the first correspondence between the single-page content image of the current media content and the playback time interval, and acquiring the single-page content image corresponding to the playback time interval as the target content image at the target playback position; and

[0266] Based on the second corresponding relationship between the single-page content picture of the current media content and the audio, the audio corresponding to the target content picture is obtained as the audio at the target playback position.

[0267] In one embodiment, the current media content includes multiple pages of content images and multiple audios, and there is a third correspondence between a single page of content images and multiple video frames, and there is a fourth correspondence between video frames and audios. In response to a trigger instruction for the option of the second mode, when acquiring content images and audios corresponding to a target playback position based on the saved playback progress, the method further includes:

[0268] When the first mode is the video mode and the second mode is the book-turning mode, the target video frame corresponding to the playback progress is determined; based on the third correspondence, the content picture corresponding to the target video frame is determined as the target content picture at the target playback position; and based on the fourth correspondence, the audio corresponding to the target video frame is determined as the audio at the target playback position.

[0269] In one embodiment, the method further includes: when the first mode is the video mode and the second mode is the book flipping mode, in response to a trigger instruction for an option of the second mode, obtaining an animation identifier and a first network address of a book flipping animation corresponding to the current media content, and obtaining an audio identifier and a second network address of a book flipping audio corresponding to the current media content; and

[0270] The communication device is controlled to access the first network address to obtain the book-turning animation corresponding to the animation identifier and to the local display device, and the communication device is controlled to access the second network address to obtain the book-turning audio corresponding to the audio identifier and to the local display device.

[0271] In one embodiment, the method further includes: after receiving a trigger instruction from a user for any option in the content play mode options, hiding the content play mode options through a display.

[0272] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0273] Based on the same inventive concept, the embodiment of the present application also provides a content playback mode switching device. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned content playback mode switching method, so the specific limitations in one or more content playback mode switching device embodiments provided below can refer to the limitations of the content playback mode switching method above, and will not be repeated here.

[0274] In an exemplary embodiment, as shown in FIG. 12 , a content play mode switching device 1200 is provided, which is applied to a display device, and the display device includes a display, an audio output device, and a controller;

[0275] The switching device 1200 comprises:

[0276] a mode setting unit, configured to display content playback mode options in a current playback interface through a display in response to a playback mode setting instruction during playback of media content in a target application in a first mode, wherein the content playback mode options include at least options for a first mode and options for a second mode, wherein the first mode and the second mode correspond to different content playback forms; and

[0277] A switching control unit, configured to respond to a trigger instruction for the option of the second mode, continue to play the current media content from the target playback position in the second mode through the display, display the target content picture at the target playback position, and output the audio corresponding to the target content picture through the audio output device;

[0278] The duration of the target content image displayed on the display corresponds to the duration of the audio corresponding to the target content image;

[0279] The target application includes at least one media content; the target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

[0280] Each module in the above-mentioned content playback mode switching device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in the processor in the display device in the form of hardware, or can be stored in the memory in the display device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0281] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned content playback mode switching method embodiments are implemented.

[0282] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned content playback mode switching method embodiments when executed by a processor.

[0283] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0284] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0285] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0286] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: include: A display, configured to display information, wherein the information at least includes a content image included in the media content played in the target application; an audio output device, configured to output audio, the audio including at least audio of the media content being played; A controller is coupled to the display and the audio output device respectively, and is configured to: In the process of playing the current media content in the first mode, in response to the play mode setting instruction, displaying content play mode options in the current play interface through the display, the content play mode options at least including options of the first mode and options of the second mode, and the first mode and the second mode correspond to different content play forms; as well as, In response to a trigger instruction for the option of the second mode, the current media content is continuously played from the target playback position in the second mode through the display to display the target content picture at the target playback position, and the audio corresponding to the target content picture is output through the audio output device, and the duration of the display displaying the target content picture corresponds to the broadcasting duration of the audio corresponding to the target content picture; The target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

2. The display device according to claim 1, characterized in that One of the first mode and the second mode is a book flipping mode, and the book flipping mode is a playback mode including a book flipping animation and a book flipping audio during the playback of the media content; The controller is further configured to: In the case where the display plays the current media content in a book flipping mode, when the playback progress reaches a preset book flipping animation time node or in response to a user's book flipping instruction, a book flipping animation is displayed through the display, and book flipping audio is output through the audio output device.

3. The display device according to claim 2, characterized in that The other of the first mode and the second mode is a video mode; The controller is further configured to: In response to an instruction to start playing the current media content, reading a default play mode of the target application, and playing the current media content in the default play mode through the display; Wherein, the default playback mode is the video mode or the book flipping mode.

4. The display device according to claim 3, characterized in that The controller is further configured to: After responding to a trigger instruction for the option of the second mode, if the second mode is different from the default play mode, the default play mode of the target application is updated according to the second mode.

5. The display device according to claim 2, characterized in that The controller is further configured to: In response to the play mode setting instruction, pausing the play of the current media content through the display, and displaying the content play mode option in the pause play interface; After receiving a trigger instruction from the user for any of the content play mode options, the current media content continues to be played through the display.

6. The display device according to claim 5, characterized in that In response to the play mode setting instruction, the controller pauses playing the current media content through the display, and is further configured to: Saving the playback progress of the current media content when the playback is paused; In response to a trigger instruction for the option of the second mode, acquiring content pictures and audio corresponding to the target playback position based on the saved playback progress; The current media content is continuously played from the target playback position in a second mode through the display to display a target content picture of the current media content at the target playback position, and audio corresponding to the target content picture is output through the audio output device.

7. The display device according to claim 6, characterized in that The current media content includes multiple pages of content pictures and multiple audios, and there is a first corresponding relationship between a single page of content pictures and a playback time interval, and there is a second corresponding relationship between a single page of content pictures and a single audio; When the controller, in response to a trigger instruction for the option of the second mode, acquires the content picture and audio corresponding to the target playback position based on the saved playback progress, the controller is further configured to: Determining a target playback time interval corresponding to the playback progress; Based on the first corresponding relationship, obtaining a single-page content picture corresponding to the target playback time interval as a target content picture at the target playback position; as well as, Based on the second corresponding relationship, the audio corresponding to the target content picture is obtained as the audio at the target playback position.

8. The display device according to claim 6, characterized in that The current media content includes multiple pages of content pictures and multiple audios, and there is a third corresponding relationship between a single page of content pictures and multiple video frames, and there is a fourth corresponding relationship between the video frames and the audios; When the controller, in response to a trigger instruction for the option of the second mode, acquires the content picture and audio corresponding to the target playback position based on the saved playback progress, the controller is further configured to: When the first mode is a video mode and the second mode is a book flipping mode, determining a target video frame corresponding to the playback progress; Based on the third corresponding relationship, determining a content picture corresponding to the target video frame as a target content picture at the target playback position; and Based on the fourth corresponding relationship, the audio corresponding to the target video frame is determined as the audio at the target playback position.

9. The display device according to claim 2, characterized in that: Also includes: communication devices; The controller is further configured to: When the first mode is a video mode and the second mode is a book flipping mode, in response to a trigger instruction for an option of the second mode, acquiring an animation identifier and a first network address of a book flipping animation corresponding to the current media content, and an audio identifier and a second network address of a book flipping audio corresponding to the current media content; as well as The communication device is controlled to access the first network address to obtain the book-turning animation corresponding to the animation identifier to the local display device, and to access the second network address to obtain the book-turning audio corresponding to the audio identifier to the local display device.

10. The display device according to any one of claims 1 to 9, characterized in that: The controller is further configured to: After receiving a trigger instruction for any one of the content play mode options, hiding the content play mode option through the display.

11. A method for switching content playback mode, characterized in that: Applicable to a display device, the display device comprising a display, an audio output device and a controller; The method comprises: In the process of playing the media content in the target application in the first mode, in response to the play mode setting instruction, displaying content play mode options in the current play interface through the display, the content play mode options at least including options of the first mode and options of the second mode, the first mode and the second mode corresponding to different content play forms; and In response to a trigger instruction for the option of the second mode, the current media content is continued to be played from the target playback position in the second mode through the display, the target content picture at the target playback position is displayed, and the audio corresponding to the target content picture is output through the audio output device, and the duration of the display displaying the target content picture corresponds to the broadcasting duration of the audio corresponding to the target content picture; The target playback position corresponds to the playback progress of the current media content when the user triggers the option of the second mode.

Images