Display device and data processing method

Abstract

The invention provides a display device and a data processing method, the display device comprises a display, a user interface, a communication device, a memory, and at least one processor connected with the display, the user interface, the communication device and the memory, and the at least one processor is configured to be connected with the display, the user interface, the communication device and the memory; in response to the playing instruction, controlling the display to play the course media assets, and periodically generating first learning behavior information of the course media assets; if the first learning behavior information is the learning behavior information generated for the first time after the course media asset starts to be played, reporting the first learning behavior information to a server; and if the first learning behavior information is not the learning behavior information generated for the first time after the course media asset starts to be played, not reporting the first learning behavior information to a server. According to the invention, the learning reminding of the course media assets is optimized.

Description

Display device and data processing method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese patent application No. 2023109471752, filed on July 28, 2023; this application claims priority to Chinese patent application No. 202211706767.7, filed on December 29, 2022, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to the field of video playback, and in particular to a display device and a data processing method. Background Art

[0004] When children engage in online learning on a display device, such as watching educational videos, related technologies monitor online learning by reporting the child's online learning behavior to a server in real time. The server then pushes this online learning behavior to other devices, such as the child's parent's smartphone. Based on this online learning behavior, the child's parents can learn about the educational videos their child is watching and their viewing progress in real time. However, this push notification frequency is relatively high, which can be disruptive to the child's parents.

[0005] Summary of the Invention

[0006] An embodiment of the present disclosure provides a display device, comprising: a display configured to display images and / or a user interface; a user interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with the display device; at least one processor connected to the display, user interface, communication device and memory, and configured to execute computer instructions so that the display device performs the following: receiving an input playback instruction for a course media resource; in response to the playback instruction, controlling the display to play the course media resource and periodically generating first learning behavior information of the course media resource; if the first learning behavior information is the learning behavior information generated for the first time after the course media resource starts broadcasting, reporting the first learning behavior information to a server; if the first learning behavior information is not the learning behavior information generated for the first time after the course media resource starts broadcasting, not reporting the first learning behavior information to the server.

[0007] An embodiment of the present disclosure provides a data processing method, including: receiving an input playback instruction of a course media resource; in response to the playback instruction, playing the course media resource, and periodically generating first learning behavior information of the course media resource; if the first learning behavior information is the learning behavior information generated for the first time after the course media resource starts broadcasting, reporting the first learning behavior information to a server; if the first learning behavior information is not the learning behavior information generated for the first time after the course media resource starts broadcasting, not reporting the first learning behavior information to the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG1 is a schematic diagram showing an operation scenario between a display device and a control apparatus in some embodiments;

[0009] FIG2 shows a block diagram of the hardware configuration of the control device 100 in some embodiments;

[0010] FIG3 shows a hardware configuration block diagram of a display device 200 in some embodiments;

[0011] FIG4 shows a schematic diagram of software configuration in the display device 200 in some embodiments;

[0012] FIG5 shows a schematic diagram of a home page of a video-on-demand program in some embodiments;

[0013] FIG6 is a schematic flow chart showing a data processing method in some embodiments;

[0014] FIG7 shows a schematic diagram of a details page of a course media resource in some embodiments;

[0015] FIG8 shows a schematic diagram of a learning plan page in some embodiments;

[0016] FIG9 is a schematic flow chart showing a data processing method in some embodiments;

[0017] FIG10 is a schematic flow chart showing a data processing method in some embodiments;

[0018] FIG11 is a schematic diagram showing a timing diagram of interaction among a display device, a server, and a smart device in some embodiments;

[0019] FIG12 shows a schematic diagram of a learning management interface in some embodiments;

[0020] FIG13 shows a timing diagram of interaction between a user, a display device, and a server in some embodiments;

[0021] FIG14 is a schematic flow chart showing a video display method in some embodiments;

[0022] FIG15 is a schematic flow chart showing another video display method in some embodiments;

[0023] FIG16 is a schematic flow chart showing another video display method in some embodiments;

[0024] FIG17 shows an interface display diagram of a display device in some embodiments;

[0025] FIG18 shows another interface display diagram of a display device in some embodiments;

[0026] FIG19 shows another interface display diagram of a display device in some embodiments;

[0027] FIG20 shows another interface display diagram of a display device in some embodiments;

[0028] FIG21 shows another interface display diagram of a display device in some embodiments;

[0029] FIG22 is a schematic flow chart showing yet another video display method in some embodiments;

[0030] FIG23 is a schematic flow chart showing yet another video display method in some embodiments. DETAILED DESCRIPTION

[0031] The display device provided in the embodiments of the present disclosure can have various implementation forms, for example, it can be a television, a smart TV, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figures 1 and 2 illustrate a specific embodiment of the display device of the present disclosure.

[0032] FIG1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. As shown in FIG1 , a user can operate a display device 200 through a smart device 300 or a control device 100 .

[0033] In some embodiments, the control device 100 may be a remote controller. Communication between the remote controller and the display device may include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, and the display device 200 may be controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, and the like.

[0034] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) can also be used to control the display device 200. For example, the display device 200 can be controlled using an application running on the smart device.

[0035] In some embodiments, the display device may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0036] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0037] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be a single cluster or multiple clusters, and may include one or more types of servers.

[0038] Figure 2 illustrates a block diagram of the configuration of a control device 100 according to an exemplary embodiment. As shown in Figure 2 , the control device 100 includes a processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 receives user input commands and converts them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200.

[0039] As shown in FIG3 , the display device 200 includes at least one of a tuner and demodulator 210 , a communication device 220 , a detector 230 , an external device interface 240 , a processor 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user interface.

[0040] In some embodiments, the processor includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to nth interfaces for input / output.

[0041] The display 260 includes a display screen component for presenting images, a driving component for driving image display, a component for receiving image signals output from a processor, and a component for displaying video content, image content, and a menu control interface and a user control UI interface.

[0042] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0043] The communication device 220 is a component used to communicate with external devices or servers using various communication protocols. For example, the communication device may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. The display device 200 can use the communication device 220 to send and receive control signals and data signals with the external control device 100 or the server 400.

[0044] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0045] Detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or detector 230 includes an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or detector 230 includes a sound collector, such as a microphone, for receiving external sounds.

[0046] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0047] The tuner-demodulator 210 receives broadcast television signals via a wired or wireless reception method, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0048] In some embodiments, the processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the processor 250 is located, such as an external set-top box.

[0049] Processor 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. Processor 250 controls the overall operation of display device 200. For example, in response to receiving a user command to select a UI object to be displayed on display 260, processor 250 may perform operations related to the object selected by the user command.

[0050] In some embodiments, the processor includes a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), etc., and the display device may also include RAM (Random Access Memory, RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0051] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific voice or gesture, and the user input interface may recognize the voice or gesture through a sensor to receive the user input command.

[0052] Referring to Figure 4, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (referred to as the "application layer"), the application framework layer (referred to as the "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as the "system runtime library layer"), and the kernel layer.

[0053] In some embodiments, at least one application runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0054] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0055] As shown in Figure 4, the application framework layer in the embodiment of the present disclosure includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0056] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling the exit, opening, and backing of an application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling display window changes (such as shrinking the display window, shaking the display, distorting the display, etc.).

[0057] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0058] In some embodiments, the kernel layer is a layer between hardware and software. As shown in FIG4 , the kernel layer includes at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a Wi-Fi driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver.

[0059] In some embodiments, the application of the application layer includes a video-on-demand program, and the display device can be configured to enter the video-on-demand program after startup.

[0060] Referring to Figure 5 , the homepage of the video-on-demand program includes a navigation bar 510 and a content display area 520 located below the navigation bar 510. The navigation bar 510 is used to display multiple title controls, such as a VIP control, an education control, and a movie control. When the user moves the focus of the display device 200 between the multiple title controls, the content display area 520 displays the recommended content corresponding to the focused title control. When the focus is on a title control, the page displayed on the display device 200 is the homepage corresponding to that title control.

[0061] When the focus moves to the education control in the navigation bar 510, the current page can be called the education homepage. On the education homepage, multiple media resource recommendation controls 521 are displayed in the content display area 520. Each media resource recommendation control 521 corresponds to a course media resource. As shown in Figure 5, media resource 1, media resource 2, media resource 3, media resource 4, media resource 5 and media resource 6 are all different course media resources, and each course media resource corresponds to one or more educational videos.

[0062] In some embodiments, the smart device 300 is provided with an application corresponding to the video on demand program of the display device 200. For ease of description, the video on demand program on the display device 200 can be referred to as application A, and the corresponding application on the smart device 300 can be referred to as application B. Application A and application B can communicate through the server 400. In order to ensure communication between application A and application B, application A and application B need to log in to the same user account.

[0063] In some embodiments, when user one, such as child A, watches course media on the display device 200, in order to facilitate supervision of user one, application A can report the viewing behavior of the course media to server 400, and server 400 can push the viewing behavior to user two, such as application B on parent B's smart device 300, so that user two can understand user one's viewing behavior in real time.

[0064] In some embodiments, frequent reporting of viewing behavior by application A may disturb user 2. To solve this technical problem, the disclosed embodiment provides a data processing method suitable for reporting user behavior of children watching course media resources.

[0065] 6 is a flow chart of a data processing method according to some embodiments. As shown in FIG6 , the method may include the following steps:

[0066] Step S101: receiving an input instruction for playing course media resources.

[0067] In some embodiments, the user can trigger the media resource recommendation control 521 on the course media resource recommendation page, such as the page shown in FIG5 , to input a playback instruction for the course media resource.

[0068] Step S102: In response to the play instruction, control the display to play the course media resource and regularly generate first learning behavior information of the course media resource.

[0069] In some embodiments, after receiving the instruction to play the course media asset, the display device obtains the media asset identifier of the course media asset, generates a detail page request including the media asset identifier, and sends the detail page request to the server.

[0070] In some embodiments, after receiving the detail page request, the server obtains the detail page data of the media asset identifier and returns the detail page data to the display device.

[0071] In some embodiments, the details page data may include information such as the playback address of the course media resource, a media resource introduction, and the total duration of the video.

[0072] In some embodiments, after receiving the details page data, the display device obtains the video stream data of the course media resource according to the playback address in the details page data, generates a details page including a video playback window, and plays the video stream data in the video playback window. The user can click on the video playback window to adjust the video playback window to a full-screen playback window.

[0073] In some embodiments, after the display device starts broadcasting the course media resource, it can periodically generate first learning behavior information of the course media resource. For example, a first learning behavior information is generated every preset period, such as every 2 minutes. The first learning behavior information includes at least the following parameters of the course media resource: media resource ID and media resource name.

[0074] Step S103: if the first learning behavior information is the first learning behavior information generated after the course media asset starts broadcasting, reporting the first learning behavior information to the server;

[0075] Step S104: If the first learning behavior information is not the first learning behavior information generated after the course media asset starts broadcasting, the first learning behavior information is not reported to the server.

[0076] In some embodiments, in order to reduce the disturbance of learning behavior information to User 2, the display device may report the first learning behavior information generated for the first time after the course media resource starts to be broadcast to the server, and not report the first learning behavior information generated subsequently to the server, so that the server only sends a notification message corresponding to the first learning behavior information to the smart terminal of User 2 based on the first learning behavior information reported for the first time by the display device after the course media resource starts to be broadcast. During the playback of the course media resource, the notification message corresponding to the first learning behavior information will no longer be sent to the smart terminal of User 2, which causes less disturbance to User 2.

[0077] In some embodiments, the first learning behavior information may include a first reporting identifier, which is used to determine whether the first learning behavior information generated by the display device needs to be reported. When the display device generates the first learning behavior information for the first time after the course media asset starts broadcasting, the first learning behavior information can be set to include the first reporting identifier, and the first reporting identifier is the first identifier, indicating that the first learning behavior information is the first learning behavior information generated after the course media asset starts broadcasting. In the first learning behavior information generated subsequently, the first reporting identifier is the second identifier. Therefore, the display device can report the first learning behavior information to the server when the first reporting identifier in the first learning behavior information is the first identifier, and will not report the first learning behavior information to the server when the first reporting identifier in the first learning behavior information is the second identifier.

[0078] In some embodiments, the learning behavior information identified as the first identifier and reported for the first time may be referred to as start-up learning behavior information, and the learning behavior information identified as the second identifier and reported for the first time may be referred to as continuous learning behavior information.

[0079] In some embodiments, when user 1 is watching a course media resource and finds that he or she does not like the course media resource, the user will exit the playback and select a new course media resource to play. If user 1 frequently switches the course media resource to be watched in a short period of time, the display device will report the first learning behavior information of each course media resource to the server, so that the server will send notification messages corresponding to these first learning behavior information to the smart terminal of user 2, which will frequently disturb user 2. To solve this technical problem, the display device can be configured to report the first learning behavior information to the server only when the playback time of the course media resource is longer than a preset time, such as 2 minutes. Therefore, when user 1 frequently switches the course media resource to be watched, if the playback time of these course media resources is less than the preset time, the display device will not report the first learning behavior information of these course media resources to the server, and the server will not send notification messages corresponding to these first learning behavior information to the smart terminal of user 2, thereby avoiding frequent disturbances to user 2.

[0080] It can be seen from the above embodiment that after the course media resource starts broadcasting, although the display device periodically generates the first learning behavior information of the course media resource, it only reports the first learning behavior information generated for the first time after the course media resource starts broadcasting to the server, so that the server only sends a notification message corresponding to the first learning behavior information to the smart terminal of user two after the course media resource starts broadcasting. During the playback of the course media resource, the notification message corresponding to the first learning behavior information is not sent to the smart terminal of user two, thereby reducing the disturbance to user two. In order to further reduce the disturbance to user two, the display device reports the first learning behavior information to the server when the playback time of the course media resource is greater than the preset time, thereby realizing that the first learning behavior information is not reported to the server in the scenario where the user exits the playback within a short time after watching the course media resource, thereby further reducing the disturbance to user two by the server.

[0081] The above embodiment shows that by reducing the reporting of the first learning behavior information to the server on the display device side, the disturbance to the second user can be reduced. In other embodiments, the disturbance to the second user can also be reduced by reducing the sending of notification messages on the server side.

[0082] In some embodiments, for user one, course media are divided into two categories. One category is course media added to user one's learning plan list, for example, first-grade Chinese course media, first-grade mathematics course media, and first-grade English course media. This type of course media is usually course media that user one needs to focus on learning, and user two needs to pay special attention to user one's learning behavior. The other category is course media that is not added to the learning plan list, such as scientific exploration course media. This type of course media is usually course media that user one learns randomly, and user two needs to pay appropriate attention to user one's learning behavior.

[0083] For these two types of course media resources, in order to reduce the disturbance to user 2, the display device can adopt different user behavior reporting methods, and the server can also adopt different notification message sending methods.

[0084] Referring to Figure 7, in some embodiments, the details page of the course media resource may be provided with a video playback window 601, an episode control 602, and a join learning plan control 603. When child A clicks the join learning plan control 603, the display device may add the course media resource to the learning plan list of the user account currently logged in to the display device, and synchronize the learning plan list to the server.

[0085] In some embodiments, the join learning plan control 603 may also be set on a recommendation page of the course media resource, such as an education homepage, which is not specifically limited in this embodiment of the present disclosure.

[0086] In some embodiments, parent B can also add some course media resources to the learning plan list through application B on smart device 300. Application B can report the learning plan list to server 400, and application A and application B can obtain the learning plan list by interacting with server 400. Referring to Figure 8, a learning management page can be provided on application B. The learning management page can be provided with a learning plan control, a learning dynamics control, and a learning report control. The learning plan control is used to display the learning plan list when triggered, the learning dynamics control is used to display push messages from server 400 when triggered, such as messages that characterize child A's viewing behavior, and the learning report control is used to display learning reports sent by server 400 when triggered. For example, the learning report can be a summary of learning dynamics within a period of time, such as this week.

[0087] In some embodiments, a page of a video-on-demand program, such as an education homepage, may include a learning space control. This learning space control is configured to display a course learning management interface of application A when triggered. This course learning management interface may include a learning plan list control, which is configured to display a list of learning plans when triggered. Child A can quickly find a course media asset that has been added to the list in this list and then click on it to watch it.

[0088] In some embodiments, User 1 may request a course media asset that is not included in the learning plan list. This may be a random or accidental request, leading to the user exiting the video after a short viewing period and continuing on to the next course asset. If, after this user requests this type of course media asset, their learning behavior information is reported to the server in real time, and the server frequently pushes the corresponding learning behavior records to User 2, this could disrupt User 2. Furthermore, since course media assets not included in the learning plan list are typically not something User 2 needs to pay attention to, frequently pushing learning behavior records for these types of course media assets to User 2 could disrupt User 2.

[0089] In order to reduce the disturbance to user 2, the display device can be configured to report the learning behavior information to the server only once when the user orders this type of course media. In other scenarios, the first learning behavior information of this type of course media is not reported to the server. For course media that is not included in the learning plan list, the first learning behavior information includes the media asset ID, media asset name, and unplanned mark of the course media asset. Then, the server does not accumulate the viewing time of this type of course media based on the unplanned nature, and does not generate a learning behavior record containing a prompt for the completion of the course. Alternatively, the first learning behavior information includes the media asset ID and media name of the course media asset, but does not include the unplanned mark. Since the display device no longer continues to report after reporting the first learning behavior information once, the server does not accumulate the viewing time based on the fact that it has not received any subsequent reported first learning behavior information.

[0090] In order to further reduce the disturbance to user 2, the display device can also be configured to report the first learning behavior information of such course media resources to the server when the playback time of such course media resources exceeds the preset time, so that the learning behavior information of such course media resources will not be reported to the server in the scenario where the user watches for a short time and then exits.

[0091] After receiving the first learning behavior information containing the unplanned identification, the server generates a corresponding notification message and sends the notification message to the smart terminal of user 2. The notification message may be a learning behavior record representing the user's learning of unplanned course media resources, and may include the course name, the location where the course starts, and the time when the course starts. The time may be the time when the learning behavior information is received.

[0092] In some embodiments, if a user requests a course media resource that has been added to the learning plan list, the display device may execute the data processing method shown in FIG9 , which may include the following steps:

[0093] Step S201: receiving an input instruction for playing course media resources.

[0094] In some embodiments, a user may input a playback instruction for a course media resource by clicking on the course media resource in the learning plan list.

[0095] In some embodiments, user 1 may also directly click on a course resource that has been added to the learning plan list on the education homepage.

[0096] Step S202: In response to the playback instruction, control the display to play the course media resource. When the media resource data of the course media resource includes an in-plan identifier representing that the course has been added to the learning plan and an incomplete status identifier representing that the plan has not been completed, periodically generate second learning behavior information of the course media resource, and the second learning behavior information includes the in-plan identifier and the incomplete status identifier.

[0097] In some embodiments, after receiving the instruction to play the course media asset, the display device obtains the media asset identifier of the course media asset, generates a detail page request including the media asset identifier, and sends the detail page request to the server.

[0098] In some embodiments, after receiving the detail page request, the server obtains the detail page data of the media asset identifier and returns the detail page data to the display device.

[0099] In some embodiments, the details page data may include media asset data such as the playback address, media asset description, and total video duration of the course media asset. If the course media asset is already included in the learning plan list, the details page data may also include an "in-plan" indicator and a learning status indicator, indicating that the course media asset has been added to the learning plan list. The display device may determine that the course media asset has been added to User 1's learning plan list based on the "in-plan" indicator included in the details page data.

[0100] The learning status identifier is used to determine whether User 1 has completed the learning plan for the course media resource. The learning status identifier may include an unfinished status identifier and a completed status identifier. The unfinished status identifier indicates that User 1 has not completed the learning plan for the course media resource, and the completed status identifier indicates that User 1 has completed the learning plan for the course media resource. For example, completing the learning plan for the course media resource may mean that User 1 has finished watching the course media resource, or the viewing progress has reached a threshold progress, such as 95%, or the viewing time is greater than or equal to the total video length of the course media resource, or the viewing time has reached a preset ratio of the total video length, such as 95%.

[0101] In some embodiments, after receiving the details page data, the display device obtains the video stream data of the course media resource according to the playback address in the details page data, generates a details page including a video playback window, and plays the video stream data in the video playback window. The user can click on the video playback window to adjust the video playback window to a full-screen playback window.

[0102] In some embodiments, the display device regularly generates second learning behavior information of the course media asset based on the details page data including a planned identification and a learning status identification, and the learning status identification is an unfinished identification. The second learning behavior information includes the following parameters of the course media asset: media asset ID, media asset name, planned identification and unfinished status identification.

[0103] In some embodiments, the second learning behavior information may include a first-reporting flag, which is used to enable the server to determine whether the second learning behavior information is the first learning behavior information reported by the display device after the course media asset starts broadcasting. Alternatively, the second learning behavior information may not include a first-reporting flag, and the server determines whether the second learning behavior information is the first learning behavior information reported by the display device after the course media asset starts broadcasting based on the time the second learning behavior information is received. Because the display device may encounter network failures or other reasons when reporting the second learning behavior information, resulting in the first generated second learning behavior information not being reported to the server in a timely manner, the server, after receiving multiple pieces of second learning behavior information, may determine the second learning behavior information generated earliest by the display device based on the first-reporting flag. Step S203: Periodically report the second learning behavior information to the server, so that the server, based on the planned flag and the unfinished status flag, pushes a notification message corresponding to the second learning behavior information to the terminal only when the display device first reports the second learning behavior information and the viewing time of the course media asset reaches a preset duration.

[0104] In some embodiments, the display device includes a planned identifier and a learning status identifier based on the details page data. After periodically generating the second learning behavior information, the display device reports each second learning behavior information to the server, so that the server can perform statistics on the viewing progress and the viewing time, and then generate a viewing record for the media asset.

[0105] Among them, the planned identifier and unfinished status identifier in the second learning behavior information are used to enable the server to generate a corresponding notification message based on the second learning behavior information reported for the first time by the display device, and send the notification message to the smart terminal of user 2. The notification message may be a learning behavior record representing the start of the learning plan for the course media resource. The learning behavior record may include the course name, the location where the course starts, and the time when the course starts. The time may be the time when the learning behavior information is received.

[0106] The planned identifier and unfinished status identifier in the second learning behavior information also enable the server to count the viewing time of the course media asset. The statistical method may include obtaining the viewing time of the course media asset based on the product of the number of times the second learning behavior information is received and the sending period of the second learning behavior information. The sending period can be pre-stored in the server, or the display device can also set it in the second learning behavior information.

[0107] In some embodiments, because the course media material has an ending credit, User 1 typically does not watch to the end of the course media material. The server can be configured to determine that User 1 has completed viewing the course media material when the viewing time of the course media material reaches a preset ratio to the total video length, such as 95% of the total video length, and update the learning status information of the course media material, generate a learning behavior record for the course media material, and push the learning behavior record to User 2's terminal.

[0108] The server can determine that the user has completed watching the course media material based on the ratio of the viewing time of the course media material to the total video length of the course media material. It can update the unfinished mark to the completed mark in the media data of the course media material, so that the next time the user watches the course media material, the display device will receive the details page data including the completed mark. The server also generates a corresponding notification message. The notification message can be a learning behavior record that represents the completion of the learning plan for the course media material. The learning behavior record can include the course name, the location where the course was completed, and the time when the course was completed. The time can be the time when the learning behavior information is received so that the viewing time and the total video length reach a preset ratio, such as 95%. The notification message is sent to the smart terminal of user 2.

[0109] Before the viewing duration of the course media resource reaches a preset ratio of the total video duration of the course media resource, the server does not generate a corresponding notification message.

[0110] After the viewing time of the course media reaches a preset ratio of the total video length of the course media, such as between 96%-100% of the total video length, the server generates a corresponding notification message and no longer continues to generate learning behavior records representing the completion of the learning plan for the course media, so as to avoid generating multiple duplicate learning behavior records.

[0111] In some embodiments, if the user exits playback, the display device generates an exit message that includes the learning plan identifier and the media asset ID. The display device sends this exit message to the server, which, upon receiving it, stops accumulating viewing time for the course media asset.

[0112] In some embodiments, for a course media asset with a learning status flagged as "incomplete plan," if a user exits playback and then re-orders the course media asset, the display device generates a second learning behavior information piece for the course media asset after the course media asset begins playing. In this learning behavior information, the learning plan flag is the "in-plan flag" and the first reporting flag is the "first flag." The display device then reports the second learning behavior information for the course media asset to the server.

[0113] After receiving the second learning behavior information reported by the display device, the server generates a learning behavior record of the course media resource, pushes the learning behavior record to the terminal of user 2, and re-accumulates the viewing time based on the learning behavior information after the learning behavior information.

[0114] It can be seen that when the second learning behavior information reported by the display device includes the planned mark and the unfinished mark, the server can not only notify user 2 of the start of the course media resource, but also notify user 2 again only when the learning status of the course media resource is updated, which will cause less disturbance to user 2.

[0115] In some embodiments, the server does not count the learning status of course media resources that are not included in the learning plan. Therefore, the second learning behavior information may not need to include an in-plan identifier. The server can determine that the course media resource is a course media resource that has been added to the learning plan list based on the incomplete identifier, and its learning status needs to be monitored.

[0116] It should be noted that when a user is watching a course media resource, they may manually change the playback progress by moving forward or backward. The viewed time in the embodiment of the present disclosure is not equivalent to the playback progress, but rather the time from the start time, which can more accurately reflect the learning process of the user in learning the course media resource. For example, if the user drags the playback progress to the end after watching for 5 minutes, if the completion of the learning plan is determined based on the playback progress, the result will be that the learning plan is completed, which is less accurate. However, the embodiment of the present disclosure determines the completion of the learning plan based on the total viewing, which will result in an incomplete learning plan, which is more accurate. In some embodiments, after receiving the learning behavior information of the course media resource, the server can obtain the viewed time of the course media resource based on the cumulative amount of learning behavior information received for the course media resource. Since the display device reports at preset intervals, there is a certain error between the viewed time of the course media resource counted by the server and the actual viewed time of the course on the display device. This error has a small impact on the embodiment of the present disclosure and can be ignored.

[0117] It can be seen that although the display device reports the second learning behavior information regularly, the server can send a notification message to the smart terminal of user two only when the second learning behavior information is received for the first time and when it is determined that the viewing time of the course media resource reaches a preset ratio of the total video time of the course media resource, based on the planned flag and unfinished status flag in the second learning behavior information, instead of sending a notification message to the smart terminal of user two based on each piece of second learning behavior information, thereby reducing the disturbance to user two.

[0118] In some embodiments, to further reduce disturbance to user 2, the display device may be configured to report the second learning behavior information to the server for the first time when the playback duration of the course media resource added to the learning plan list exceeds a preset duration.

[0119] In some embodiments, the second learning behavior information may also include a first reporting identifier. The display device may set the first reporting identifier as the first identifier in the second learning behavior information generated for the first time after the course media resources start broadcasting, and set the first reporting identifier as the second identifier in the second learning behavior information generated thereafter, so that the server can determine whether the received second learning behavior information is the learning behavior information reported for the first time after the course media resources start broadcasting based on the first reporting identifier.

[0120] In some embodiments, for some course media materials that have been added to the learning plan list, in order to improve learning effects, users may continue to watch them multiple times after finishing watching. Therefore, after the user clicks on a course media material on demand, the learning status indicator obtained by the display device from the details page data may be completed.

[0121] In some embodiments, for course media materials whose learning status is marked as completed, the time for user one to re-learn may be longer or shorter. In order to avoid causing more disturbance to user two, the display device can be configured to generate and report the second learning behavior information of the course media material after the playback time of the course media material exceeds the preset time.

[0122] In some embodiments, for completed course media materials, when the server receives the second learning behavior information of the course media materials, it can determine that user 1 has re-watched the completed course media materials, and can update the learning status flag of the course media materials to incomplete, and generate a notification message corresponding to the second learning behavior information. The notification message can be a learning behavior record representing user 1's re-learning of the course media materials, and the learning behavior record is pushed to the terminal of user 2.

[0123] In some embodiments, for completed and re-watched course media, the display device can be configured to, after sending the second learning behavior information to the server once after the course media begins playing, no longer periodically report the second learning behavior record for the course media to the server. Furthermore, the server can be configured to no longer accumulate the viewing time for the course media and no longer push the learning behavior record for the course media containing the course completion location prompt to user 2's smart terminal.

[0124] It can be seen from the above embodiments that for the course media resources added to the learning plan list, by adding a planned mark and an unfinished status mark in the user behavior information sent by the display device, even if the display device uploads the learning behavior information of the course media resources multiple times, the server will only notify User 2 in two scenarios: when the course media resources start to be broadcast and when the user finishes watching, and will not frequently disturb User 2.

[0125] To compare and illustrate the course media resources in the unfinished state and the completed state, FIG10 shows a data processing method, which may include the following steps:

[0126] Step 301: Receive input instructions for playing course media resources.

[0127] Step 302: In response to the play instruction, control the display to play the course media resource and regularly generate third learning behavior information of the course media resource.

[0128] In some embodiments, after receiving the instruction to play the course media asset, the display device obtains the media asset identifier of the course media asset, generates a detail page request including the media asset identifier, and sends the detail page request to the server.

[0129] In some embodiments, after receiving the detail page request, the server obtains the detail page data of the media asset identifier and returns the detail page data to the display device.

[0130] In some embodiments, the details page data may include media resource data such as the playback address of the course media resource, media resource introduction, total video duration, etc.

[0131] If the course media asset is a course media asset that has been added to the learning plan list, the details page data also includes a plan identifier and a learning status identifier indicating that it has been added to the learning plan list. The learning status identifier is an unfinished status identifier or a completed status identifier.

[0132] In some embodiments, after starting to play a course media asset, the display device may periodically generate third learning behavior information for the course media asset. For example, the third learning behavior information may be generated at a preset interval, such as every two minutes. The third learning behavior information includes the following parameters of the course media asset: the asset ID and the asset name. If the learning status of the course media asset is an incomplete status indicator, the third learning behavior information also includes the incomplete status indicator. If the learning status of the course media asset is a completed status indicator, the third learning behavior information also includes the completed status indicator.

[0133] It can be understood that the present disclosure divides the learning behavior information generated by the display device into first learning behavior information, second learning behavior information, and third learning behavior information for introduction. In fact, for course media assets that have been added to the learning plan list and whose learning status is incomplete, the third learning behavior information generated by the server may be the same as the second learning behavior information described in Figure 9; and for course media assets that have been added to the learning plan list and whose learning status is completed, the third learning behavior information generated by the server may be the same as the first learning behavior information described in Figure 6. In the embodiment of the present disclosure, the learning behavior information generated by the display device for the course media assets that have been added to the learning plan list is collectively referred to as the third learning behavior information, only for the convenience of describing the user behavior reporting method for the course media assets that have been added to the learning plan list.

[0134] Step 303: If the media asset data of the course media asset contains an unfinished status identifier representing an unfinished plan, the third learning behavior information is reported to the server regularly, so that the server pushes a notification message corresponding to the third learning behavior information to the terminal based on the unfinished status identifier only when the display device reports the third learning behavior information for the first time and the viewing time of the course media asset reaches a preset time.

[0135] In some embodiments, when the media asset data of the course media asset includes an unfinished status indicator, the display device is configured to periodically report the third learning behavior information to the server. Since the third learning behavior information includes an unfinished status indicator, the server only pushes a notification message corresponding to the third learning behavior information to the terminal when the display device reports the third learning behavior information for the first time and the viewing time of the course media asset reaches a preset time. Step 304: If the media asset data of the course media asset includes a completed status indicator representing a completed plan, if the third learning behavior information is the first learning behavior information generated after the course media asset starts broadcasting, the third learning behavior information is reported to the server. If the third learning behavior information is not the first learning behavior information generated after the course media asset starts broadcasting, the third learning behavior information is not reported to the server.

[0136] In some embodiments, when the media asset data of the course media asset includes a completed status indicator, the display device is configured to report the third learning behavior information to the server only once after the course media asset starts broadcasting, and not report the subsequently generated third learning behavior information to the server, so that the server only notifies user two after the course media asset starts broadcasting, and does not notify user two of other notifications corresponding to the course media asset, so as to reduce disturbing user two.

[0137] It can be seen from the above embodiments that for the course media resources that have been added to the learning plan list, the reporting frequency of the display device is different for different learning statuses. Although the reporting frequency of the display device is different, it can achieve the technical effect of reducing the frequency of the server sending notification messages to the smart terminal of user two, thereby reducing the disturbance to user two.

[0138] In some embodiments, if the course media resource is a course media resource that is not included in the learning plan list, the details page data does not include the first identifier, and the display device can determine that the course media resource is not included in the learning plan list based on the fact that the details page data does not include the first identifier.

[0139] To further describe the reporting method of the above-mentioned viewing behavior, Figure 11 shows a timing diagram of the interaction between a display device A1, a server A10 and a smart device A100. As shown in Figure 11, the server A10 is provided with an education system A11, a behavior recording system A12 and a message system A13.

[0140] Among them, the education system A11 is used to store page data of the education homepage and its secondary pages, tertiary pages and other subordinate pages. The page data includes the data of the course media resources displayed on the page, such as the playback address of the course media resources, media resource identification, learning plan identification, learning status identification and other data.

[0141] The behavior recording system A12 is used to store user behavior information, such as viewing behavior information, collection behavior information, etc.

[0142] The message system A13 is used to push messages to the display device A1 and the smart device A100. The specific implementation can be shown in Figure 11 and includes the following steps:

[0143] Step S111: Display device A1 sends a detail page request to education system A11;

[0144] In some embodiments, after receiving a playback instruction for a user's course media asset, display device A1 sends a details page request containing the user's account to education system A11, requesting the course media asset's details page data. The course media asset's details page is a secondary page within the education homepage. Education system A11 returns the course media asset's details page data to display device A1.

[0145] Step S112: The education system A11 returns the details page data to the display device A1;

[0146] In some embodiments, the details page data stored in the education system A11 does not include a learning plan identifier and a learning status identifier. After receiving the details page request, the education system A11 obtains the learning plan identifier and learning status identifier of the user account from the behavior recording system A12, adds the obtained learning plan identifier and learning status identifier to the details page data, and then returns the added details page data to the display device A1.

[0147] Step S113: Display device A11 starts playing the video;

[0148] In some embodiments, after receiving the detail page data, the display device A1 displays the detail page of the course media asset according to the detail page data, and plays the course media asset in the playback window of the detail page.

[0149] Step S114: After determining that the learning behavior information meets the reporting conditions, the display device A11 reports the learning behavior information to the behavior recording system A12;

[0150] Step S115: The behavior recording system A12 stores the learning behavior information and generates a notification message;

[0151] Step S116: the behavior recording system A12 distributes a notification message to the message system A13;

[0152] Step S117: The message system A13 searches for the target device logged in to the user account according to the user account in the notification message;

[0153] The target device in this step is the smart device A100 of the aforementioned user 2;

[0154] Step S118: The message system A13 distributes the notification message to the smart device A100;

[0155] Step S119: the smart device A100 displays the notification message;

[0156] In some embodiments, after receiving the notification message, User 2's smart device displays the notification message. Referring to Figure 12, which is a schematic diagram of a learning management interface, Application B on the smart terminal can receive the notification message and display the learning management interface. The learning management interface can include a learning dynamic control and a content display area corresponding to the learning dynamic control. The content display area can display multiple notification messages corresponding to User 1's learning behavior information in chronological order.

[0157] It can be seen from the above embodiments that, in the embodiments of the present disclosure, when the course media resources requested by the user have been added to the learning plan list, the first learning behavior information is reported to the server based on the learning status information, so that the server pushes a notification message to the target device when the learning status information is updated, thereby reducing the frequency of sending notification messages; when the course media resources requested by the user have not been added to the learning plan list, the second learning behavior information of the course media resources is reported to the server after the preset time of the course media resource start broadcasting, so that the learning behavior information is not reported to the server when the course media resource start broadcasting is less than the preset time, thereby reducing the frequency of sending notification messages. The present disclosure realizes categorized reporting based on the learning plan list and learning status information, effectively reducing the frequency of the server sending notification messages to the target device, and reducing the degree of disturbance to the user.

[0158] In video-on-demand tasks, users may be interested in a specific segment within a long video, such as a dramatic fight scene in a TV series or the explanation of a specific knowledge point in an educational video. To meet users' needs for watching the segments of a long video that interest them, the system currently allows users to select the start time to play the segment of interest. This prevents or improves the situation where users need to watch other segments of the long video while watching. This enriches the presentation of TV videos, allows users to focus more on the content they are interested in, and improves the user experience.

[0159] However, a complete video is composed of multiple ts slice files, and in the process of encoding a complete video, in order to effectively compress the data, a ts slice file is usually divided into I frames, P frames, and B frames. The I frame is a key frame, which contains the information of the P frame and B frame and is the starting point of video decoding. If you need to play the content of the P frame and B frame, you must first perform decoding operations based on the I frame before you can start playing. That is, the content of the P frame and B frame must be played from the I frame. In this case, if the start time is a P frame or a B frame, it will start playing from the I frame before the start time, and the starting content will be inconsistent with the subsequent content, which will give users a bad viewing experience.

[0160] On this basis, in order to improve the viewing experience of course media resources, the present disclosure also provides a video display method. The application scenario of the video display method is described below: As shown in Figure 13, Figure 13 is a complete interaction diagram between the user 100, display device 200 and server 300 provided in an embodiment of the present disclosure.

[0161] Step S501: After receiving the original video uploaded by the partner, the server 300 processes and stores the original video;

[0162] In some embodiments, the server 300 divides the original video into multiple video segments, tags the divided multiple video segments, and marks the start and end time of each video segment. A target information table is generated based on the tag, start and end time of each video segment, and stored in a memory of the server 300.

[0163] Step S502: The user 100 requests the display device 200 to watch a course media segment;

[0164] In some embodiments, the user 100 opens the display device 200 or a video playback application on the display device 200, and requests the display device 200 to view a course media segment by inputting the name of the course media segment to be viewed or a keyword contained in the content of the course media segment in a search bar on the display screen of the display device 200;

[0165] Step S503: the display device 200 sends a request message to the server 300;

[0166] In some embodiments, after receiving the request from the user 100 to watch the course media asset segment, the display device 200 sends a request message to the server 300 to obtain the start time and end time of the course media asset segment from the server 300;

[0167] Step S504: the server 300 sends the start time, end time and ts slice of the course media resource to the display device 200;

[0168] In some embodiments, the server 300 parses the message sent by the display device 200, obtains the tag corresponding to the course media asset segment, and matches the start and end times of the corresponding video segment from the information table based on the tag; at the same time, the server 300 matches the corresponding ts slice based on the start time of the course media asset, and sends the start and end times of the course media asset and the ts slice to the display device 200;

[0169] Step S505: the display device 200 displays a dynamic or static picture at the beginning of playing the course media segment;

[0170] In some embodiments, the display device 200 determines whether the video frame corresponding to the start time of the course media asset segment is a key frame based on the start time of the course media asset segment and the information of the ts slice. If not, the display device 200 plays the target key frame before the start time of the course media asset segment and the video between the target key frame and the start time of the course media asset segment in at least one of a fast-forward mode, a silent mode, and a mode of overlaying a target image on the upper layer of the player, so as to display a dynamic or static image at the beginning of the course media asset segment, thereby avoiding displaying content irrelevant to the course media segment to the user 100;

[0171] Step S506: the display device 200 plays the course media resource segment at a preset volume and hides the initially displayed image until the end of the course media resource segment;

[0172] In some embodiments, when the display device 200 detects the start time of the course media segment, the course media segment is played at a preset volume and the initially displayed picture is hidden until the end time of the course media segment. In this way, when the user 100 is watching the course media, the video presented to the user 100 is only the course media segment, thereby improving the viewing experience.

[0173] To facilitate further understanding of the present disclosure, the following describes in detail the various steps performed by the display device 200 and the server 300 in conjunction with some specific embodiments and drawings.

[0174] First, as shown in FIG14 , FIG14 illustrates steps implemented by a processor of a display device provided by an embodiment of the present disclosure, the steps including:

[0175] Step S600: receiving a viewing request for a course media asset segment from a target user, and determining a start time and an end time for playing the course media asset segment. The course media asset segment is a video segment in the course media asset that can be loaded in segments.

[0176] The target user may be a user registered on the display device, or any user who can use the display device to view videos, and this disclosure is not limited to this. The display interface of the display device may display a screen corresponding to each video clip, and the target user may request the display device to view the course media clip by clicking on the display area of ​​the corresponding video clip.

[0177] A course media asset segment may be a short video segment in a long course media asset video. For example, the course media asset segment may be a wonderful fighting scene in a TV series, a knowledge point in a complete educational video, and the like.

[0178] In one embodiment, the present disclosure may determine the start and end time of playing a course media asset segment in the following manner:

[0179] After receiving the video uploaded by the partner, the server determines the start and end time of each video segment in the video during the video playback process and records them. It then determines the label of each video segment based on the content of each video segment and finally generates an information table based on the label of each video segment and the start and end time of each video segment.

[0180] After receiving the target user's viewing request for the course media resource segment, the display device generates a viewing request carrying the course media resource segment tag;

[0181] The display device sends a viewing request to the server;

[0182] After receiving the viewing request, the server parses the viewing request, obtains the label of the course media asset segment, reads the information table from the corresponding memory address, matches the label of the course media asset segment with the label in the information table, and obtains the start and end playback times of the course media asset segment after a successful match.

[0183] After determining the start and end times of the course media asset segment, the server returns the determination result to the display device via the network, so that the display device obtains the data of the start and end times of the course media asset segment.

[0184] Of course, the display device may also have other ways to determine the start and end times of playing the course media asset segments, and this disclosure does not limit this.

[0185] It should also be noted here that when the display device plays a video clip, it needs to download multiple ts slices corresponding to the video clip from the server through the HLS protocol. HLS (HTTP Live Streaming) is Apple's dynamic bitrate adaptive technology. It is mainly used for audio and video services on PCs and Apple terminals. It includes an M3U8 index file, multiple ts slices and a key encryption string file. The M3U8 index file includes information such as the serial number, playback duration, and download address of the ts slice corresponding to each video. The display device obtains the download address of the ts slice corresponding to the video clip from the server, and downloads the ts slice from the corresponding download address while playing.

[0186] Step S700: Play the course media asset segment from the start time of the course media asset segment to the end time of the course media asset segment.

[0187] Among them, after the display device downloads the ts slice corresponding to the course media resource segment from the server, it plays the course media resource segment from the start playback time of the course media resource segment to the end playback time.

[0188] In one embodiment, as shown in FIG15 , FIG15 is an optional method embodiment of a display device playing a course media asset segment provided by an embodiment of the present disclosure, and the method embodiment includes the following steps:

[0189] Step S7001: Obtain the target ts slice corresponding to the course media asset segment from the server;

[0190] Among them, as mentioned above, the display device needs to download the target ts slice corresponding to the video clip from the server through the HLS protocol.

[0191] In one embodiment, after obtaining the course media resources uploaded by the partner, the server may transcode the course media resources into multiple ts slices and generate indexes of the multiple ts slices.

[0192] In one embodiment, the server may transcode the course media assets into multiple TS slices according to the following method:

[0193] The server decapsulates the course media resource through a decoding multiplexer to obtain the MPEG2 format file.

[0194] The server decodes the mpeg2packet through the mpeg2 decoder to obtain multiple video frames corresponding to the course media resources;

[0195] The server determines the encoder, sets the first and second parameters of the encoder, and inputs each video frame into the encoder for encoding, so as to encode each video frame into a corresponding I frame, P frame, and B frame. It should be noted here that the I frame is a key frame, and the I frame is the starting point of video decoding. If you want to play the content of the P frame and the B frame, you must first perform a decoding operation based on the I frame before you can start playing, that is, the content of the P frame and the B frame must be played from the I frame. In this case, if the start time of the course media asset segment is a P frame or a B frame, it will start playing from the I frame before the start time of the course media asset segment. Then, when the display device plays the course media asset segment, the starting content and the subsequent content will be inconsistent, which will cause a bad viewing experience for the user.

[0196] The first parameter is used to limit the number of other coded frames between two adjacent intra-frame coded frames; the second parameter is used to limit the number of bidirectionally predicted coded frames between two adjacent inter-frame predicted coded frames;

[0197] After the server completes encoding of all frames corresponding to the course media resources, it packages all the encoded frames according to the preset encapsulation protocol to obtain multiple ts slice files;

[0198] The server encapsulates each ts slice file according to the ts encapsulation format, obtains multiple ts slice files corresponding to the course media resources and indexes of multiple ts slice files, and the ts slice file index includes information such as file serial number, playback duration and download address.

[0199] The server may determine the target ts slice corresponding to the course media asset segment according to the determined start playing time, end playing time and playing duration of each ts slice of the course media asset segment.

[0200] Step S7002: Determine whether the video frame corresponding to the start time of playing the course media asset segment is a frame other than a key frame in the target ts slice; wherein the target ts slice contains at least one key frame;

[0201] After obtaining the start playback time, end playback time and target ts slice of the course media asset segment sent by the server, the display device can determine whether the start playback time is a frame other than the key frame in the target ts slice based on the start playback time of the course media asset segment.

[0202] It should be noted here that the display device may obtain the frame attributes of each frame of the course media resource and the corresponding playback duration of each frame from the server. For example, if the first frame is an I frame, the corresponding playback duration is 00:10:00-00:10:10;

[0203] The second to fifth frames are P frames, and the corresponding playback duration is 00:10:11-00:12:00;

[0204] The sixth to eighth frames are B frames, and the corresponding playback duration is 00:12:01-00:15:00;

[0205] The ninth frame is an I frame, and the corresponding playback duration is 00:15:01-00:16:10;

[0206] The tenth to fifteenth frames are P frames, and the corresponding playback duration is 00:16:11-00:18:20;

[0207] The 16th frame to the 20th frame are B frames, and the corresponding playback duration is 00:18:21-00:20:00.

[0208] Then, if the start playback time of the course media asset segment is 00:11:0, the video frame corresponding to the start playback time is a frame other than the key frame in the target ts slice.

[0209] If the video frame corresponding to the start of a course media clip is a P-frame or B-frame other than a keyframe, a problem will arise. When playing the course media clip, the display device must start playing from the keyframe before the start of the course media clip. This will cause the image displayed at the beginning of the course media clip to be discontinuous with the image displayed during the course media clip, resulting in a poor viewing experience for users.

[0210] For example, if the start time of the course media segment is 00:11:0, the corresponding video frame is encoded as a P frame, and the key frame before 00:11:0 corresponds to a playback time of 00:10:00-00:10:10, so the course media segment needs to be played starting from 00:10:00. There will be a video with a playback time of 1 minute in the middle. The video content corresponding to this 1-minute video may be content that is irrelevant to the content of the course media segment, or it may be a video that is inconsistent with the starting screen of the course media segment.

[0211] To address this issue, the present disclosure proposes different playback methods for course media asset segments and videos before and between the start playback time of the course media asset segments and the key frames when the video frame corresponding to the start playback time of the course media asset segments is a frame other than the key frame in the target ts slice. See steps S7003 and S7004 for details.

[0212] Step S7003 is to play the video from the key frame to the moment when the course media asset segment starts playing, specifically:

[0213] Step S7003: If yes, then play the target ts slice starting from the target key frame in the target ts slice using the first playback mode until the start playback time of the course media asset segment is reached; wherein the first playback mode is at least two of the following modes: fast forward mode, silent mode, and mode of overlaying the target image on the upper layer of the player; the target key frame is the key frame that is located before the start playback time of the course media asset segment and is closest to the start playback time of the course media asset segment;

[0214] Based on the above description, if the video frame corresponding to the start time of the course media asset segment is not a key frame, but the key frame plays a decoding role in the video, the course media asset segment must be played from the key frame closest to the start time of the course media asset segment. In order to avoid or improve the occurrence of similar problems such as discontinuous images and incoherent content, and to improve the user's viewing experience, the display device can use the first playback mode when playing the video between the key frame closest to the start time of the course media asset segment and the start time of the course media asset segment, thereby achieving the effect of not displaying content irrelevant to the course media asset segment and improving the user's viewing experience.

[0215] In order to achieve the above effect, the first playback method can be to fast forward and play the video silently. Due to the limited resolution of the human eye, the viewing effect of the combination of images frame by frame is the effect of a dynamic video. Therefore, the fast forward multiple can be set to a high multiple, and fast forward directly to the start time of the course media resource segment. The display screen of the display device may jump momentarily, which is not distinguishable by the human eye, and the video is played silently, and the auditory consistency before and after the playback can be maintained, so that the above effect can be achieved.

[0216] Another way to achieve the above effect is to fast-forward the video and overlay the target image on the player during playback. If the fast-forward process still shows video content unrelated to the course media asset, you can fast-forward the video and overlay the content. During the instantaneous jump, the user can view the corresponding image without affecting the viewing experience.

[0217] Another first playback method to achieve the above-mentioned effect is to play the video silently, and overlay the target image on the upper layer of the player during the playback of the video. In this way, while the user is waiting for the content corresponding to the course media asset segment to be played, what the user can see is the image corresponding to the content of the course media asset segment, which does not affect the user's viewing effect. Of course, in order to avoid the user's waiting time and improve the efficiency of the start of the course media asset segment, a fast-forward method can be added to this playback method. This not only improves the playback efficiency of the course media asset segment, but also avoids the user from watching or hearing content that is irrelevant to the content of the course media asset segment, thereby improving the user's viewing experience.

[0218] The display device can select any playback mode according to the display requirements to play the video between the key frame closest to the start time of the course media segment and the start time of the course media segment. There is no limitation here.

[0219] Step S7004 is the instructions for playing the course media clips, specifically:

[0220] Step S7004: When the playback reaches the start time of the course media asset segment, the course media asset segment is played in a second playback mode until the playback reaches the end time of the course media asset segment; wherein the second playback mode is to play at a preset volume and hide the target image;

[0221] Since the purpose of the display device is to present the content corresponding to the course media resource segment to the target user, when the display device plays the course media resource segment, the target image covering the image display layer on the upper layer of the player can be set to invisible. In this way, the picture displayed on the display screen of the display device jumps from the target image to the picture corresponding to the video frame at the start of the course media resource segment. The target image is a picture corresponding to the content of the course media resource segment, so the transition is smooth and there is no abruptness. The volume of the display device is set from mute to the preset volume to play the course media resource segment. This allows the target user to view the content corresponding to the requested course media resource segment.

[0222] Step S7005: Otherwise, play the course media asset segment according to the second play mode until the end of the play time of the course media asset segment.

[0223] In one embodiment, the target image includes an image transmitted by the server, or an image included in a target ts slice played in the first playback mode; the target image includes an animation image and / or a static image.

[0224] The display device disclosed in the present invention can play the video and course media segments between the start time of the target segment and the key frame in different ways when the video frame corresponding to the start time of the course media segment is not a key frame, so that the user will not watch content irrelevant to the content of the course media segment during the viewing process. It can avoid or improve the situation where the starting content and subsequent content are inconsistent when playing the course media segment when the video frame corresponding to the start time of the course media segment is not a key frame, thereby achieving the effect of improving the user's viewing experience.

[0225] In another embodiment, as shown in FIG16 , FIG16 is an optional method embodiment of setting a progress bar executed by a processor of a display device, and the method embodiment includes the following steps:

[0226] Step S801: Setting a progress bar for playing the course media asset segment according to the start and end time of the segment;

[0227] Among them, when the display device is playing the course media clip, the original progress bar of the course media is displayed on the upper layer of the display device player, but the original progress bar of the course media clip does not match the playback time of the course media clip very well. In order to make the playback of the course media clip correspond to the progress bar and give users a better playback experience, the display device will generate a new progress bar for the course media clip according to the start and end playback times of the course media clip, so that the target user can change the playback progress of the course media clip by dragging the progress bar according to his or her viewing intention during the process of watching the course media clip, thereby providing users with a better viewing experience.

[0228] Step S802: When the display plays the course media asset segment, the progress bar is overlaid on the original progress bar of the course media asset for display.

[0229] As shown in FIG17 , the display device includes multiple layers, such as a player window layer, a progress bar display layer, an image display layer, etc. In order to display the new progress bar to the target user, a new layer can be set on the upper layer of the progress bar display layer, and the progress bar display layer can be set to be hidden, and the new layer can be set to be displayed to display the new progress bar on the display screen of the display device, so that the display device can display a progress bar that matches the course media resource segment during the process of playing the course media resource segment.

[0230] Based on the above description, as shown in FIG18 , when the display device is playing a course media asset segment, if a first event is detected, that is, when the target user drags the progress bar to the start playback moment of the course media asset segment while watching the course media asset segment, that is, when the progress bar is initially positioned, the display device may continue to play the ts slice corresponding to the target key frame starting from the target key frame using the first playback mode. When the playback reaches the start playback moment of the course media asset segment, the target ts slice corresponding to the course media asset segment is played using the second playback mode.

[0231] In another case, as shown in FIG19 , when the display device is playing a course media asset segment, if a second event is detected, the second event is when the course media asset is playing to the end of the course media asset segment. If the target user does not perform operations such as pause or exit on the display device, then the display device may continue to play the ts segment corresponding to the target key frame through the first playback mode starting from the target key frame in order to only display content related to the course media asset segment to the target user, and when it reaches the start of the course media asset segment, play the target ts segment of the course media asset segment through the second playback mode until the end of the course media asset segment. At the same time, the display device jumps the progress bar to the starting position.

[0232] In another embodiment, as shown in FIG20 or FIG21, if the display device detects that a third event occurs during the process of playing the course media asset segment, that is, the target ts slice fails to be played or the ts segment corresponding to the target key frame fails to be played. The display device may display a first picture containing information indicating a playback failure on the image display layer. After the preset time period of the first picture is displayed, the progress bar may be jumped to the starting position, and the ts slice corresponding to the target key frame may be played in the first playback mode until the start playback moment of the course media asset segment is played, and the ts slice corresponding to the course media asset segment is played in the second playback mode until the end playback moment.

[0233] In addition, if the target user, while watching a course media clip, requests the display device to play the course media clip at a target speed by touching the display device, the display device can adjust the playback speed of the course media clip to the target speed according to the target speed selected by the target user and move the progress bar according to the target speed. The target speed may be, for example, 1.5x, 2x, 2.5x, etc., and this disclosure is not limited to this.

[0234] The above description of the present disclosure is based on the display device side. The following description is based on aspects related to the server of the present disclosure. As shown in Figure 22, Figure 22 shows steps implemented by a processor of the server provided in an embodiment of the present disclosure. The steps include:

[0235] Step S1401, determining the start and end time of each video segment in the original video;

[0236] Step S1402: determining the content tags corresponding to the course media resources according to the content of each video clip;

[0237] Step S1403: Generate a target information table based on the content tags corresponding to each course media asset and the start and end playback times of each course media asset, and store the target information table in a memory.

[0238] In another embodiment, as shown in FIG23 , the processor of the server is further configured to:

[0239] Step S1501: Receive an acquisition request sent by a display device. The acquisition request is used to obtain playback parameters of a course media asset slice. The playback parameters include the start playback time, end playback time, and target ts slice of the course media asset slice. The acquisition request carries a label of the course media asset slice.

[0240] Step S1502: Match the start and end time of the course media asset slice from the target information table according to the label of the course media asset slice;

[0241] Step S1503: determining the target ts slice according to the start time of the course media asset slice and the ts slice index of the original video;

[0242] Step S1504: Send the start time, end time, and target ts slice of the course media resource slice to the display device.

[0243] It should be noted here that the description of each step of the server has been explained when the processor of the above-mentioned display device executes the corresponding steps, and will not be repeated here.

[0244] For ease of explanation, the above description has been made with reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations are possible. The above embodiments are selected and described to better explain the principles and practical applications, so that those skilled in the art can better utilize the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, comprising: a display configured to display an image and / or a user interface; A user interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with a display device; At least one processor, connected to the display, user interface, communication device and memory, is configured to execute computer instructions to cause the display device to perform: Receive input instructions for playing course media resources; In response to the play instruction, controlling the display to play the course media resource and regularly generating first learning behavior information of the course media resource; If the first learning behavior information is the first learning behavior information generated after the course media asset starts broadcasting, reporting the first learning behavior information to the server; If the first learning behavior information is not the learning behavior information generated for the first time after the course media asset starts broadcasting, the first learning behavior information is not reported to the server.

2. The display device according to claim 1, wherein the first learning behavior information of the course media asset includes a first reporting flag, and the at least one processor is specifically configured to execute computer instructions to cause the display device to execute: Setting a first reporting flag as a first flag in the first learning behavior information generated for the first time after the course media resource starts broadcasting; The first reporting identifier is set as the second identifier in the first learning behavior information generated for the second time and thereafter.

3. The display device according to claim 1 , wherein the step of reporting the first learning behavior information to the server comprises the step of executing computer instructions to cause the display device to: When the playing time of the course media resource is longer than a preset time, the first learning behavior information is reported to the server.

4. The display device according to claim 1 , wherein the periodic generation of the first learning behavior information of the course media asset is performed, and the at least one processor is specifically configured to execute computer instructions to cause the display device to perform: When the media asset data of the course media asset contains an in-plan identifier representing that the course has been included in the learning plan and a completed status identifier representing that the plan has been completed, or when the media asset data contains an out-of-plan identifier representing that the course has not been included in the learning plan, the first learning behavior information of the course media asset is generated regularly.

5. The display device according to claim 1 , wherein the at least one processor is further configured to execute computer instructions to cause the display device to execute computer instructions to cause the display device to execute: When the media asset data of the course media asset contains an in-plan identifier representing that the learning plan has been added, and an incomplete status identifier representing an incomplete plan, second learning behavior information of the course media asset is generated regularly, and the second learning behavior information includes the in-plan identifier and the incomplete status identifier. The second learning behavior information is reported to the server regularly, so that the server pushes a notification message corresponding to the second learning behavior information to the terminal only when the display device reports the second learning behavior information for the first time and the viewing time of the course media asset reaches a preset time based on the in-plan identifier and the incomplete status identifier.

6. The display device according to claim 1, after playing the course media resource, the at least one processor is further configured to execute computer instructions to cause the display device to execute: When the media asset data of the course media asset includes an in-plan identifier indicating that the course has been added to the learning plan and an incomplete status identifier indicating that the course has not been completed, periodically generating second learning behavior information of the course media asset, the second learning behavior information including the in-plan identifier and the incomplete status identifier; Regularly report the second learning behavior information to the server, so that the server pushes a notification message corresponding to the second learning behavior information to the terminal based on the planned identifier and the unfinished status identifier, only when the display device reports the second learning behavior information for the first time and the viewing time of the course media resource reaches a preset time.

7. The display device according to claim 6, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: When the media asset data of the course media asset includes an in-plan flag indicating that the course has been added to the learning plan and a completed status flag indicating that the course has been completed, or when the media asset data includes an out-of-plan flag indicating that the course has not been added to the learning plan, first learning behavior information of the course media asset is periodically generated; If the first learning behavior information is the first learning behavior information generated after the course media asset starts broadcasting, reporting the first learning behavior information to the server; If the first learning behavior information is not the learning behavior information generated for the first time after the course media asset starts broadcasting, the first learning behavior information is not reported to the server.

8. The display device according to claim 7, wherein the at least one processor is configured to: When the playing time of the course media resource is longer than a preset time, the first learning behavior information is reported to the server.

9. The display device according to claim 1, after playing the course media resource, the at least one processor is further configured to execute computer instructions to cause the display device to execute: In response to the play instruction, controlling the display to play the course media resource and regularly generating third learning behavior information of the course media resource; If the media asset data of the course media asset contains an unfinished status indicator indicating an unfinished plan, periodically reporting the third learning behavior information to the server, so that the server pushes a notification message corresponding to the third learning behavior information to the terminal based on the unfinished status indicator only when the display device reports the third learning behavior information for the first time and the viewing time of the course media asset reaches a preset time period; If the media asset data of the course media asset contains a completed status identifier representing a completed plan, when the third learning behavior information is the learning behavior information generated for the first time after the course media asset starts broadcasting, the third learning behavior information is reported to the server; when the third learning behavior information is not the learning behavior information generated for the first time after the course media asset starts broadcasting, the third learning behavior information is not reported to the server.

10. The display device according to any one of claims 1 to 9, wherein the at least one processor is configured to: Determine the start time and end time of playing the course media asset segment, wherein the course media asset segment is a video segment in the course media asset that can be loaded in segments; The course media asset segment is played from the start playing time of the course media asset segment to the end playing time.

11. The display device according to claim 10, wherein the at least one processor is further configured to execute computer instructions to cause the display device to: Get the target ts slice corresponding to the course media resource segment; Determine whether the video frame corresponding to the start time of playing the course media asset segment is a frame other than a key frame in the target ts slice, where the target ts slice contains at least one key frame; If so, starting from the target key frame, the ts slice corresponding to the target key frame is played by a first playback mode until the start playback time of the course media asset segment is played, the first playback mode being at least two of a fast forward mode, a silent mode, and a mode of overlaying a target image on the upper layer of the display, and the target key frame is a key frame that is located before the start playback time of the course media asset segment and is closest to the start playback time of the course media asset segment; When the playback reaches the start time of the course media segment, the course media segment is played through the second playback mode until the playback reaches the end time of the course media segment. The second playback mode is to play at a preset volume and hide the target image.

12. The display device according to claim 11, wherein the at least one processor is further configured to execute computer instructions to cause the display device to: Play the course media segment according to the start and end time of the course media segment Progress bar; When the display plays the course media asset segment, the progress bar is overlaid on the original progress bar of the course media asset for display.

13. The display device according to claim 12, wherein the at least one processor is further configured to execute computer instructions to cause the display device to: If a first event is detected during the playback of the course media asset segment, the ts slice corresponding to the target key frame is played starting from the target key frame through the first playback method and the progress bar is jumped to the position of the start playback time of the course media asset segment until the start playback time of the course media asset segment is played. The first event is that the course media asset segment jumps to the start playback time during the playback process.

14. The display device according to claim 13, wherein the at least one processor is further configured to execute computer instructions to cause the display device to: If a second event is detected during the playback of the course media asset segment, the ts slice corresponding to the target key frame is played starting from the target key frame through the first playback method and the progress bar is jumped to the position of the start playback time of the course media asset segment until the start playback time of the course media asset segment is played. The second event is when the course media asset segment is played to the end playback time.

15. The display device according to claim 10, wherein the at least one processor is further configured to execute computer instructions to cause the display device to: If a third event is detected during the playback of the course media asset segment, the first picture is displayed on the upper layer of the display, the third event being the failure to play the target ts slice, and the first picture includes information indicating the playback failure.

16. The display device according to claim 14, wherein the at least one processor is further configured to execute computer instructions to cause the display device to: During the process of playing the course media segment, if a speed adjustment request is received from the target user, the course media segment is adjusted to the target multiple for playback and the progress bar is moved according to the target multiple. The speed adjustment request is used to request that the course media segment be played at the target speed.

17. The display device according to claim 10, wherein the target image includes an image transmitted by the server, or an image contained in the target ts slice played in the first playback mode; the target image includes an animated image and / or a static image.

18. A data processing method for a display device, comprising: Receive input instructions for playing course media resources; In response to the play instruction, play the course media resource and regularly generate first learning behavior information of the course media resource; If the first learning behavior information is the first learning behavior information generated after the course media asset starts broadcasting, reporting the first learning behavior information to the server; If the first learning behavior information is not the learning behavior information generated for the first time after the course media asset starts broadcasting, the first learning behavior information is not reported to the server.

Images