Display device, image display control method, and storage medium

Abstract

The present disclosure relates to a display device, an image display control method, and a storage medium. The display device comprises a controller configured to: send a first notification message to a player through an application program interface of an application layer, the first notification message being used to instruct the application layer to replace a first surface window with a second surface window; when detecting that a value of a first state flag bit is true, notify the player to pause a video decoding operation and a video refreshing display operation through the first surface window; the value of the first state flag bit being used to indicate whether the first surface window is changed; when receiving a first response message returned by the player, destroy the first surface window, and create the second surface window, and set the second surface window to the player, so that the player performs the video decoding operation and the video refreshing display operation through the second surface window; the first response message being used to indicate that the player has received the first notification message.

Description

Technical Field

[0001] This disclosure relates to the field of video playback technology, and in particular to a display device, an image display control method, and a storage medium. Background Technology

[0002] Multimedia video playback is a core function of smart TVs. With the development of video recording equipment and the enhanced processing power of smart TV chips, the form of video playback on smart TVs is also changing. For example, current smart TVs can support simultaneous playback of multiple videos, which are displayed as floating windows in multiple locations on the screen. Although Android provides a picture-in-picture (PiP) function, this function imposes many restrictions on the operation and layout of controls. When developing customized features, the provided PiP function cannot meet design requirements; therefore, it is necessary to optimize and improve the existing functions of the Android system.

[0003] In existing technologies, during video playback, the operation of changing the surface in the Java layer is notified to the underlying player through cross-process asynchronous calls. Because of the asynchronous processing, the underlying player cannot detect the change in the Java layer surface in time (the time difference is tens of milliseconds). When the underlying player performs decoding and display processing, it will use the destroyed Java variables, which will cause various abnormal problems, such as image freezing, black screen, playback errors, etc.

[0004] Therefore, how to avoid playback abnormalities caused by changes in the surface during video playback is an urgent problem that needs to be solved. Summary of the Invention

[0005] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a display device and an image display control method, which can avoid problems such as black screen, freezing, and abnormal playback exit caused by asynchronous calls between different processes that cannot guarantee timing.

[0006] In a first aspect, this disclosure provides a display device, comprising:

[0007] The controller is configured as follows:

[0008] A first notification message is sent to the player through the application layer's application programming interface. The first notification message is used to instruct the application layer to replace the first surface window with the second surface window.

[0009] When the value of the first status flag is detected to be true, the player is notified to pause the video decoding and video refresh display operations through the first surface window; the value of the first status flag is used to indicate whether the first surface window has changed.

[0010] When the first response message returned by the player is received, the first surface window is destroyed and the second surface window is created. The second surface window is then assigned to the player so that the player can perform video decoding and video refresh display operations through the second surface window. The first response message is used to indicate that the player has received the first notification message.

[0011] As an optional implementation of this invention, after sending the first notification message to the player through the application programming interface of the application layer, the method further includes:

[0012] If the player receives the first notification message, it sets the value of the first status flag to true;

[0013] If the player does not receive the first notification message, the value of the first status flag is set to false.

[0014] As an optional implementation of this invention, the controller is further configured to:

[0015] Before notifying the player to pause video decoding and video refresh operations when the value of the first status flag is detected to be true, the following steps are included:

[0016] Determine whether the value of the first status flag is true;

[0017] If the value of the first status flag is true, then the player is notified to pause the video decoding operation and the video refresh display operation;

[0018] If the value of the first status flag is false, the player continues to perform video decoding and video refresh display operations.

[0019] As an optional implementation of this invention, the controller is specifically configured as follows:

[0020] After informing the player to pause video decoding and video refresh operation if the value of the first status flag is true, the method further includes:

[0021] The player sends the first response message to the application layer via a callback function.

[0022] As an optional implementation of this invention, the controller is further configured to:

[0023] After setting the second surface window to the player, the method further includes:

[0024] If the player receives the second surface window, it sets the value of the first status flag to false.

[0025] As an optional implementation of this invention, the controller is further configured to:

[0026] Destroying the first surface window and creating the second surface window includes:

[0027] The first surface window is destroyed by calling the window destruction function;

[0028] The second surface window is created by calling the window creation function.

[0029] As an optional embodiment of the present invention, the player includes: a video decoding and display control module, a video decoding module, and a video display module.

[0030] Secondly, an image display control method is provided, the method comprising:

[0031] A first notification message is sent to the player through the application layer's application programming interface. The first notification message is used to instruct the application layer to replace the first surface window with the second surface window.

[0032] When the value of the first status flag is detected to be true, the player is notified to pause the video decoding and video refresh display operations through the first surface window; the value of the first status flag is used to indicate whether the first surface window has changed.

[0033] When the first response message returned by the player is received, the first surface window is destroyed and the second surface window is created. The second surface window is then assigned to the player so that the player can perform video decoding and video refresh display operations through the second surface window. The first response message is used to indicate that the player has received the first notification message.

[0034] As an optional implementation of this invention, after sending the first notification message to the player through the application programming interface of the application layer, the method further includes:

[0035] If the player receives the first notification message, it sets the value of the first status flag to true;

[0036] If the player does not receive the first notification message, the value of the first status flag is set to false.

[0037] As an optional implementation of this invention, before notifying the player to pause video decoding and video refresh display operations when the value of the first status flag is detected to be true, the following steps are included:

[0038] Determine whether the value of the first status flag is true;

[0039] If the value of the first status flag is true, then the player is notified to pause the video decoding operation and the video refresh display operation;

[0040] If the value of the first status flag is false, the player continues to perform video decoding and video refresh display operations.

[0041] As an optional implementation of this invention, after notifying the player to pause video decoding and video refresh display operations if the value of the first status flag is true, the method further includes:

[0042] The player sends the first response message to the application layer via a callback function.

[0043] As an optional implementation of this invention, after setting the second surface window to the player, the method further includes:

[0044] If the player receives the second surface window, it sets the value of the first status flag to false.

[0045] As an optional implementation of this invention, the step of destroying the first surface window and creating the second surface window includes:

[0046] The first surface window is destroyed by calling the window destruction function;

[0047] The second surface window is created by calling the window creation function.

[0048] As an optional embodiment of the present invention, the player includes: a video decoding and display control module, a video decoding module, and a video display module.

[0049] Thirdly, a computer-readable storage medium is provided, comprising: storing a computer program on the computer-readable storage medium, wherein when the computer program is executed by a processor, it implements the image display control method as described in the second aspect.

[0050] Fourthly, a computer program product is provided, comprising: when the computer program product is run on a computer, causing the computer to implement the image display control method as shown in the second aspect.

[0051] The technical solution provided in this disclosure has the following advantages compared with the prior art: A first notification message is sent to the player through the application layer's application programming interface (API). This first notification message instructs the application layer to replace the first surface window with a second surface window. When the value of the first status flag is detected to be true, the player is notified to pause video decoding and video refresh operations through the first surface window. The value of the first status flag indicates whether the first surface window has changed. Upon receiving a first response message from the player, the first surface window is destroyed, and a second surface window is created and assigned to the player, allowing the player to perform video decoding and video refresh operations through the second surface window. The first response message indicates that the player has received the first notification message. By monitoring the surface window's state change through the value of the first status flag, and sending the first notification message to the player before the application layer changes the surface window, the player, upon receiving the message that the surface window is about to change, informs the application layer to begin switching the surface window via the first response message. This avoids problems such as black screen, freezing, and abnormal playback exit caused by asynchronous calls between different processes that cannot guarantee timing. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0053] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1A A schematic diagram of the software architecture of an Android system media player provided in an embodiment of this disclosure;

[0055] Figure 1BThis is a schematic diagram illustrating an application scenario of an image display control method for a display device provided in an embodiment of this disclosure;

[0056] Figure 1C This is a hardware configuration block diagram of a control device 100 according to one or more embodiments of the present disclosure;

[0057] Figure 2 This is a hardware configuration block diagram of a computer device according to one or more embodiments of the present disclosure;

[0058] Figure 3A This is a schematic diagram of the software configuration of a computer device according to one or more embodiments of the present disclosure;

[0059] Figure 3B This is a schematic diagram showing an icon control page of an application program on a computer device according to one or more embodiments of the present disclosure;

[0060] Figure 4 A flowchart illustrating an image display control method provided in an embodiment of this disclosure;

[0061] Figure 5A This is a schematic diagram of the structure of a player provided in an embodiment of the present disclosure;

[0062] Figure 5B A flowchart illustrating another image display control method provided in an embodiment of this disclosure;

[0063] Figure 6 A flowchart illustrating yet another image display control method provided in this disclosure embodiment;

[0064] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0065] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0066] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0067] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0068] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0069] Terminology Explanation:

[0070] Asynchronous method calls: These are methods that allow operations to continue without waiting for the return value of the called function. For example, implemented using multiple processes / threads, the main process / thread initiates a request, the system selects another process / thread to take over the request task, and then, after the asynchronous call completes, the system selects an available thread to execute the callback function, pushing the result back to the main process / thread. The main purpose of asynchronous method calls is to prevent the calling process / thread from synchronously waiting on the current function call, allowing it to perform other request operations.

[0071] Surface window: also known as the surface, is where Android's graphics are drawn.

[0072] mediaPlayer: Media Player.

[0073] Android-based electronic devices require a player to render images within a system-provided Surface window to play videos. While Android offers a picture-in-picture (PiP) function, this feature imposes significant limitations on control operation and layout. When developing customized features, Android's PiP functionality cannot meet design requirements; therefore, optimization and improvement of existing Android features are necessary. (Refer to...) Figure 1A As shown, Figure 1A The software architecture of the Android media player includes: the Android player Java layer, the Android player native layer, the Android player C layer, and the Android player middleware layer. Applications frequently use the API (Application Programming Interface) of the MediaPlayer Java layer. Figure 1AThe Android media player middleware layer is a customized development module for the Android native MediaPlayer system. As shown in the diagram, the native layer receives the surface created by the Java layer for image display via JNI. The native layer obtains the content to be rendered from the image data, renders the content, and refreshes it onto the surface. From the application layer to the underlying player, the interface calls need to pass through Java, JNI, and C layers. In the Android system, this is achieved through cross-process calls. For example, if the application changes the surface during video playback, current Android systems may experience image freezing, black screens, and playback errors. This is because the application and the player are not in the same process. The application's operation of changing the surface in the Java layer is notified to the underlying player through asynchronous cross-process calls. Due to asynchronous processing, the underlying player cannot detect the application's surface change in the Java layer in time. During this process, the underlying player uses released Java variables during decoding and display processing, leading to various exceptions.

[0074] To address the aforementioned issues, in this embodiment, a first notification message is sent to the player via the application layer's application programming interface (API). This first notification message instructs the application layer to switch from a first surface window to a second surface window. When a first status flag is detected to be true, the player is notified to pause video decoding and refresh operations via the first surface window. The value of the first status flag indicates whether the first surface window has changed. Upon receiving a first response message from the player, the first surface window is destroyed, and a second surface window is created and assigned to the player, enabling the player to perform video decoding and refresh operations via the second surface window. The first response message indicates that the player has received the first notification message. By monitoring the surface window's state change using the value of the first status flag, and sending the first notification message to the player before the application layer changes the surface window, the player, upon receiving the message that the surface window is about to change, informs the application layer to begin switching the surface window via the first response message. This avoids issues such as black screens, freezing, and abnormal playback exits caused by asynchronous calls between different processes that cannot guarantee timing.

[0075] For example, such as Figure 1A As shown, Figure 1A This is a schematic diagram illustrating an application scenario of an image display control method for a display device provided in an embodiment of this disclosure. Figure 1AIn this system, the user can operate the display device 200 through the control device 100 or the terminal device 300. For example, during video playback, the user can switch from the current application to another application or open another small window on the current application to play another video. The control flow for changing the surface during video playback is as follows: a first notification message is sent to the player through the application layer's application programming interface (API). The first notification message instructs the application layer to replace the first surface window with a second surface window. When the value of the first status flag is detected to be true, the player is notified to pause video decoding and video refresh operations through the first surface window. The value of the first status flag indicates whether the first surface window has changed. When the first response message returned by the player is received, the first surface window is destroyed, and the second surface window is created and assigned to the player so that the player can perform video decoding and video refresh operations through the second surface window. The first response message indicates that the player has received the first notification message. Among them, the display device 200 can be a smart TV, a laptop, a projector, a desktop computer, etc.; the terminal device 300 can be a smartphone, a tablet, a portable wristband, etc.

[0076] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0077] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can input user commands through buttons on the remote control, voice input, control panel input, etc., to control the display device 200. Figure 1B An exemplary configuration block diagram of the control device 100 is shown. (Refer to...) Figure 1BAs shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a storage device, and a power supply. The control device 100 can receive user input operation commands and convert them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200. The communication interface 130 is used for external communication and includes at least one of the following: a Wi-Fi chip, a Bluetooth module, NFC (Near Field Communication), or a suitable alternative module. The user input / output interface 140 includes at least one of the following: a microphone, a touchpad, a sensor, buttons, or a suitable alternative module.

[0078] In some embodiments, a terminal device 300 (e.g., a mobile terminal) can also be used to control the display device 200. For example, an application running on the terminal device 300 can be used to control the display device 200. The terminal device 300 can install software applications with the display device 200 to establish a connection and communication via network communication protocols, achieving one-to-one control operations and data communication. Semantic content displayed on the terminal device 300 can also be transmitted to the display device 200 to achieve synchronous display.

[0079] In some embodiments, the display device 200 may receive commands without using the terminal device 300 described above, but instead receive user control via touch or gestures. For example, when the display device 200 is a smart TV, the user accesses the wireless network display interface via touch screen or gesture control.

[0080] The image display control method provided in this disclosure can be implemented based on a computer device, or a functional module or functional entity in the computer device.

[0081] The computer equipment can be a personal computer (PC), server, mobile phone, tablet computer, laptop computer, mainframe computer, etc., and this disclosure does not specifically limit it.

[0082] For example, Figure 2 This is a hardware configuration block diagram of a computer device according to one or more embodiments of the present disclosure. Figure 2As shown, the computer device includes at least one of the following: a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (Random Access Memory), ROM (Read-Only Memory), a first to nth interface for input / output, and a communication bus. The display 260 can be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and can also be a projection device and a projection screen. The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG audio and video data signals, from multiple wireless or wired broadcast television signals. The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. Computer equipment can establish the transmission and reception of control signals and data signals with a server or local control device through the communicator 220. The detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, the detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, the detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, the detector 230 includes a sound acquisition device, such as a microphone, for receiving external sound. The external device interface 240 may include, but is not limited to, one or more of the following: a high-definition multimedia interface (HDMI), an analog or data high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces. The controller 250 and the tuner 210 can be located in different separate devices, that is, the tuner 210 can also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0083] In some embodiments, the controller 250 controls the operation of the computer device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the computer device. The user can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0084] In some embodiments, the display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, an assembly for receiving image signals output from a controller, and a user interface (UI) for displaying video content, image content, menu control interface, and user control interface. For example, the display may be at least one of a liquid crystal display (LCD), an OLED (Organic Light-Emitting Diode) display, a touch display, and a projection display, and may also be a projection device and a projection screen. Users can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the GUI. Alternatively, users can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors. A "user interface" is the medium through which an application or operating system interacts and exchanges information with a user; it realizes the conversion between the internal form of information and a form acceptable to the user. A common form of user interface is the graphical user interface, which refers to a user interface related to computer operation displayed graphically. For example, it can be an interface element such as an icon, window, or control displayed on the screen of a computer device. Controls can include visible interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, and navigation bars. The user interface can be used to receive control signals from terminal device 100 (such as an infrared remote control).

[0085] Figure 3A This is a schematic diagram of the software configuration of a computer device according to one or more embodiments of the present disclosure, such as... Figure 3A As shown, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Library Layer"), and the kernel layer.

[0086] Figure 3B This is a schematic diagram showing the icon control interface of an application included in a smart device (mainly a smart playback device, such as a smart TV, digital cinema system, or audio-visual server) according to one or more embodiments of this disclosure. Figure 3B As shown, the application layer contains at least one application whose corresponding icon control can be displayed on the screen, such as: live TV application icon control, video-on-demand (VOD) application icon control, media center application icon control, application center icon control, game application icon control, etc. Live TV applications can provide live television from different signal sources. Video-on-demand (VOD) applications can provide video from different storage sources. Unlike live TV applications, video-on-demand provides video display from certain storage sources. Media center applications can provide applications for playing various multimedia content. The application center can provide storage for various applications.

[0087] The image display control method provided in this disclosure can be implemented based on the aforementioned computer equipment.

[0088] To illustrate this solution in more detail, the following will use examples to illustrate it. Figure 4 To explain, it is understandable that Figure 4 The steps involved may include more or fewer steps in actual implementation, and the order of these steps may also be different, in order to achieve the image display control method for the controlled device side provided in the embodiments of this application.

[0089] Figure 4 This is a flowchart illustrating an image display control method provided in an embodiment of this disclosure. Figure 4 As shown, the method specifically includes the following steps:

[0090] S41. Send the first notification message to the player through the application programming interface of the application layer.

[0091] The first notification message is used to instruct the application layer to replace the first surface window with the second surface window.

[0092] Specifically, a first notification message is sent to the player via the MediaPlayer Java Application Programming Interface (API) to instruct the first surface window to be replaced with the second surface window.

[0093] For example, add a message to the existing Android system's MediaPlayer.java to notify the player layer of surface changes from the application layer.

[0094] For example, add the INVOKE message INVOKE_ID_SET_NEW_SURFACE to MediaPlayer.java (see MediaPlayer.java file) and mediaplayer.h (see mediaplayer.h file) to indicate that the surface needs to be changed.

[0095] The INVOKE message defined in the MediaPlayer.java file is as follows:

[0096] private static final int INVOKE_ID_GET_TRACK_INFO=1;

[0097] private static final int INVOKE_ID_ADD_EXTERNAL_SOURCE=2;

[0098] private static final int INVOKE_ID_ADD_EXTERNAL_SOURCE_FD=3;

[0099] private static final int INVOKE_ID_SELECT_TRACK=4;

[0100] private static final int INVOKE_ID_DESELECT_TRACK=5;

[0101] private static final int INVOKE_ID_SET_VIDEO_SCALE_MODE=6;

[0102] private static final int INVOKE_ID_GET_SELECTED_TRACK=7;

[0103] private static final int INVOKE_ID_SET_NEW_SURFACE=8;

[0104] The INVOKE message defined in the mediaplayer.h file is as follows:

[0105] / / keep INVOKE_ID_*in sync with MediaPlayer.java.

[0106] Enum media_player_invoke_ids{

[0107] INVOKE_ID_GET_TRACK_INFO=1;

[0108] INVOKE_ID_ADD_EXTERNAL_SOURCE=2;

[0109] INVOKE_ID_ADD_EXTERNAL_SOURCE_FD=3;

[0110] INVOKE_ID_SELECT_TRACK = 4;

[0111] INVOKE_ID_UNSELECT_TRACK=5;

[0112] INVOKE_ID_SET_VIDEO_SCALEING_MODE=6;

[0113] INVOKE_ID_GET_SELECTED_TRACK=7;

[0114] INVOKE_ID_SET_NEW_SURFACE=8;

[0115] }

[0116] It should be noted that in the MediaPlayer.java and mediaplayer.h files, "INVOKE_ID_SET_NEW_SURFACE=8" is the newly added first notification message; the other seven items are INVOKE messages defined in the native Android system. The purpose of using INVOKE messages instead of adding new API interfaces is to reduce changes to the Android system and maintain API consistency.

[0117] S42. When the value of the first status flag is detected to be true, the player is notified to pause the video decoding operation and video refresh display operation through the first surface window.

[0118] The value of the first status flag is used to indicate whether the first surface window has changed.

[0119] Specifically, when the first status flag is detected to be true, it means that the player has received the first notification message sent by the application layer. The application layer notifies the player to pause the video decoding and video refresh display operations through the first surface window. This pause operation is the preparatory work.

[0120] Reference Figure 5A As shown, Figure 5A This is a schematic diagram of the player's structure, which includes: a video decoding and display control module, a video decoding module, and a video display module.

[0121] Video decoding and display control module: Used to receive messages from the application layer indicating that the surface needs to be changed, namely the first notification message INVOKE_ID_SET_NEW_SURFACE. After receiving the first notification message, the video decoding and display module changes the relevant status flags and sends the MEDIA_INFO_GOT_NEW_SURFACE message to the application layer, namely the first response message.

[0122] Video decoding module: Used to decode video, and controls the video decoding process according to the value of the first status flag.

[0123] Video display module: Used to control the synchronous display of decoded video data, and controls the video display process according to the value of the first status flag.

[0124] For example, when the value of the first status flag is detected to be true, the player's video decoding and display control module receives the message that the surface set by the application layer needs to be changed, i.e., the first notification message, and then notifies the video decoding module to pause decoding of video data and notifies the video display module to pause refreshing of video data.

[0125] S43. When the first response message returned by the player is received, the first surface window is destroyed and the second surface window is created. The second surface window is set to the player so that the player can perform video decoding and video refresh display operations through the second surface window.

[0126] The first response message is used to indicate that the player has received the first notification message.

[0127] In some embodiments, after performing step S62 above, the following steps may also be performed:

[0128] The player sends the first response message to the application layer via a callback function.

[0129] Specifically, when the player receives the first response message, it indicates that the player has received the application layer's message to change the surface and has completed pre-prepared work, namely, pausing video decoding and video refresh operations through the first surface window. In this case, the player sends a first response message to the application layer, notifying it that the first notification message has been received. Then, the application layer destroys the first surface window, creates a second surface window, and assigns the second surface window to the player, allowing the player to perform video decoding and video refresh operations through the second surface window.

[0130] For example, after the application layer notifies the player of the intention to change the surface via the MediaPlayer's INVOKE message, the application layer enters a message waiting state, waiting for the player to complete its preparatory work. For instance, adding MEDIA_INFO_GOT_NEW_SURFACE = 704 to the media_info_type in the mediaplayer.h file, in the native Android system, adding this value 704 signifies that the player has received the application layer's message to change the surface. That is, the first response message indicates that the player has received the first notification message. The player sends the value 704 to the application layer through a callback function in the native Android system. Upon receiving the value 704, the application layer considers the player to be ready to change the surface.

[0131] enum media_info_type{

[0132] / / 0xx

[0133] MEDIA_INFO_UNKNOWN = 1,

[0134] MEDIA_INFO_STARTED_AS_NEXT=2,

[0135] MEDIA_INFO_RENDERING_STRAT=3,

[0136] / / 7xx

[0137] MEDIA_INFO_VIDEO_TRACK_LAGGING=700,

[0138] MEDIA_INFO_BUFFERING_START=701,

[0139] MEDIA_INFO_BUFFERING_END=702,

[0140] MEDIA_INFO_NETWORK_BANDWIDTH=703,

[0141] MEDIA_INFO_GOT_NEW_SURFACE=704,

[0142] }

[0143] In this embodiment, a first notification message is sent to the player via the application layer's application programming interface (API). This first notification message instructs the application layer to switch from a first surface window to a second surface window. When a first status flag is detected to be true, the player is notified to pause video decoding and video refresh operations via the first surface window. The value of the first status flag indicates whether the first surface window has changed. Upon receiving a first response message from the player, the first surface window is destroyed, and a second surface window is created and assigned to the player, allowing the player to perform video decoding and video refresh operations via the second surface window. The first response message indicates that the player has received the first notification message. By monitoring the surface window's state change using the value of the first status flag, and sending the first notification message to the player before the application layer changes the surface window, the player, upon receiving the message that the surface window is about to change, informs the application layer to begin switching the surface window via the first response message. This avoids problems such as black screens, freezing, and abnormal playback exits caused by asynchronous calls between different processes that cannot guarantee timing.

[0144] Figure 5B This is a schematic flowchart of another image display control method provided in this embodiment. This embodiment is... Figure 4 This can be further expanded and optimized based on the previous steps. Optionally, after executing step S51, the following steps S52-S53 can also be executed:

[0145] S51. Send the first notification message to the player through the application programming interface of the application layer.

[0146] The first notification message is used to instruct the application layer to replace the first surface window with the second surface window.

[0147] The specific execution method of step S51 is the same as that of step S41 in the above embodiment, and will not be described again here.

[0148] S52. If the player receives the first notification message, it sets the value of the first status flag to true.

[0149] S53. If the player does not receive the first notification message, then the value of the first status flag is set to false.

[0150] Specifically, after the first notification message is sent, the application layer is in a message waiting state. After the player receives the first notification message, it sets the value of the first state flag to true.

[0151] For example, after the first notification message is sent, the application layer is in a message waiting state. If the player receives the INVOKE_ID_SET_NEW_SURFACE message, it will set the status flag indicating that the surface has changed to true; if the player does not receive the INVOKE_ID_SET_NEW_SURFACE message, it will set the status flag indicating that the surface has changed to false.

[0152] S54. When the value of the first status flag is detected to be true, the player is notified to pause the video decoding operation and video refresh display operation through the first surface window.

[0153] The value of the first status flag is used to indicate whether the first surface window has changed.

[0154] S55. When the first response message returned by the player is received, the first surface window is destroyed and the second surface window is created. The second surface window is set to the player so that the player can perform video decoding and video refresh display operations through the second surface window.

[0155] The first response message is used to indicate that the player has received the first notification message.

[0156] It should be noted that steps S54-S55 are executed in the same way as steps S42-S43 in the above embodiments, and will not be described again here.

[0157] In this embodiment, a first notification message is sent to the player via the application layer's application programming interface (API). This first notification message instructs the application layer to switch from a first surface window to a second surface window. When a first status flag is detected to be true, the player is notified to pause video decoding and video refresh operations via the first surface window. The value of the first status flag indicates whether the first surface window has changed. Upon receiving a first response message from the player, the first surface window is destroyed, and a second surface window is created and assigned to the player, allowing the player to perform video decoding and video refresh operations via the second surface window. The first response message indicates that the player has received the first notification message. By monitoring the surface window's state change using the value of the first status flag, and sending the first notification message to the player before the application layer changes the surface window, the player, upon receiving the message that the surface window is about to change, informs the application layer to begin switching the surface window via the first response message. This avoids problems such as black screens, freezing, and abnormal playback exits caused by asynchronous calls between different processes that cannot guarantee timing.

[0158] Figure 6 This is a schematic flowchart of another image display control method provided in this embodiment. This embodiment is... Figure 4 This can be further expanded and optimized based on the above. Optionally, before performing step S42, the following steps can also be performed:

[0159] S61. Send the first notification message to the player through the application programming interface of the application layer.

[0160] The first notification message is used to instruct the application layer to replace the first surface window with the second surface window.

[0161] The specific execution method of step S61 is the same as that of step S41 in the above embodiment, and will not be described again here.

[0162] S62. Determine whether the value of the first status flag is true.

[0163] If the value of the first status flag is true, then execute steps S63-S64; if the value of the first status flag is false, then execute step S65.

[0164] S63. Notify the player to pause video decoding and video refresh display operations.

[0165] S64. When the first response message returned by the player is received, the first surface window is destroyed and the second surface window is created. The second surface window is set to the player so that the player can perform video decoding and video refresh display operations through the second surface window.

[0166] The first response message is used to indicate that the player has received the first notification message.

[0167] The specific execution methods of steps S63-S64 are the same as those of steps S42-S43 in the above embodiments, and will not be described again here.

[0168] S65, the player continues to perform video decoding and video refresh display operations.

[0169] The player's video decoding and display module detects that the value of the first status flag is false, and sends the value of the first status flag to the video decoding module and the video display module. Since the value of the first status flag is false, it means that the application layer has no need to change the surface, so the player's video decoding module continues to decode the video data, and the video display module continues to refresh and display the video.

[0170] By monitoring the value of the first status flag to see if the surface state changes, the execution method of the decoding and display module was adjusted to avoid playback abnormalities caused by changes in the surface during video playback.

[0171] In this embodiment, a first notification message is sent to the player via the application layer's application programming interface (API). This first notification message instructs the application layer to switch from a first surface window to a second surface window. When a first status flag is detected to be true, the player is notified to pause video decoding and video refresh operations via the first surface window. The value of the first status flag indicates whether the first surface window has changed. Upon receiving a first response message from the player, the first surface window is destroyed, and a second surface window is created and assigned to the player, allowing the player to perform video decoding and video refresh operations via the second surface window. The first response message indicates that the player has received the first notification message. By monitoring the surface window's state change using the value of the first status flag, and sending the first notification message to the player before the application layer changes the surface window, the player, upon receiving the message that the surface window is about to change, informs the application layer to begin switching the surface window via the first response message. This avoids problems such as black screens, freezing, and abnormal playback exits caused by asynchronous calls between different processes that cannot guarantee timing.

[0172] In some embodiments, after performing the step (assigning the second surface window to the player), the following steps may also be performed:

[0173] If the player receives the second surface window, it sets the value of the first status flag to false.

[0174] WindowManager is an important service in the Android system. It is global and unique. WindowManager inherits from ViewManager. WindowManager is mainly used to manage some states and attributes of windows, add, delete, and update window views, window order, message collection and processing, etc. In addition, there is an important static class LayoutParams in WindowManager, which can be used to set and get some attributes of the current window.

[0175] Since the player has already obtained the relevant information about the second surface window through WindowManager, it performs video decoding and refresh operations based on the second surface window. Simultaneously, it sets the first state flag to false to monitor changes in the application-layer surface state in the future.

[0176] In some embodiments, destroying the first surface window and creating the second surface window includes:

[0177] The first surface window is destroyed by calling the window destruction function;

[0178] The second surface window is created by calling the window creation function.

[0179] The function to destroy the first surface window is called, and the function to create the second surface window is called. The second surface window is used to display the video data subsequently parsed by the player's video parsing module.

[0180] The image display control method provided in this disclosure implements interface and display control management for surface changes at the Android player multimedia middleware layer, avoiding excessive reliance on underlying decoding manufacturers. At the same time, it can accommodate various video window display methods of applications, enabling flexible and effective management and control of video display windows, and improving the software stability and flexibility of the Android system.

[0181] In this embodiment, a first notification message is sent to the player via the application layer's application programming interface (API). This first notification message instructs the application layer to switch from a first surface window to a second surface window. When a first status flag is detected to be true, the player is notified to pause video decoding and video refresh operations via the first surface window. The value of the first status flag indicates whether the first surface window has changed. Upon receiving a first response message from the player, the first surface window is destroyed, and a second surface window is created and assigned to the player, allowing the player to perform video decoding and video refresh operations via the second surface window. The first response message indicates that the player has received the first notification message. By monitoring the surface window's state change using the value of the first status flag, and sending the first notification message to the player before the application layer changes the surface window, the player, upon receiving the message that the surface window is about to change, informs the application layer to begin switching the surface window via the first response message. This avoids problems such as black screens, freezing, and abnormal playback exits caused by asynchronous calls between different processes that cannot guarantee timing.

[0182] This disclosure provides a computer device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the image display control methods described in this disclosure.

[0183] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Figure 7 As shown, the computer device includes a processor 710 and a storage device 720; the number of processors 710 in the computer device can be one or more. Figure 7 Taking a processor 710 as an example; the processor 710 and the storage device 720 in a computer device can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0184] The storage device 720, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the image display control method in the embodiments of this disclosure. The processor 710 executes various functional applications and data processing of the computer device by running the software programs, instructions, and modules stored in the storage device 720, thereby realizing the image display control method provided in the embodiments of this disclosure.

[0185] Storage device 720 may primarily include a stored program area and a stored data area. The stored program area may store the operating system and at least one application program required for a given function; the stored data area may store data created based on terminal usage. Furthermore, storage device 720 may include a high-speed random access storage device, and may also include a non-volatile storage device, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, storage device 720 may further include storage devices remotely located relative to processor 710, which can be connected to computer equipment via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0186] The computer device provided in this embodiment can be used to execute the image display control method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0187] This disclosure also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions implement the various processes of the methods provided in any of the above embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here.

[0188] The computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0189] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the discussion in some embodiments above is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of the embodiments suitable for specific application considerations.

Claims

1. A display device, characterized in that, include: The controller is configured as follows: A first notification message is sent to the player through the application layer's application programming interface. The first notification message is used to instruct the application layer to replace the first surface window with the second surface window. When the value of the first status flag is detected to be true, the player is notified to pause the video decoding operation and video refresh display operation through the first surface window; The value of the first status flag is used to indicate whether the first surface window has changed; When the first response message returned by the player is received, the first surface window is destroyed and the second surface window is created. The second surface window is then assigned to the player so that the player can perform video decoding and video refresh display operations through the second surface window. The first response message is used to indicate that the player has received the first notification message.

2. The display device according to claim 1, characterized in that, The controller is also configured to: After sending the first notification message to the player through the application layer's application programming interface, the method further includes: If the player receives the first notification message, it sets the value of the first status flag to true; If the player does not receive the first notification message, the value of the first status flag is set to false.

3. The display device according to claim 2, characterized in that, The controller is also configured to: Before notifying the player to pause video decoding and video refresh operations when the value of the first status flag is detected to be true, the following steps are included: Determine whether the value of the first status flag is true; If the value of the first status flag is true, then the player is notified to pause the video decoding operation and the video refresh display operation; If the value of the first status flag is false, the player continues to perform video decoding and video refresh display operations.

4. The display device according to claim 3, characterized in that, The controller is also configured to: After informing the player to pause video decoding and video refresh operation if the value of the first status flag is true, the method further includes: The player sends the first response message to the application layer via a callback function.

5. The display device according to claim 1, characterized in that, The controller is also configured to: After setting the second surface window to the player, the method further includes: If the player receives the second surface window, it sets the value of the first status flag to false.

6. The display device according to claim 1, characterized in that, The controller is specifically configured as follows: Destroying the first surface window and creating the second surface window includes: The first surface window is destroyed by calling the window destruction function; The second surface window is created by calling the window creation function.

7. The display device according to claim 1, characterized in that, The player includes: a video decoding and display control module, a video decoding module, and a video display module.

8. An image display control method, characterized in that, The method includes: A first notification message is sent to the player through the application layer's application programming interface. The first notification message is used to instruct the application layer to replace the first surface window with the second surface window. When the value of the first status flag is detected to be true, the player is notified to pause the video decoding and video refresh display operations through the first surface window; the value of the first status flag is used to indicate whether the first surface window has changed. When the first response message returned by the player is received, the first surface window is destroyed and the second surface window is created. The second surface window is then assigned to the player so that the player can perform video decoding and video refresh display operations through the second surface window. The first response message is used to indicate that the player has received the first notification message.

9. The method according to claim 8, characterized in that, After sending the first notification message to the player through the application layer's application programming interface, the method further includes: If the player receives the first notification message, it sets the value of the first status flag to true; If the player does not receive the first notification message, the value of the first status flag is set to false.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image display control method as described in any one of claims 8-9.

Images