Display method and display equipment

Abstract

The invention discloses a display method and a display device.The display method comprises the steps that after zoom adjustment operation is received, zoom processing is executed on a display picture of media assets, meanwhile, target duration delay waiting matched with picture zoom duration is started, and after delay timing reaches the standard, the size of a display window is synchronously adjusted; finally, the zoomed picture and window are presented, synchronous display of the media asset picture and the display window is achieved, and the visual experience of a user when zooming operation is executed on the media assets is improved.

Description

Technical Field

[0001] This application relates to the field of display devices, and more particularly to display methods and display devices. Background Technology

[0002] As the intelligence level of display devices continues to improve, when users watch media assets (such as videos) on display devices, they often perform scaling operations on the display of media assets (i.e., media asset images and display windows) according to their own needs, such as switching from full-screen playback mode to picture-in-picture (PIP) or floating window mode.

[0003] Currently, during the scaling of media assets and display windows, the rendering time of the display window is shorter than that of the media assets, leading to a synchronization discrepancy during presentation. For example, when switching from a small window preview to full-screen playback, the rendering progress of the media assets lags behind the resizing progress of the display window, causing the display window to enlarge before the media assets, resulting in a filled area or border (commonly a black area or black border) that matches the background color. Similarly, when switching from full-screen playback to a small window preview, the rendering progress of the media assets lags behind the resizing progress of the display window, causing the display window to shrink before the media assets, resulting in the media assets being cropped around the edges and an incomplete playback image with cropped edges. Summary of the Invention

[0004] This application provides a display method and display device that can reduce the presentation deviation between media assets and display windows during scaling, achieve synchronous display of media assets and display windows, and improve the user's visual experience.

[0005] In a first aspect, a display device is provided, including a display and a controller, the display being configured to present media assets; the controller being configured to: receive a first operation, the first operation being used to instruct scaling adjustments to the display screen and display window of the media assets; in response to the first operation, perform scaling processing on the display screen of the media assets, and initiate a delay wait for a duration of a target duration, the target duration being used to indicate the duration corresponding to the scaling process of the display screen of the media assets; when the delay wait reaches the target duration, perform scaling processing on the display window of the media assets; and present a target screen and a target window of the media assets, the target screen being the display screen of the media assets after scaling, and the target window being the display window of the media assets after scaling.

[0006] Scaling the displayed media assets and initiating a target duration delay that matches the scaling duration are two parallel asynchronous processing flows. They start independently and in parallel, without waiting for one flow to complete before proceeding to the other.

[0007] This application accurately controls the duration of the delay by starting a delay with a target duration, thus providing a reliable time reference for subsequent window scaling operations.

[0008] This application allows for processing time to be reserved for media asset display screens that take a long time to process, and enables faster display windows to process them. The scaling process is started only after the scaling processing of the media asset display screen is expected to be completed or nearly completed, ensuring that the media asset display screen and the display window present the adjusted new size synchronously, thus improving the user's visual experience when performing scaling operations on media assets.

[0009] In one possible implementation, the controller is configured with a first interface for issuing instructions to perform scaling adjustments on the display screen of the media asset. The controller, in response to the first operation, performs scaling processing on the display screen of the media asset and is configured to: determine target size information in response to the first operation, the target size information being used to indicate the display size of the media asset after scaling; determine a second interface, the second interface being an interface formed by adding the target size information to the first interface; call the second interface and scale the display screen of the media asset based on the target size information to obtain the target screen of the media asset.

[0010] The target size information includes the layout parameters of the scaled media asset display area within the overall display area of ​​the monitor. The layout parameters include the coordinates of the first vertex of the media asset display area and the side length of the media asset display area.

[0011] The second interface is a derived interface obtained by expanding the fields of the first interface for the first operation. The second interface and the first interface are functional extensions of the same interface.

[0012] The second interface / first interface is a general term for a functional interface. This interface can directly interact with the hardware layer of the display device, reducing intermediate links in the display device system. In the scaling display of media assets, the position and size of the display screen can be directly specified through the target size information, reducing the delay of layout calculation in the system.

[0013] In one possible implementation, the controller includes a display module for scaling the display screen of the media asset; the controller executes a call to a second interface and scales the display screen of the media asset based on the target size information to obtain the target screen of the media asset, and is configured to: call the second interface and send the target size information to the display module through the second interface; the display module scales the display screen of the media asset based on the target size information to obtain the target screen of the media asset.

[0014] The display module can skip layout calculations in the rendering process (no typesetting required) and simplify compositing. It can directly scale the media assets based on the target size information, improving the response speed and frame rate of media asset scaling.

[0015] The display module can directly scale the media asset display screen according to the position and size of the display screen specified by the target size information, reducing the delay of layout calculation in the system, thereby reducing the intermediate links in the display device system and improving the scaling processing efficiency of the media asset display screen.

[0016] In one possible implementation, the controller is configured with a third interface for determining the target duration; before performing the first receiving operation, the controller is configured to: call the third interface and obtain its return value; and determine the target duration based on the return value of the third interface.

[0017] The third interface is used to determine the target duration. The third interface is also a general term for a type of functional interface. This interface is used to obtain the duration of the media asset's display scaling, accurately control the delay time, and provide a reliable time reference for subsequent display window scaling operations.

[0018] In one possible implementation, the controller calls a third interface and obtains the return value of the third interface, which is configured to: obtain a reference delay duration based on the configuration information of the media asset; determine the first actual delay duration of the media asset; if the relative error between the first actual delay duration and the reference delay duration is less than or equal to a preset first threshold, then the first actual delay duration is determined as the return value of the third interface.

[0019] In one possible implementation, the controller calls a third interface and obtains its return value, which is configured to: obtain a reference delay duration based on the configuration information of the media asset; determine the first actual delay duration of the media asset; if the relative error between the first actual delay duration and the reference delay duration is greater than a preset first threshold, determine the second actual delay duration of the media asset; and if the relative error between the second actual delay duration and the reference delay duration is less than or equal to a preset second threshold, determine the reference delay duration as the return value of the third interface.

[0020] This application combines the comparison results of the reference delay time and the actual delay time to comprehensively determine the target time, which can realize the verification and confirmation of the target time, ensure the accuracy and rationality of the target time, reduce the deviation caused by relying solely on the preset reference delay time, and improve the reliability and adaptability of the target time.

[0021] In one possible implementation, the controller obtains a reference delay duration based on the media asset's configuration information, and is configured to: obtain the reference delay duration corresponding to the media asset from a preset mapping relationship between media asset configuration and scaling delay value according to the media asset's configuration information.

[0022] The configuration information of media assets may include key parameters such as resolution, encoding / decoding format, compatible hardware type, and system load.

[0023] The display device obtains the reference latency for the media asset from a preset mapping relationship between media asset configuration and scaling latency value, based on the media asset configuration information. This preset mapping relationship between media asset configuration and scaling latency value can be stored in a database on the display device or on a server wirelessly connected to the display device. The preset reference latency provides a basis and analytical standard for determining the target latency.

[0024] In one possible implementation, when the media asset is a target video, the controller determines the first actual delay duration of the media asset by: determining the basic delay duration, hardware processing delay duration, software processing delay duration, encoding delay duration, resolution adaptation delay duration, and network transmission delay duration of the target video during scaling; wherein, the basic delay duration is determined by the frame rate and buffer data of the target video, and the first actual delay duration is determined based on the basic delay duration, hardware processing delay duration, software processing delay duration, encoding delay duration, resolution adaptation delay duration, and network transmission delay duration.

[0025] The basic latency duration is the main factor in the first actual latency duration, and it provides more comprehensive coverage through core aspects such as hardware and software processing, encoding adaptation, and network transmission, ensuring the integrity and accuracy of the first actual latency duration.

[0026] In a second aspect, a display method is provided, applied to any of the display devices in the first aspect, comprising: receiving a first operation, the first operation being used to instruct scaling adjustments to be performed on the display screen and display window of the media asset; responding to the first operation, performing scaling processing on the display screen of the media asset, and initiating a delay waiting for a preset target duration; when the delay waiting time reaches the target duration, performing scaling processing on the display window of the media asset; and displaying a target screen and a target window of the media asset, the target screen being the display screen of the media asset after scaling, and the target window being the display window of the media asset after scaling.

[0027] Thirdly, a display device is provided, including a unit for performing any of the methods in the second aspect. This device may be a terminal device or a chip within a terminal device.

[0028] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a display device, causes the display device to perform any of the display methods described in the second aspect.

[0029] Fifthly, a computer program product is provided, comprising: a computer program that, when run by a display device, causes the display device to perform any of the display methods described in the second aspect.

[0030] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0031] Figure 1 The flowchart of the first application video playback scaling display is shown; Figure 2 A schematic diagram of the video rendering thread in the Video layer is shown; Figure 3 An example diagram of a video scaling switching scenario is shown; Figure 4 The diagram illustrates an operational scenario between a display device and a control device according to some embodiments of this application; Figure 5 A structural block diagram of the display device in an embodiment of this application is shown; Figure 6 A system block diagram of the display device in an embodiment of this application is shown; Figure 7 A flowchart of a display method provided in an embodiment of this application is shown; Figure 8 An example diagram showing the scaled display position and size of media assets provided in this application embodiment is shown; Figure 9 This document illustrates a flowchart of a first application video playback scaling display provided in an embodiment of this application; Figure 10 A flowchart illustrating the determination of the target duration in the display method provided in an embodiment of this application is shown; Figure 11 A timing flowchart of the display method provided in an embodiment of this application is shown; Figure 12 An example diagram of a video scaling and switching scenario provided in an embodiment of this application is shown; Figure 13 A timing comparison diagram of the display method and related solutions provided in the embodiments of this application is shown; Figure 14 A schematic diagram of a display device provided in one embodiment of this application is shown. Detailed Implementation

[0032] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0033] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0034] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. Furthermore, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0035] Before providing a detailed explanation of the display method and display device provided in the embodiments of this application, the application scenarios and related technologies of the display method and display device will be described first.

[0036] As the intelligence level of display devices continues to improve, users often scale the display size of media assets (i.e., the media asset image and the display window) according to their needs when watching media assets (such as videos) on display devices. For example, they can switch from full-screen playback mode to picture-in-picture or floating window mode, or they can use split-screen operation to display the video interface and other application interfaces simultaneously.

[0037] Media assets encompass various multimedia resources such as videos, music, and images. During the scaling of media asset images and display windows, the rendering time of the display window is shorter than that of the media asset images, leading to synchronization discrepancies during presentation. These synchronization discrepancies are particularly noticeable in video scaling scenarios. Taking video as an example, its display process is essentially a collaborative presentation process between the video image (i.e., the video display screen) and the display window. This process is executed by two independent rendering layers, which are then merged and output.

[0038] The video image is decoded and rendered through the Video Layer, while the display window is defined by the OSD / UI Layer and carries interactive elements. Finally, the system compositor overlays the two types of layers to form the complete video display interface seen by the user.

[0039] The Video layer is the layer that loads and displays video footage. It receives the decoded video data and can perform pre-processing such as scaling, rotation, and color adjustment using a dedicated hardware decoding module or a separate media application. The rendering process of this layer does not occupy the system's main thread; it follows the video's playback speed and synchronizes with the screen refresh rate to ensure smooth video playback, clear image quality, and reduce stuttering and screen tearing caused by interference from other system tasks.

[0040] The OSD / UI layer is the user interface rendering layer that carries the attributes and interactive logic of the display window. It determines the position, size, and hierarchy of the display window on the screen, as well as its appearance features such as rounded corners, borders, and transparency. It can also carry interactive buttons such as progress bars and volume controls, and overlay information such as subtitles and watermarks. The OSD / UI layer is driven by the application's UI thread, uses vector graphics technology for rendering, and maintains the same refresh rate as the system interface, enabling it to quickly respond to user operations such as adjusting window size and switching between full-screen and picture-in-picture modes.

[0041] Figure 1 The flowchart of the first application video playback scaling display is shown, such as... Figure 1 As shown, when a user performs a zoom operation on the video (i.e., the video played by the first application) (such as clicking the full-screen / picture-in-picture button or using a zoom gesture), the display device responds to this zoom operation by synchronously triggering two threads within the execution module through the event management module: one is the application UI thread in the OSD / UI layer, and the other is the video rendering thread in the Video layer, thus forming two parallel computation tasks. The OSD / UI layer is used to recalculate the new size and layout parameters of the display window, while the Video layer is used to synchronously derive the video frame scaling matrix and complete the scaling of video frames to adapt to the new size.

[0042] Taking Android display devices as an example, the user's zoom operation is first captured by the Android system's input management service (InputManagerService). For example, it triggers the OnClickListener event of the application's UI control (such as the full-screen button), and the event is passed to the application's main thread (i.e., the application UI thread in the OSD / UI layer) through the Android event dispatch mechanism. At this time, the application UI thread will first receive the zoom event, and encapsulate the zoom instruction (such as target coordinates / size) through the Android system's event management module (such as the EventBus / Handler message mechanism, or a custom event management component), while triggering the processing of the Video layer.

[0043] The Android system uses a thread model that synchronizes the main thread (UI thread) and child threads. The video rendering thread in the Video layer is usually a separate child thread created by the application. The event management module achieves synchronous triggering of the two threads in the following way: 1. Triggering of OSD / UI layer application UI thread: After receiving the scaling instruction, the application UI thread recalculates the new size and layout parameters (such as LayoutParams) of the display window, and updates the window container of the OSD / UI layer through Android's View system (such as the layout update of SurfaceView / TextureView).

[0044] 2. Synchronous triggering of the video rendering thread in the video layer: The event management module sends the scaling command synchronously to the video rendering thread in the video layer through Android's HandlerThread (a child thread with a message queue) or AIDL cross-thread communication; the triggering timing is synchronized with the timing of the UI thread's window layout calculation: The event management module ensures that the two threads receive the scaling command and start processing almost simultaneously through the CountDownLatch (countdown lock) or Handler's synchronous message barrier mechanism, forming a parallel computing task.

[0045] Figure 2 A schematic diagram of the video rendering thread in the Video layer is shown, as follows: Figure 2 As shown, the video rendering process of the Video layer includes video capture, encoding, decoding, and rendering. The total latency of this process is the difference between the rendering completion time (Timestamp2) and the capture time (Timestamp1).

[0046] It's important to note that video scaling relies on the decoded raw frame data, which is first written to the decoding buffer. The system doesn't support directly processing the real-time data stream from the decoded output; it must wait for each frame to be completely written to the decoding buffer before reading the data and initiating scaling calculations. The decoding buffer acts as an intermediary data container connecting video decoding, scaling, and rendering. The writing, reading, format conversion, and synchronization operations within it inevitably incur time overhead that cannot be completely eliminated; this overhead will be included in the total latency mentioned above.

[0047] Decoding buffers are typically buffers in system memory (such as host memory) or graphics card memory (graphics processing unit (GPU) memory).

[0048] It should be noted that the rendering process includes pre-rendering processing, layout calculation, and compositing. Pre-rendering processing optimizes and adapts the decoded raw frame data (such as format conversion and resolution scaling). Layout calculation determines the display position, size, and layer of video frames on the screen for scenarios such as picture-in-picture and split-screen. Compositing merges these different layers into a complete image when there are multiple display elements (such as video frames, UI controls, subtitles, and status bars).

[0049] Therefore, the rendering time of the display window is significantly shorter than that of the video frame (for example, the rendering time of the display window is 50ms, while the rendering time of the video frame is 150ms). After obtaining the synchronization command (including the same display coordinates), the two cannot be presented synchronously, which can lead to the following adaptation problems: When switching from small window preview to full-screen playback, because the rendering progress of the video frame lags behind the resizing progress of the display window, the size of the display window will be enlarged before the size of the video frame, resulting in a fill area or border that matches the background color (commonly a black area or black border); Similarly, when switching from full-screen playback to small window preview, the rendering progress of the video frame lags behind the resizing progress of the display window, and the size of the display window will be reduced before the size of the video frame, resulting in the video frame being cropped around the edges, resulting in an incomplete cropped playback screen.

[0050] For example, after receiving a scaling operation, the display device simultaneously notifies the VIDEO layer to perform scaling of the video image and the OSD / UI layer to perform scaling of the display window. The image is rendered only after both have completed scaling. However, during this waiting period, the display window will complete scaling first, resulting in problems such as black borders. Figure 3 An example diagram of a video scaling switching scenario is shown, such as... Figure 3As shown in (a), after the user clicks the full-screen button, the video of episode 1 of TV series A switches from a small window preview to full-screen playback mode. The changes in the video frame and display window during this process are as follows: When the display window has completed scaling and adaptation, but the video frame has not yet completed scaling, as shown... Figure 3 As shown in (b), the size of the display window 21 is larger than the size of the video screen 11, and a fill area 31 will appear around the video screen 11. After both the display window and the video screen have been scaled and adapted, as shown in (b), Figure 3 As shown in (c), the display window 22 and the video screen 12 are perfectly matched, achieving normal full-screen display.

[0051] To this end, this application provides a display method and a display device. The display method, after receiving a scaling adjustment operation, first performs scaling processing on the display screen of the media asset, and simultaneously starts a target duration delay waiting that matches the screen scaling duration. After the delay time reaches the target, the display window size is adjusted synchronously, and finally the scaled screen and window are presented, realizing the synchronous display of the media asset screen and the display window, and improving the user's visual experience when performing scaling operations on the media asset.

[0052] The display device provided in this application can have various implementation forms, such as a television, a smart television, a laser projection device, an electronic whiteboard, an electronic desktop, a mobile phone, a tablet computer, an in-vehicle device, etc.

[0053] Figure 4 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 4 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0054] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0055] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0056] In some embodiments, the display device 200 may receive instructions not through the aforementioned smart device 300 or control device 100, but through touch or gestures.

[0057] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0058] Display device 200 also communicates with server 400. Display device 200 can communicate via local area network (LAN), wireless local area network (WLAN), and other networks. Server 400 can provide various content and interactive features to display device 200. Server 400 can be a cluster or multiple clusters, and may include one or more types of servers.

[0059] Figure 5 A structural block diagram of the display device in an embodiment of this application is shown, such as... Figure 5 As shown, the display device 200 includes at least one of the following: a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0060] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0061] The display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0062] The display 260 can be an LCD display, an OLED display, or a projection display, and can also be a projection device and a projection screen.

[0063] It should be noted that the embodiments of this application are applied to scenarios where a display is configured to present media assets. These media assets can be multimedia resources such as videos, music, and images.

[0064] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0065] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote control).

[0066] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds. The external device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0067] The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0068] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving an instruction to select a UI object to display on the monitor 260, the controller 250 can perform operations related to the object selected by the user instruction.

[0069] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processor, random access memory (RAM), read-only memory (ROM), a first to an nth interface for input / output, a communication bus, etc.

[0070] Users can input user commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, users can input user commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0071] A user interface (GUI) is the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form that the user can accept. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0072] Figure 6 A system block diagram of the display device in an embodiment of this application is shown, such as Figure 6 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the application layer), the Application Framework layer (referred to as the framework layer), the Android runtime and system library layer (referred to as the system runtime library layer), and the kernel layer.

[0073] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the applications in the application layer are not limited to the examples above. For instance, the application layer may include a first application, which is a media asset application used to play multimedia resources such as videos, music, and / or images.

[0074] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0075] like Figure 6As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0076] In some embodiments, the window manager is used to manage all window programs, such as obtaining the screen size, determining whether there is a status bar, and controlling changes to the display window (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0077] The system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to achieve.

[0078] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 6 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0079] For ease of description, the framework layer, system runtime library layer and kernel layer in this application embodiment are collectively referred to as the system bottom layer.

[0080] In some embodiments, a hardware layer may also be added to render the scaled-down rendering effect.

[0081] To facilitate a further understanding of the technical solutions in some embodiments of this application, the technical solutions of the display device and display control method, and how these solutions solve the aforementioned technical problems, are described in detail below with reference to specific embodiments and accompanying drawings. The embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0082] Figure 7A flowchart of a display method provided in an embodiment of this application is shown, which is applied to the above. Figure 5 , Figure 6 The display device shown is a smart TV, for example. Figure 7 As shown, the display method includes the following steps S510~S540: S510. When the display device is displaying media materials, the user's first operation is received. The first operation is used to instruct the display screen and display window of the media materials to be scaled and adjusted.

[0083] For example, the first operation includes interactive operations such as clicking and gestures. The first operation can trigger the display device to scale and adjust the size of the media asset display (including the display screen and the display window). The embodiments of this application do not limit the type of the first operation.

[0084] For example, switching from picture-in-picture or floating window mode to full-screen playback mode, switching from full-screen playback mode to picture-in-picture or floating window mode, and switching from full-screen playback mode to split-screen display mode, etc.

[0085] S520, in response to the first operation, the display device performs scaling processing on the display screen of the media asset, and initiates a delay wait for a target duration, the target duration being used to indicate the duration corresponding to the scaling process of the display screen of the media asset.

[0086] In this embodiment of the application, the display device responds to the first operation by performing scaling processing on the display screen of the media asset, and simultaneously initiates a target duration delay waiting that matches the scaling duration of the display screen.

[0087] It's important to note that scaling the displayed media assets and the target duration delay matching the startup and scaling time are two parallel asynchronous processing flows. They start independently and in parallel, without waiting for one flow to complete before proceeding to the other, thus shortening the overall execution cycle. Given the long rendering time and simple timing delays of VIDEO layer media assets, the asynchronous mode allows the two processes to proceed independently without blocking each other's startup and execution. This asynchronous approach ensures sufficient time for scaling the displayed images and allows timers to operate independently according to preset logic, preventing the time spent in a single step from impacting overall process efficiency.

[0088] In this process, the display device performs scaling processing on the display screen of the media asset, so that the scaled display screen of the media asset (i.e., the target screen) adapts to the display size of the media asset corresponding to the first operation.

[0089] The target duration indicates the time required for the scaling process of the media asset's display. This target duration is determined by the display device based on the current system environment (such as hardware performance, software stack, video format, resolution, etc.) to complete one scaling process. For example, the target duration can be determined by directly retrieving data from the database maintained within the display device system, calculated using a preset algorithm, or determined by combining preset reference data with the results of actual calculations.

[0090] This application embodiment accurately controls the duration of the delay by starting a delay with a target duration, thus providing a reliable time reference for subsequent window scaling operations.

[0091] S530: When the delay duration reaches the target duration, the display device performs scaling processing on the display window of the media asset.

[0092] When the delay time reaches the target time, the scaling process of the media asset display screen is expected to be completed or nearly completed. Then, the scaling process of the media asset display window is restarted so that the scaled display window (i.e. the target window) adapts to the media asset display size corresponding to the first operation.

[0093] In some embodiments, scaling processing is performed on the display window of the media asset. The application layer can notify the OSD / UI layer to perform the scaling operation so that the scaled display window of the media asset adapts to the new display size.

[0094] S540, The display device presents the target screen and target window of the media asset on the display. The target screen is the display screen after the media asset has been scaled up, and the target window is the display window after the media asset has been scaled up.

[0095] It should be noted that related technologies involve pre-processing steps such as texture uploading and rendering before compositing the display screen and window using the GPU. This application's embodiment eliminates the need for texture uploading and rendering, directly interacting with the GPU, thus reducing overhead.

[0096] In this embodiment, if there is a need for layer overlay, the layers are adjusted based on the target size information and mixed with the video layer to obtain the target image of the media asset, and then the target image of the media asset is presented; if there is no need for layer overlay, it can be presented directly.

[0097] The display method provided in this application embodiment involves a display device performing scaling processing on the media asset display screen after responding to a first operation. Simultaneously, a target duration delay matching the scaling time of the display screen is independently initiated. Once the delay reaches the target duration, the display window is then scaled. This method reserves processing time for media asset display screens with longer processing times, allowing for faster-processing display windows. The scaling process is initiated only after the scaling processing of the media asset display screen is expected to be completed or nearly completed, ensuring that the media asset display screen and the display window synchronously present the adjusted new size, thus improving the user's visual experience when scaling media assets.

[0098] As described in the media asset display solution above, this section provides a detailed explanation of the coordination between the media asset display screen and the scaling process of the display window. To ensure sufficient processing time is allocated for media asset displays that are time-consuming, the display device controller is equipped with a first interface. This first interface is used to issue scaling adjustment commands for the media asset display screen. The following section provides a detailed explanation of this first interface and its related execution logic; other details can be found in the previous section and will not be repeated here.

[0099] In step S520, in response to the first operation, the display device performs scaling processing on the display screen of the media asset, including the following steps (1) to (3): (1) In response to the first operation, the display device determines the target size information.

[0100] The first operation is used to trigger the display device to scale and adjust the display size of the media asset. The adjustment objects include the display screen and display window of the media asset. The first operation carries the display position and size parameters of the scaled media asset, i.e., the target size information.

[0101] Specifically, the target size information includes the layout parameters of the scaled media asset display area within the overall display area of ​​the monitor. The layout parameters include the coordinates of the first vertex of the media asset display area and the side length of the media asset display area.

[0102] Figure 8 An example diagram showing the display position and size of the scaled media assets provided in this application embodiment is illustrated. The diagram illustrates how both the overall display area of ​​the monitor and the media asset display area are rectangular, and how the media asset display area is defined by configuring a unified coordinate system for the overall display area of ​​the monitor.

[0103] like Figure 8As shown, the overall display area 41 of the monitor has its origin (0,0) in the coordinate system with the top left corner of the screen. The maximum width of the overall display area is screenwidth, and the maximum height is screenheight. After the media asset display size is switched to picture-in-picture or floating window mode, the corresponding new size is the media asset display area 42. The layout parameters of the media asset display area 42 can be determined based on the coordinate system of the overall display area 41. These layout parameters include the coordinates (x,y) of the first vertex of the top left corner of the media asset display area 42, as well as the side length of the media asset display area 42, i.e., width and height.

[0104] Where x is a non-negative number, and x + width ≤ screenwidth; y is a non-negative number, and y + height ≤ screenheight; width is a positive number, and height is a positive number.

[0105] (2) Determine the second interface, which is the interface formed by adding target size information on the basis of the first interface.

[0106] It should be noted that the second interface is a derived interface obtained by expanding the fields of the first interface for the first operation. The second interface and the first interface are functional extensions of the same interface.

[0107] It should also be noted that the second interface / first interface is a general term for a functional interface. This interface can directly interact with the hardware layer of the display device, reducing intermediate links in the display device system. In the scaling display of media assets, the position and size of the display screen can be directly specified through the target size information, reducing the delay of layout calculation in the system.

[0108] The second interface is for issuing scaling adjustment instructions for the display screen used for media assets. These instructions contain information about the new size of the display screen and display window after scaling.

[0109] For example, the second interface is a function interface, such as Bool setScreenOutputDispRect(int x, int y, unsigned int width, unsigned int height); where the target size information is x, y, width, and height.

[0110] (3) The display device calls the second interface and scales the display screen of the media asset based on the target size information to obtain the target screen of the media asset.

[0111] The display device can directly obtain the target size information by calling the second interface; then, based on the coordinates (x, y) of the first vertex and the width and height of the side in the target size information, the display screen of the media asset can be scaled to generate the target screen of the media asset.

[0112] This application embodiment introduces a first interface / second interface to reserve sufficient processing time for media asset display screens that take a long time to process.

[0113] As explained above, the main focus is on how the first / second interface provides sufficient processing time for displaying media assets that require significant processing time. The display device calls the second interface to send target size information to the display module, thereby scaling the media asset display. This will be explained in more detail below; other details can be found in the previous description and will not be repeated here.

[0114] It should be noted that the display module can be implemented in hardware, software, or a combination of both.

[0115] It should also be noted that this display module can skip layout calculations in the rendering process (no typesetting required) and simplify compositing. It can directly scale the media assets using the target size information, thereby improving the response speed and frame rate of media asset scaling.

[0116] Specifically, the media asset display screen can be scaled as the main layer. This scaling can be performed independently by the display module without triggering layout reflow or waiting for the main thread. The scaling and output of this data do not involve the full merging of other layers, reducing data copying and waiting time.

[0117] In some embodiments, the display device calls the second interface to interact directly with the hardware layer, and then uses the corresponding hardware (i.e., the display module) to scale the display of the media asset based on the target size information. Figure 9 The flowchart illustrating the first application video playback scaling display provided in this embodiment is shown, as follows: Figure 9 As shown, the controller has a built-in display module, which is a hardware component at the hardware layer. Its function is to perform scaling processing on the display screen of the media asset. Specifically, the controller calls the second interface and sends the target size information to the display module through the second interface; the display module scales the display screen of the media asset according to the target size information, and finally generates the target screen of the media asset.

[0118] The display method provided in this application embodiment includes a display module in the display device. This module can directly scale the media asset display screen according to the position and size of the display screen specified by the target size information, thereby reducing the delay of layout calculation in the system, reducing intermediate links in the display device system, and improving the scaling processing efficiency of the media asset display screen.

[0119] As described above regarding the media asset display scheme, this section primarily explains how the display device responds to the first operation by performing scaling processing on the displayed media asset. In execution step S520, in response to the first operation, the display device also initiates a parallel delay wait for a target duration, which indicates the duration corresponding to the scaling process of the displayed media asset. The following section provides further explanation on how the target duration is determined.

[0120] The controller in the display device is configured with a third interface, which is used to determine the target duration. The third interface is a newly added functional interface in the display device, and the specific form of the interface is not limited in the embodiments of this application.

[0121] Before the controller receives the first operation in step S510, the display device calls the third interface to determine the target duration. Specifically, it can also be configured as follows: the display device calls the third interface to obtain the return value of the third interface; and determines the target duration based on the return value of the third interface. The return value of the third interface can be the target duration directly, or it can be converted into the target duration.

[0122] It should be noted that the stage before receiving the first operation in step S510 may include the following situations: first, the media asset playback initialization stage; second, any time after the media asset playback initialization is completed and before the first operation is received.

[0123] The third interface is also a general term for a type of functional interface. This interface is used to obtain the duration of the display screen scaling of media assets, accurately control the delay duration, and provide a reliable time reference for subsequent scaling operations of the display window. For example, the third interface can also be a function interface, such as int GetVideoDelay().

[0124] For example, when the return value of the GetVideoDelay() function is greater than or equal to 0, the return value is used to indicate the total delay (VideoDelay) from acquisition to display of the data corresponding to the current media asset, in milliseconds (ms). For example, if the return value is 50, it means that the target duration is 50ms.

[0125] It should be noted that, in order to improve the redundancy of the method, when the return value of the GetVideoDelay() function is -1, this return value is used to indicate that the target duration acquisition failed. The corresponding reasons may be that the media asset has not been started, the hardware module is abnormal, or there is no effective data stream. Further solutions can be added to improve the cause analysis of this scenario. This application embodiment does not limit this.

[0126] It should also be noted that when the GetVideoDelay() function returns -1 (i.e., the display device cannot obtain the target duration), the following solutions can be adopted: First, follow the solutions in the related technologies mentioned above; second, configure a fixed preset duration value for the target duration; third, add a fixed-duration delay waiting logic in the application UI thread of the OSD / UI layer to reduce the timing deviation between the display screen and the display window during the presentation process.

[0127] It should be noted that the target duration can be determined by directly retrieving data from the database maintained within the display device system, or by calculating it using a preset algorithm, or by combining the results of comparing preset reference data with actual calculated data. The following section will provide a detailed explanation of how to determine the target duration by combining the results of comparing preset reference data with actual calculated data.

[0128] Figure 10 A flowchart illustrating the determination of the target duration in the display method provided in this application is shown, as follows: Figure 10 As shown, the process of the display device calling the third interface to determine the target duration (or obtaining the return value of the third interface) includes the following steps S610~S640: S610: The display device obtains the reference latency based on the configuration information of the media assets.

[0129] The configuration information for media assets may include key parameters such as resolution, encoding / decoding format, compatible hardware type, and system load. It should be noted that the configuration information may differ between different media assets; for example, the resolution parameters for high-definition video and standard-definition video are drastically different. Alternatively, they may be completely identical; for example, media assets of the same series with the same encoding format and resolution can maintain consistent configuration information.

[0130] In some embodiments, the display device obtains the reference latency duration corresponding to the media asset from a preset mapping relationship between media asset configuration and scaling latency value based on the media asset configuration information. The preset mapping relationship between media asset configuration and scaling latency value may be stored in the display device in the form of a database or on a server wirelessly connected to the display device.

[0131] S620, The display device determines the first actual delay duration of the media asset.

[0132] The first actual latency is the time it takes for the media asset to complete scaling processing in the current system environment of the display device. This first actual latency mainly includes the time spent processing the buffer and scaling calculations, and this value can be calculated based on the media asset's frame rate and the number of buffers with buffer latency.

[0133] Different media assets have different latency data during scaling. For example, in a scenario where the media asset is a target video, determining the first actual latency of the media asset (target video) may include the following steps (1) to (2): (1) Determine the basic delay duration, hardware processing delay duration, software processing delay duration, encoding delay duration, resolution adaptation delay duration and network transmission delay duration of the target video during the scaling process; wherein, the basic delay duration is determined by the frame rate and buffer data of the target video.

[0134] The base delay duration (BaseDelay) is calculated based on the frame rate of the target video and the number of buffers corresponding to the video buffer delay. The calculation method is BaseDelay = 1000ms / frame rate of the target video × number of buffers.

[0135] For example, if the target video has a frame rate of 60fps and the number of buffers is 5, then BaseDelay = 1000ms / 60 × 5 = 83ms.

[0136] It should be noted that the basic delay duration is the main factor in the first actual delay duration.

[0137] Hardware processing latency (HardwareDelay) is related to the performance of the CPU and GPU. It is calculated based on performance test data and the current hardware load. The calculation method is: HardwareDelay = ((100-CPUScore)×CPULoad / 100×α+(100-GPUScore)×GPULoad / 100×β), where CPUScore is the CPU performance score (determined by the chip, with a value range of 0-100), CPULoad is the current hardware load of the CPU (determined by the current usage, with a value range of 0-100%), GPUScore is the GPU performance score (determined by the chip, with a value range of 0-100%), GPULoad is the current hardware load of the GPU (determined by the current usage, with a value range of 0-100%), and α and β are coefficients.

[0138] For example, if the CPU and GPU performance are both 80 points, the load is 50%, and α=β=0.5, then HardwareDelay=(100-80)×50 / 100×0.5+(100-80)×50 / 100×0.5=10ms.

[0139] The software processing delay duration (softwareDelay) is determined based on the player corresponding to the target video (such as a self-developed player, system player, or third-party player). For example, the delay is 5ms for the self-developed player, 8ms for the system player, and 8ms for the third-party player (which can be determined by the average of multiple open-source players).

[0140] The encoding delay (FormatDelay) varies depending on the complexity of decoding and scaling across different video encoding formats. It is calculated as FormatDelay = γ × 10ms, where γ is the delay coefficient corresponding to the video encoding format. For example, the delay coefficient for H.264 is 0.5, and for H.265 it is 0.7.

[0141] Resolution Delay is a factor that affects video resolution. Higher resolution videos require more computation for decoding and scaling, resulting in increased latency. For example, at 720p, Resolution Delay is 10ms; at 1080p, it's 13ms; and at 4K, it's 19ms.

[0142] Network transmission delay (NetworkDelay) is 0 during target video playback; when network video starts playing, is dragged to jump / video positioning (seek), it is calculated as NetworkDelay = (100 / Bandwidth) × (1 + PacketLossRate / 100) × 3, where Bandwidth is the network bandwidth (Mbps) and PacketLossRate is the packet loss rate (range 0-100%).

[0143] For example, if the network bandwidth is 10Mbps and the packet loss rate is 10%, then NetworkDelay = (100 / 10) × (1 + 0.1) × 3 = 33ms.

[0144] It should be noted that this network transmission delay duration only applies when the target video is not cached locally.

[0145] (2) Determine the first actual delay based on the basic delay duration, hardware processing delay duration, software processing delay duration, encoding delay duration, resolution adaptation delay duration and network transmission delay duration.

[0146] In some embodiments, the basic latency, hardware processing latency, software processing latency, encoding latency, resolution adaptation latency, and network transmission latency can be summed to obtain the first actual latency.

[0147] In some other embodiments, the first actual delay duration can also be calculated based on the weights of each data point.

[0148] S630. If the relative error between the first actual delay duration and the reference delay duration is less than or equal to a preset first threshold, the display device determines the first actual delay duration as the return value of the third interface.

[0149] If the relative error between the first actual delay duration and the reference delay duration is less than or equal to a preset first threshold, the first actual delay duration can be determined to be within an accurate range. The display device then determines the first actual delay duration as the return value of the third interface, i.e., the target duration. The preset first threshold can be 10%.

[0150] In some embodiments, the first actual delay duration can be stored in a temporary database. After accumulating more than a preset number of times, the average of the delay duration can be calculated and the reference delay duration updated.

[0151] S640. If the relative error between the first actual delay duration and the reference delay duration is greater than a preset first threshold, the display device determines the second actual delay duration of the media data. If the relative error between the second actual delay duration and the reference delay duration is less than or equal to the preset second threshold, the reference delay duration is determined as the return value of the third interface.

[0152] If the relative error between the first actual delay duration and the reference delay duration is greater than a preset first threshold, then the first actual delay duration is determined to have an error and needs to be recalculated. For example, the second actual delay duration of the media asset is determined by recalculating after 10-second intervals, and the relative error between the second actual delay duration and the reference delay duration is analyzed: if the relative error between the second actual delay duration and the reference delay duration is less than or equal to the preset second threshold, then the reference delay duration is determined to be the return value of the third interface, i.e., the target duration; if the second actual delay duration is different in M ​​consecutive calculations (M is a positive integer), then the network and system fluctuations are considered large, and the second actual delay duration is determined to be the return value of the third interface, i.e., the target duration.

[0153] It should be noted that the method for determining the second actual delay duration is the same as that for determining the first actual delay duration. Please refer to the detailed description of the first actual delay duration above, which will not be repeated here.

[0154] This embodiment of the application combines the comparison results of the reference delay time and the actual delay time to comprehensively determine the target time, which can realize the verification and confirmation of the target time, ensure the accuracy and rationality of the target time, reduce the deviation caused by relying solely on the preset reference delay time, and improve the reliability and adaptability of the target time.

[0155] Figure 11 A timing flowchart of the display method provided in an embodiment of this application is shown, as follows: Figure 11 As shown, the display device performs the following steps S701~S711: S701. When the first application starts playing video, the third interface, namely GetVideoDelay(), is called.

[0156] S702. The system analyzes the reference delay duration and the first actual delay duration to determine the target duration.

[0157] S703, Return VideoDelay (i.e., target duration) to the application layer.

[0158] S704, the application layer receives the user's first operation.

[0159] S705, The application layer determines the target size information (x, y, width, and height) based on the first operation.

[0160] S706, the application layer calls the second interface, namely setScreenOutputDispRect(x,y,width,height), and sends it to the display module of the hardware layer.

[0161] S707, The application layer simultaneously starts a delay wait for the target duration (starts a timer).

[0162] S708, The lower layer of the system receives the scaled display screen (i.e., the target screen) returned by the display module.

[0163] S709. When the timeout period reaches the target duration, send a scaling command for the display window to the underlying system.

[0164] S710, the system's underlying layer determines the scaled display window (i.e., the target window).

[0165] S711: The system's underlying layer displays the target screen and target window, and returns the completed display status to the application layer.

[0166] For details on the implementation process of steps S701 to S711, please refer to the detailed description above; it will not be repeated here.

[0167] The display method provided in this application first obtains the delay duration corresponding to the media asset; after receiving the scaling adjustment operation, it first performs scaling processing on the display screen of the media asset, and at the same time starts the target duration delay waiting that matches the screen scaling duration. After the delay time reaches the target, the display window size is adjusted synchronously, and finally the scaled screen and window are presented, realizing the synchronous display of the media asset screen and the display window, and improving the user's visual experience when performing scaling operations on the media asset.

[0168] For example, the display device first obtains the delay duration (i.e., target duration) corresponding to the target video; after receiving the first operation, it calls the second interface to notify the display module to perform video scaling processing according to the target size information, and starts the timer corresponding to the target duration; after the timer ends (the video is expected to have completed scaling or is close to completing scaling), it notifies the OSD / UI layer to perform display window scaling processing, and finally renders the screen. Figure 12 An example diagram of a video scaling and switching scenario provided in an embodiment of this application is shown, such as... Figure 12 As shown in (a), after the user clicks the full-screen button, the video of episode 2 of TV series B switches from a small window preview to full-screen playback mode. During this process, the video frame will first be scaled, and after the display window has also scaled, as shown... Figure 12 As shown in (b), the display window 23 and the video screen 13 are perfectly matched, achieving normal full-screen display.

[0169] Figure 13 A timing comparison diagram of the display method and related solutions provided in the embodiments of this application is shown, as follows: Figure 13 As shown in the relevant solution, after the user clicks to zoom, the Video layer and the OSD / UI layer start to zoom simultaneously. However, the OSD / UI layer completes the zoom in the second time, while the Video layer completes the zoom in the fourth time. During the time interval between the second and fourth times, only the display window has completed zooming, and the display screen is not yet ready, so black borders will appear in the rendering output.

[0170] The timing sequence of the display scheme provided in this application embodiment is as follows: after the user clicks zoom, the display module first starts to display the zoomed screen, and at the same time, the application layer starts a timer that matches the zoom duration of the screen; when the timer expires (the video is expected to have completed or is close to completing the zoom), the OSD / UI layer starts to zoom. After both are completed, the rendering output is completed. There are no black borders in the whole process, and the display is synchronous.

[0171] It should be noted that, Figure 13 Furthermore, in this embodiment of the application, the total time for the display module to scale the display screen is less than the total time for the video layer to scale the display screen. This is because the display module has higher scaling efficiency in the rendering process than the video layer, resulting in shorter processing time.

[0172] It should be understood that the sequence numbers of the processes in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention. The various embodiments described herein can be independent solutions or combinations based on internal logic, and all such solutions fall within the protection scope of this application.

[0173] It should also be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0174] Based on the same concept, as an implementation of the above method, this application provides a display device. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can implement all the contents of the aforementioned method embodiment.

[0175] Figure 14 A schematic diagram of a display device provided in one embodiment of this application is shown. Figure 14 As shown, the display device 800 includes a trigger unit 810, a screen processing unit 820, a window processing unit 830, and a display unit 840. The functions of each unit are as follows.

[0176] The trigger unit 810 is used to receive a first operation, which is used to instruct the display screen and display window of the media asset to perform scaling adjustments.

[0177] The screen processing unit 820 is used to perform scaling processing on the display screen of the media asset in response to the first operation, and to start a delay wait for a preset target duration.

[0178] The window processing unit 830 is used to perform scaling processing on the display window of the media asset when the delay waiting time reaches the target time.

[0179] Display unit 840 is used to display the target screen and target window of the media asset. The target screen is the display screen after the media asset has been scaled up, and the target window is the display window after the media asset has been scaled up.

[0180] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the display methods provided in the above embodiments.

[0181] This application also provides a chip, which includes a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the display methods provided in the above embodiments.

[0182] This application also provides a computer program product, which includes a computer program that, when run by a terminal device, enables the terminal device to implement the display methods provided in the above embodiments.

[0183] It should be understood that the processor mentioned in the embodiments of this application can be a CPU, GPU, image signal processor (ISP), and / or neural network processing unit (NPU), or other general-purpose processors. A general-purpose processor can be a microprocessor or any conventional processor.

[0184] It should also be understood that the memory mentioned in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0185] In the embodiments provided in this application, the division of various frameworks, modules, or units is merely a logical functional division. In actual implementation, there may be other division methods. For example, multiple frameworks, modules, or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0186] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0187] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A display device, characterized in that, include: A display configured to present media materials; The controller is configured to: Receive a first operation, the first operation being used to instruct the display screen and display window of the media asset to be scaled and adjusted; In response to the first operation, scaling processing is performed on the display screen of the media asset, and a delay wait of a target duration is initiated, wherein the target duration is used to indicate the duration corresponding to the scaling process of the display screen of the media asset; When the delay duration reaches the target duration, the display window of the media asset is scaled. The target screen and target window of the media asset are presented. The target screen is the display screen of the media asset after scaling, and the target window is the display window of the media asset after scaling.

2. The display device according to claim 1, characterized in that, The controller is configured with a first interface, which is used to issue instructions to perform scaling adjustments on the display screen of the media asset; the controller is configured to perform scaling processing on the display screen of the media asset in response to the first operation. In response to the first operation, target size information is determined, which is used to indicate the display size of the media asset after scaling; A second interface is determined, which is an interface formed by adding the target size information to the first interface; The second interface is invoked, and the display screen of the media asset is scaled based on the target size information to obtain the target screen of the media asset.

3. The display device according to claim 2, characterized in that, The controller includes a display module, which is used to scale the display screen of the media asset; the controller executes a call to the second interface and scales the display screen of the media asset based on the target size information to obtain the target screen of the media asset, which is configured as follows: The second interface is invoked, and the target size information is sent to the display module through the second interface; The display module scales the display screen of the media asset based on the target size information to obtain the target screen of the media asset.

4. The display device according to claim 2, characterized in that, The target size information includes the layout parameters of the scaled media asset display area within the overall display area of ​​the monitor. The layout parameters include the coordinates of the first vertex of the media asset display area and the side length of the media asset display area.

5. The display device according to any one of claims 1 to 4, characterized in that, The controller is configured with a third interface for determining the target duration; the controller is configured to: before performing the first receiving operation. Call the third interface and obtain its return value; The target duration is determined based on the return value of the third interface.

6. The display device according to claim 5, characterized in that, The controller calls the third interface, and the return value of the third interface is configured as follows: Based on the configuration information of the media assets, obtain the reference delay duration; Determine the first actual delay duration of the media asset; If the relative error between the first actual delay duration and the reference delay duration is less than or equal to a preset first threshold, then the first actual delay duration is determined to be the return value of the third interface.

7. The display device according to claim 5, characterized in that, The controller calls the third interface, and the return value of the third interface is configured as follows: Based on the configuration information of the media assets, obtain the reference delay duration; Determine the first actual delay duration of the media asset; If the relative error between the first actual delay duration and the reference delay duration is greater than a preset first threshold, then the second actual delay duration of the media asset is determined. And if the relative error between the second actual delay duration and the reference delay duration is less than or equal to the preset second threshold, the reference delay duration is determined as the return value of the third interface.

8. The display device according to claim 6 or 7, characterized in that, The controller, based on the configuration information of the media asset, obtains a reference delay duration and is configured as follows: Based on the configuration information of the media asset, the reference delay duration corresponding to the media asset is obtained from the preset mapping relationship between media asset configuration and scaling delay value.

9. The display device according to claim 6 or 7, characterized in that, When the media asset is a target video, the controller determines the first actual delay duration of the media asset, which is configured as follows: The basic latency, hardware processing latency, software processing latency, encoding latency, resolution adaptation latency, and network transmission latency of the target video during the scaling process are determined; wherein, the basic latency is determined by the frame rate and buffer data of the target video. The first actual delay duration is determined based on the basic delay duration, the hardware processing delay duration, the software processing delay duration, the encoding delay duration, the resolution adaptation delay duration, and the network transmission delay duration.

10. A display method, characterized in that, Applied to the display device according to any one of claims 1 to 9, comprising: Receive a first operation, the first operation being used to instruct the display screen and display window of the media asset to be scaled and adjusted; In response to the first operation, scaling processing is performed on the display screen of the media asset, and a delay waiting time of a preset target duration is initiated; When the delay duration reaches the target duration, the display window of the media asset is scaled. The target screen and target window of the media asset are displayed. The target screen is the display screen after the media asset has been scaled up, and the target window is the display window after the media asset has been scaled up.

Images