Display method, electronic device, storage medium, and program product

By monitoring and dynamically adjusting the resource status of the smart TV system in real time, and optimizing resource allocation using independent and shared rendering pipelines, the resource management problem of the smart TV system is solved, improving smoothness and user experience.

WO2026236930A1PCT designated stage Publication Date: 2026-11-19BOE TECHNOLOGY GROUP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/094634
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-13
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Smart TV systems suffer from problems in resource management, such as excessive memory consumption, redundant background programs and resource contention, low display synthesis efficiency, and a lack of dynamic resource allocation mechanisms. These issues lead to stuttering, latency, and excessive power consumption, negatively impacting the user experience.

Method used

By monitoring the resource status data of electronic devices in real time, resource adjustment instructions are dynamically generated according to resource allocation strategies to adjust the display quality and running resources of the target window, including resource expansion and compression. Independent and shared rendering pipelines are adopted, and memory allocation and thread scheduling are optimized to achieve dynamic resource allocation.

Benefits of technology

It improves the smoothness and user experience of smart TV systems, resolves lag and latency issues, optimizes resource utilization, and enhances feature richness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025094634_19112026_PF_FP_ABST
    Figure CN2025094634_19112026_PF_FP_ABST
Patent Text Reader

Abstract

The present disclosure relates to the technical field of display, and in particular relates to the technical field of smart televisions. Provided is a display method. The display method comprises: in response to a multi-window display operation for a display apparatus in an electronic device, determining a target window among a plurality of windows to be displayed, wherein the target window represents a window with which a user is currently interacting; and in response to a resource adjustment instruction for the electronic device, on the basis of the resource adjustment instruction, adjusting at least one of display quality and operating resources for the target window, so as to control a displayed image of the target window on the basis of at least one of the adjusted display quality and the adjusted operating resources, wherein the resource adjustment instruction is dynamically determined on the basis of resource status data and a resource allocation policy of the electronic device. Further provided in the present disclosure are an electronic device, a storage medium, and a program product.
Need to check novelty before this filing date? Find Prior Art

Description

Display methods, electronic devices, storage media, and software products Technical Field

[0001] This disclosure relates to the field of display technology, particularly to the field of smart TV technology, and more specifically, to a display method, electronic device, storage medium, and program product. Background Technology

[0002] With the widespread adoption of smart electronic devices, users have increasingly higher demands for system smoothness and responsiveness. However, some electronic devices are prone to stuttering and latency, impacting user experience. Take smart TV systems as an example: their functionality is becoming increasingly rich, supporting complex application scenarios such as multitasking, high-definition video playback, and intelligent voice interaction. However, these smart electronic device systems suffer from resource management deficiencies, leading to stuttering, latency, and excessive power consumption, resulting in a poor user experience. Summary of the Invention

[0003] In view of the above problems, this disclosure provides a display method, an electronic device, a storage medium, and a program product.

[0004] According to one aspect of this disclosure, a display method is provided, the method comprising:

[0005] In response to a multi-window display operation for a display device in an electronic device, a target window among the multiple windows to be displayed is determined, the target window representing the window currently being interacted with by the user; and

[0006] In response to a resource adjustment command for an electronic device, at least one of the display quality and operating resources of a target window is adjusted according to the resource adjustment command, so as to control the display screen of the target window according to the adjusted display quality and at least one of the adjusted operating resources, wherein the resource adjustment command is dynamically determined based on the resource status data and resource allocation strategy of the electronic device.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0008] One or more processors;

[0009] Memory, used to store one or more computer programs.

[0010] In this process, one or more processors execute one or more computer programs to implement the steps of the above method.

[0011] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program or instructions thereon, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0012] According to another aspect of this disclosure, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0015] Figure 1 is a flowchart of a display method according to an embodiment of the present disclosure;

[0016] Figure 2 is a schematic diagram of a multi-window interface displayed by a display device according to an embodiment of the present disclosure;

[0017] Figure 3 is a schematic diagram of the display screen of the rendering target window according to an embodiment of the present disclosure;

[0018] Figure 4A is a schematic diagram of a rendering target window and a background window according to an embodiment of the present disclosure;

[0019] Figure 4B is a schematic diagram showing multiple views according to an embodiment of the present disclosure;

[0020] Figure 5 is a schematic diagram of the display method after the target window is changed according to an embodiment of the present disclosure;

[0021] Figure 6 is a schematic diagram of generating resource adjustment instructions according to an embodiment of the present disclosure;

[0022] Figure 7 is a schematic diagram of generating resource extension instructions according to an embodiment of the present disclosure;

[0023] Figure 8 is a schematic diagram of the system architecture of the display method according to an embodiment of the present disclosure;

[0024] Figure 9 is a schematic diagram of a display method according to an embodiment of the present disclosure;

[0025] Figure 10 is a schematic diagram of the display method after the focus window is changed according to an embodiment of the present disclosure;

[0026] Figure 11 is a structural block diagram of a display device according to an embodiment of the present disclosure; and

[0027] Figure 12 is a block diagram of an electronic device suitable for implementing a display method according to an embodiment of the present disclosure. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. Based on the described embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. It should be noted that throughout the accompanying drawings, the same elements are represented by the same or similar reference numerals. In the following description, some specific embodiments are used for descriptive purposes only and should not be construed as limiting this disclosure in any way, but are merely examples of embodiments of this disclosure. Conventional structures or configurations will be omitted where they may cause confusion in understanding this disclosure. It should be noted that the shapes and dimensions of the components in the figures do not reflect actual size and proportion, but are only schematic representations of the embodiments of this disclosure.

[0029] Unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure shall have the ordinary meaning as understood by those skilled in the art. The terms "first," "second," and similar words used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components.

[0030] Terminology Explanation:

[0031] The CPU (Central Processing Unit) is the core component of a computer, responsible for executing instructions, processing data, and controlling computer operations.

[0032] A GPU (Graphics Processing Unit) is a processor specifically designed for graphics rendering and parallel computing, and is widely used in fields such as gaming, AI, and scientific computing.

[0033] A SoC (System on Chip) is a hardware unit that integrates core modules such as a CPU, GPU, and memory controller. In this disclosure, the temperature and load of the SoC are key monitoring indicators for dynamic resource scheduling.

[0034] DDR (Double Data Rate) memory is used for storing and transmitting data. In this disclosure, DDR bandwidth utilization is monitored in real time to determine if the system is overloaded and to trigger optimization strategies.

[0035] FPS (Frames Per Second) is a core metric for measuring the smoothness of video playback. A high frame rate (≥60 FPS) results in a better user experience, while a low frame rate may cause stuttering.

[0036] ASTC (Adaptive Scalable Texture Compression) is a technique for texture compression. In this disclosure, it is used to compress texture data for non-focus windows (e.g., ASTC 6x6) to reduce memory footprint.

[0037] Multi-window mode: A functional mode that displays multiple application windows or content views simultaneously on the same display device; it may include split-screen mode, free window mode, etc.

[0038] Focus window: The main window for current user interaction, which can enjoy higher resource priority (such as CPU and GPU computing power).

[0039] Dynamic rendering pipeline refactoring: Adjust the rendering process in real time according to hardware load, such as reducing the resolution of non-focus windows in high-temperature scenarios.

[0040] Memory pool optimization allocation: a technique used to improve memory management efficiency and performance, aiming to solve problems such as inefficiency and memory fragmentation in certain scenarios of conventional memory allocation methods.

[0041] Shader: A programmable graphics rendering unit, a small program that runs on the GPU and is used to control the lighting, color, texture and other visual effects of 3D models.

[0042] Vertex Shader: Used to process vertex data in graphics. It can perform coordinate transformations on vertices, such as transforming vertices from model space to world space, and then to view space and clip space. It can also calculate and transform vertex attributes such as normals and texture coordinates.

[0043] Fragment shaders are used to calculate the final color of each pixel. They can perform lighting calculations, determining the lighting effects on pixels based on the position, intensity, and color of the light source, as well as the material properties of the object; they can also perform texture sampling, extracting color values ​​from texture images and fusing them with other calculation results; furthermore, they can achieve special effects such as shadows, reflections, refractions, and blurs.

[0044] A thread is the basic unit of execution in a computer operating system and the smallest unit of program scheduling.

[0045] With the increasing popularity of smart TVs, users have higher and higher requirements for the system performance and interactive experience of smart TV systems. However, the smart TV systems in the relevant examples have the following problems:

[0046] (1) Excessive memory usage: Smart TV systems typically need to run multiple applications simultaneously (such as video streaming, games, background services, etc.), but due to the lack of an efficient memory management mechanism, the system memory usage rate continues to rise. When available memory is insufficient, the system frequently triggers memory reclamation or forced application termination, which in turn causes problems such as interface lag and application response delays, affecting the user experience.

[0047] (2) Background Program Redundancy and Resource Contention: The smart TV system in the relevant examples allows background applications to run continuously to support fast switching, but lacks a precise background process scheduling strategy. This leads to unnecessary background programs excessively occupying CPU, GPU, and input / output (I / O) resources, and may even cause resource contention. For example, when a user is watching high-definition video, the abnormal activity of background applications may compete for decoding resources, resulting in video playback stuttering or audio-visual desynchronization.

[0048] (3) Inefficient display compositing: At the graphics processing level, smart TVs rely on display compositing engines (such as Surface Flinger) to manage multi-layer compositing. However, the smart TV systems in the relevant examples lack optimization in dynamic frame rate adaptation, layer priority scheduling, and compositing algorithms. Especially in multi-window (such as picture-in-picture mode) or high frame rate scenarios, problems such as compositing delays are prone to occur, further exacerbating display stuttering.

[0049] (4) Lack of dynamic resource allocation mechanism: The system resource allocation in the relevant examples often adopts a static strategy, which is difficult to adjust dynamically according to the real-time load. For example, in high-load scenarios such as games, the system may fail to prioritize the allocation of GPU resources to critical tasks, resulting in a decrease in rendering frame rate; while in standby mode, it fails to release idle resources in time, causing unnecessary energy waste.

[0050] To address the above technical problems, this disclosure provides a display method, comprising: in response to a multi-window display operation on a display device in an electronic device, determining a target window among a plurality of windows to be displayed, the target window representing the window currently being interacted with by the user; and in response to a resource adjustment instruction for the electronic device, adjusting at least one of the display quality and operating resources of the target window according to a resource expansion instruction for the target window in the resource adjustment instruction, so as to control the display screen of the target window according to at least one of the adjusted display quality and adjusted operating resources, wherein the resource adjustment instruction is dynamically determined based on the resource status data and resource allocation strategy of the electronic device. The method provided by this disclosure, by monitoring the resource status data of the electronic device in real time and dynamically generating resource adjustment instructions according to the resource allocation strategy, changes the resources occupied by different windows in real time. This can solve the problems of lag, inconvenient operation, and poor functional scalability existing in smart TV systems in related examples, thereby improving system smoothness, user experience, and functional richness.

[0051] It should be noted that the display method provided in this disclosure can be applied to smart TVs, multi-screen interactive systems, video conferencing systems, monitoring systems, video entertainment systems, audio entertainment systems, cloud gaming terminals, industrial control screens, etc. This disclosure uses a smart TV as an example for detailed explanation.

[0052] Figure 1 is a flowchart of a display method according to an embodiment of the present disclosure.

[0053] As shown in FIG1, the display method 100 according to an embodiment of the present disclosure includes operations S110 to S120.

[0054] In operation S110, in response to a multi-window display operation for a display device in an electronic device, a target window among the multiple windows to be displayed is determined, the target window representing the window currently being interacted with by the user.

[0055] In operation S120, in response to a resource adjustment command for the electronic device, at least one of the display quality and operating resources for the target window is adjusted according to the resource adjustment command, so as to control the display screen of the target window based on at least one of the adjusted display quality and adjusted operating resources. The resource adjustment command is dynamically determined based on the resource status data and resource allocation strategy of the electronic device.

[0056] Electronic devices may include those with a display area larger than a preset size. Examples include smart TVs, large interactive whiteboards, graphics workstation monitors, LED video walls, unfolded folding screen devices, and in-vehicle panoramic displays.

[0057] The preset size can be determined according to actual needs. For example, the preset size can be a 32-inch diagonal size of the display area.

[0058] Electronic devices may include those with video conferencing capabilities. Examples of electronic devices with video conferencing capabilities include smart conference tablets, video conferencing terminals, and smart displays with cameras.

[0059] By monitoring the resource status data of electronic devices in real time and dynamically generating resource adjustment instructions based on resource allocation strategies, the system can change the resources occupied by different windows in real time, maximizing resource savings in multi-window scenarios. Especially when multiple windows are playing videos simultaneously, the system automatically improves the display quality and operating resources of the target window through dynamic resource allocation strategies, thereby helping to improve the smoothness of the target window's display and enhance the user experience.

[0060] Multi-window display operations can be implemented in any way that allows for multi-window display. For example, multi-window display operations can include gesture operations, voice operations, remote control operations, TV menu operations, and mobile application operations. Specifically, remote control operations can include pressing buttons on the remote control labeled "Picture-in-Picture," "PIP," "Split Screen," or similar icons. TV menu operations can include using the "Menu" or "Settings" button on the remote control to access the TV settings interface, and selecting options such as "Display," "Image," "Multi-Window," "Picture-in-Picture," or "Dual Window." Mobile application operations can include installing the corresponding TV brand's application on the phone and selecting split-screen or screen mirroring options within the application.

[0061] After the user selects a multi-view display, the multi-view is displayed on the display device, and the target view among the multi-view is determined.

[0062] Figure 2 is a schematic diagram of a multi-window interface displayed by a display device according to an embodiment of the present disclosure.

[0063] As shown in Figure 2, the display device 200 in this embodiment displays four windows, namely window 210, window 220, window 230 and window 240. The four windows can display different images respectively.

[0064] It should be noted that when the electronic device is a smart TV, the four windows shown in Figure 2 are equivalent to four TVs, that is, one TV is divided into four TVs.

[0065] According to embodiments of this disclosure, at least two of the multiple windows display video feeds. For example, windows 210 and 220 in FIG2 display video feeds; or windows 210, 220, 230, and 240 in FIG2 all display video feeds, such as in a video conferencing scenario.

[0066] The target window can be determined by locating the user's focus. Methods for locating the user's focus can include touch operations, mouse cursor operations, infrared operations, and positioning sensors (such as cameras).

[0067] For example, a method for determining a target window through touch operation may include: when a user directly touches a certain area of ​​the screen, accurately locating the touch point using a touch sensor, and setting the topmost window at the touch point location as the target window.

[0068] For example, a method for determining a target window through mouse cursor operation may include: identifying the window where the cursor is located as the target window. For instance, if the cursor is positioned in window 220 as shown in Figure 2, then window 220 can be identified as the target window.

[0069] For example, the user's focus is determined by using a positioning sensor to track the user's focus location (e.g., where the user is looking). Additionally, the tracked focus location can be used by an electronic device or sensor to identify the user's focus location relative to the electronic device, such as relative to a display device or positioning sensor. For instance, the electronic device can identify a set of user coordinates corresponding to the user's extremities (e.g., eyes, head, hands, elbows, torso, joints, etc.). Furthermore, the electronic device can create one or more image maps (e.g., image maps, depth maps, audio maps, etc.) based on the coordinate set identified by the sensor.

[0070] For example, in a video conferencing scenario, the target window can be determined by voice. For instance, the speaker's location can be determined using a microphone array, and the corresponding video window can be designated as the target window. Alternatively, the target window can be determined by shared content. For example, when a participant begins sharing their screen, the window displaying the shared content can be designated as the target window.

[0071] Resource adjustment instructions can include only resource expansion instructions for the target window, or they can include both resource expansion instructions for the target window and resource compression instructions for the background window. The background window represents the window after removing the target window from a set of multiple windows.

[0072] Display quality can include at least one of image resolution, image transfer rate, and image compression level.

[0073] Operating resources can include the hardware resources required by electronic devices during operation. These may include processor resources, memory resources, storage resources, display resources, network resources, etc.

[0074] In some examples, adjusting at least one of the display quality and runtime resources for the target window may include: increasing the image resolution for the target window. It may also include: increasing the runtime memory for the target window. Alternatively, it may include both increasing the image resolution for the target window and increasing the runtime memory for the target window.

[0075] Resource status data can be the current status data of the hardware resources of an electronic device. For example, resource status data can include SoC temperature, CPU utilization, memory utilization, GPU utilization, and DDR bandwidth utilization.

[0076] Resource allocation strategies can include resolution allocation strategies and memory allocation strategies.

[0077] According to embodiments of this disclosure, an electronic device may include independent rendering pipelines and shared rendering pipelines.

[0078] In some examples, the resource adjustment directives only include resource expansion directives for the target window, which may include the target window resolution. In this case, adjusting at least one of the display quality and runtime resources for the target window according to the resource adjustment directives may include: rendering the display of the target window to the target window resolution using an independent rendering pipeline according to the resource expansion directives.

[0079] In other examples, resource adjustment directives may also include thread priority directives. In this case, adjusting at least one of the display quality and runtime resources for the target window according to the resource adjustment directive may include: adjusting the thread priority for the target window in the rendering thread pool according to the thread priority directive; and rendering the display of the target window to the target window resolution according to the independent rendering pipeline and the adjusted thread.

[0080] An independent rendering pipeline is a dedicated, parallel rendering channel established for specific graphics tasks (such as 3D rendering, UI drawing, and video decoding). It possesses the following characteristics: hardware resource isolation, with exclusive or priority allocation of GPU computing units, video memory bandwidth, and other resources; logical independence, ensuring that processes such as vertex or fragment shaders and texture sampling within the pipeline are not interfered with by other tasks; and dynamic priority, allowing for real-time adjustment of computing power allocation based on scenario requirements (such as video conferencing).

[0081] Thread priority instructions can include a target window identifier and determine the CPU thread scheduling weight and GPU computing unit allocation ratio for the target window. For example, thread priority instructions can determine an independent rendering pipeline. The target window resolution can be larger or smaller than the current resolution of the target window.

[0082] Figure 3 is a schematic diagram of the display screen of the rendering target window according to an embodiment of the present disclosure.

[0083] As shown in Figure 3, the resource adjustment instruction 310 in this embodiment 300 only includes resource expansion instructions, which include a thread priority instruction 320 and a target window resolution 330. Based on the thread priority instruction 320, the rendering pipeline of the target window can be determined to be an independent rendering pipeline 340; then, rendering is performed based on the independent rendering pipeline 340 and the target window resolution 330 to obtain a display screen 350, which is then displayed through the target window 360. At this time, the resolution of the display screen 350 is the target window resolution 330.

[0084] The process involves determining the target window's rendering pipeline as an independent rendering pipeline based on thread priority instructions. Then, rendering is performed according to the independent rendering pipeline and the target window's resolution. The resulting display may include: the rendering pipeline controller sending thread priority instructions to the shader program controller and the thread priority controller; the shader program controller loading the shader resource pool according to the shader preloading strategy; and determining the target window's rendering pipeline as an independent rendering pipeline (e.g., a high-performance pipeline) and the background window's rendering pipeline as a shared rendering pipeline (e.g., a lightweight pipeline) based on the thread priority instructions; and the thread priority controller adjusting the thread priorities in the rendering thread pool in real-time according to the thread priority instructions, increasing the priority of the thread containing the target window; and then using the adjusted thread for rendering to obtain the display. During processing, memory usage for applications within the target window is prioritized, thereby improving screen smoothness.

[0085] By forcing the target window to exclusively use an independent rendering pipeline through thread priority instructions, resource contention can be avoided, ensuring the frame rate stability of the target window (such as the main speaker's screen in a video conference). Furthermore, based on the target window resolution in the resource expansion instructions, the output precision of the rendering pipeline can be directly controlled, achieving the effect of dynamically adjusting the rendering precision of the target window.

[0086] In some examples, the resource adjustment instructions may also include memory expansion instructions. In this case, the method may further include allocating the target window's runtime memory, as contained in the memory expansion instructions, to the target window.

[0087] Memory expansion instructions can include a target window identifier and target window runtime memory. The target window identifier is used to accurately identify the target object; the target window runtime memory is used to specify the target window's runtime memory. This memory value can be determined based on the electronic device's resource operation data and resource allocation strategy. For example, it can be intelligently calculated based on the target window's current task requirements, the remaining system memory, and historical operation data to ensure that the allocated memory meets the requirements without excessively consuming system resources.

[0088] Allocating the target window's runtime memory, as specified in the memory expansion instruction, to the target window can involve the following steps: Upon receiving the memory expansion instruction, the system first checks the memory pool. The memory pool pre-allocates a certain area of ​​free memory. If there is enough free memory in the memory pool to meet the target window's runtime memory requirements, the corresponding memory is directly extracted from the memory pool and quickly allocated to the target window according to the instruction requirements. If the free memory in the memory pool is insufficient, the system will initiate a dynamic memory reclamation mechanism, pausing non-critical processes of some low-priority windows to release their occupied memory resources; simultaneously, it will reclaim memory from applications that are dormant or running in the background and have not been active for a long time. The reclaimed memory is then integrated with the remaining memory in the memory pool, and the system reassesses whether it meets the target window's runtime memory requirements. If it does, the allocation is completed; if it still does not meet the requirements, the system will request additional memory from the operating system, and after obtaining sufficient memory, complete the allocation operation for the target window.

[0089] After allocating memory to the target window, the independent rendering pipeline can also be notified simultaneously. The independent rendering pipeline can then adjust its rendering strategy based on the new memory status, such as speeding up rendering, to make full use of the additional memory and improve the window's display performance.

[0090] By precisely allocating memory according to memory expansion instructions, issues such as stuttering and slow loading caused by insufficient memory in the target window can be effectively resolved. For example, in multi-window applications on smart TVs, when a user is watching a high-definition video window while operating other windows, allocating sufficient memory to the video window ensures smooth, uninterrupted video playback, greatly improving the user's viewing experience. Similarly, in multi-person video conferencing scenarios, allocating sufficient memory to the speaker's window ensures clear video and stable audio transmission, preventing video delays and audio interruptions due to insufficient memory, thus guaranteeing a smooth meeting.

[0091] In some examples, resource adjustment instructions include resource expansion instructions and resource compression instructions. Resource expansion instructions are the same as those described above, used to adjust at least one of the display quality and runtime resources for a target window; for example, rendering the target window's display using a separate rendering pipeline to the target window's resolution. Resource compression instructions are used to adjust at least one of the display quality and runtime resources for a background window; for example, rendering the background window's display using a shared rendering pipeline to a compressed resolution. The background window represents a window among multiple windows after removing the target window.

[0092] In some examples, adjusting at least one of the display quality and runtime resources for the background window may include: compressing the image resolution for the background window. It may also include: reducing the runtime memory for the background window. It may further include: compressing the image resolution for the background window and reducing the runtime memory for the background window.

[0093] According to embodiments of this disclosure, the electronic device includes a shared rendering pipeline, and the resource compression instruction includes a reduced resolution; adjusting at least one of the display quality and operating resources for the background window according to the resource compression instruction includes: rendering the display screen of the background window to a compressed resolution using the shared rendering pipeline according to the resource compression instruction.

[0094] The shared rendering pipeline is a unified rendering resource scheduling system that encompasses the GPU core resource pool, shared memory modules, and rendering instruction queues. It supports parallel processing of multi-window rendering tasks, ensuring that the rendering needs of different windows are met through task priority allocation and time-slice rotation strategies. Simultaneously, it utilizes pipeline state caching technology to cache frequently used rendering states (such as shader programs and texture sampling parameters). When different windows use the same rendering state, they can directly access the cached data, reducing the overhead of repeatedly setting rendering states.

[0095] Resource compression commands can include a background window identifier and a compression resolution. The compression resolution can be dynamically determined based on resource status data and resource allocation strategies. For example, when system memory is severely insufficient, the compression resolution of the background window can be set to a lower level; while when system resources are relatively abundant but performance needs to be optimized, a slightly higher compression resolution can be selected.

[0096] According to resource compression instructions, rendering the background viewport to a compressed resolution using a shared rendering pipeline can include: when the shared rendering pipeline receives a compression instruction for the background viewport, it prioritizes placing the background viewport rendering task in a low-priority queue. During the rendering of the background viewport, the shared rendering pipeline can use image scaling algorithms to sample and scale the original image data according to the compression resolution requirements, quickly converting the image resolution to the compressed resolution while maintaining image quality. For texture data, resolution compression can also be performed simultaneously to reduce texture memory usage.

[0097] During compressed resolution rendering, the shared rendering pipeline can dynamically adjust rendering parameters. For example, it can disable some performance-intensive but visually insignificant effects in the background viewport, such as advanced lighting effects and motion blur; simplify shader calculations and reduce unnecessary pixel calculations. Simultaneously, by leveraging the resource reuse capabilities of the shared rendering pipeline, it can share some rendering resources with other viewports in the same or similar rendering states, further reducing resource consumption.

[0098] By using a shared rendering pipeline to compress the resolution of background windows, the computational load and memory usage of the GPU can be effectively reduced. In scenarios where multiple windows run simultaneously, more system resources can be freed up to ensure the smooth operation of the target windows, thus improving overall system performance. For example, when multiple applications such as video playback and web browsing are running simultaneously on a smart TV, resource compression of the background web browsing window can make video playback smoother and reduce stuttering.

[0099] In some of these examples, the resource compression instruction may also include a bandwidth limiting instruction; the method may also include limiting the bandwidth of the background window to the bandwidth contained in the bandwidth limiting instruction, based on the bandwidth limiting instruction.

[0100] The bandwidth limiting command is used to limit the bandwidth of DDR.

[0101] According to the bandwidth limiting instruction, the bandwidth of the background window can be limited to the bandwidth specified in the instruction. This can include: upon receiving the bandwidth limiting instruction, the memory controller can allocate specific memory access permissions and data transfer channels to the background window according to the bandwidth limit specified in the instruction, thereby limiting its data read and write volume per unit time. Alternatively, data compression technology can be used to compress data in real time before it is written to DDR memory, reducing the bandwidth required for data storage and transmission.

[0102] Figure 4A is a schematic diagram of a rendering target window and a background window according to an embodiment of the present disclosure.

[0103] As shown in Figure 4A, the resource adjustment instruction 410 of this embodiment 400A includes a resource expansion instruction 420 for the target window and a resource compression instruction 430 for the background window 450 and the background window 460. The resource expansion instruction 420 includes a target window 440 resolution 421 and a thread priority instruction 422; the resource compression instruction 430 includes a compressed resolution 431 and a compression instruction 432.

[0104] The rendering process for the target viewport 440 is as follows: The target viewport 440 submits rendering data. The vertex shader 441 performs vertex transformations on the rendering data to obtain vertex transformation data. Then, according to the thread priority instruction 422, the rendering pipeline of the target viewport 440 is determined to be the independent rendering pipeline 442. After that, the vertex transformation data is rendered according to the independent rendering pipeline 442 and the target viewport resolution 421 to obtain the display screen 443, which is then output. The display screen 443 is displayed using the target viewport 440. At this time, the resolution of the display screen 443 is the target viewport resolution 421.

[0105] The rendering process for the background viewport is as follows: This embodiment includes two background viewports, such as background viewport 450 and background viewport 460. First, background viewport 450 and background viewport 460 submit rendering data. Vertex shader 451 performs vertex transformations on the rendering data submitted by background viewport 450 to obtain vertex transformation data, and vertex shader 461 performs vertex transformations on the rendering data submitted by background viewport 460 to obtain vertex transformation data. Then, according to compression instruction 432, the rendering pipeline of background viewport 450 and background viewport 460 is determined to be a shared rendering pipeline 470. Afterwards, the vertex transformation data is rendered according to the shared rendering pipeline 470 and the compression resolution 431 to obtain display screen 452 and display screen 462. At this time, the resolution of display screen 452 and display screen 462 is the compression resolution 431. Then, display screen 452 is displayed using background viewport 450, and display screen 462 is displayed using background viewport 460.

[0106] Figure 4B is a schematic diagram showing multiple views according to an embodiment of the present disclosure.

[0107] As shown in Figure 4B, the display device 470 included in this embodiment 400B includes a focus window 471 and a non-focus window 472. The focus window 471 is rendered using a high-performance shader 480 to output a display screen 481; the non-focus window 472 is rendered using a lightweight shader 490 to output a display screen 491. Then, the display screen 481 is displayed using the focus window 471 and the display screen 491 is displayed using the non-focus window 472, thereby realizing multi-window display.

[0108] By using different shaders for different windows, resource isolation and dynamic resource balancing can be achieved.

[0109] In video conferencing scenarios, multiple windows need to maintain high resolution. In this case, multiple windows can be divided into different levels, and different resource adjustment methods can be used for windows of different levels.

[0110] For example, multiple windows can be divided into high-priority window groups, medium-priority window groups, and low-priority window groups. A high-priority window group can include multiple windows, such as two, and its resource adjustment method can be full pipeline rendering, maintaining the native resolution, such as 4K@60fps. A medium-priority window group can include multiple windows, such as three, and its resource adjustment method can be lightweight pipeline rendering with intelligent downsampling enabled, such as using ASTC 6x6 compression. A low-priority window group can include multiple windows, such as three, and its resource adjustment method can be lightweight pipeline rendering with a forced lock at 1080p@30fps and texture compression enabled.

[0111] According to embodiments of this disclosure, the shared rendering pipeline includes at least one first shared rendering pipeline and at least one second shared rendering pipeline; the background window includes at least one first-level window and at least one second-level window; rendering the display screen of the background window to a compressed resolution using the shared rendering pipeline according to resource compression instructions includes: rendering the display screen of the first-level window to a first compressed resolution using the first shared rendering pipeline according to resource compression instructions.

[0112] According to embodiments of this disclosure, the resource compression instruction may further include a first compression instruction for a first-level window and a second compression instruction for a second-level window; rendering the display screen of the background window to a compressed resolution using a shared rendering pipeline according to the resource compression instruction may include: rendering the display screen of the first-level window to a first compressed resolution using a first shared rendering pipeline according to the first compression instruction; and rendering the display screen of the second-level window to a second compressed resolution using a second shared rendering pipeline according to the second compression instruction, wherein the first compressed resolution and the second compressed resolution are different.

[0113] Dividing the background viewport into first-level (medium priority) and second-level (low priority) viewports, and binding them to the first and second shared rendering pipelines respectively, can prevent low-priority tasks from preempting critical resources. Furthermore, implementing tiered compression based on viewport level allows for differentiated compression rendering of the displayed images from different viewport levels, achieving efficient utilization of system resources.

[0114] According to embodiments of this disclosure, the method further includes: monitoring resource status data for the electronic device in response to a window change operation.

[0115] Changes to the window will trigger changes to the resource status data of the electronic device. Therefore, it is necessary to monitor the resource status data in real time so as to generate resource adjustment instructions in real time based on the resource status data.

[0116] In some examples, the rendering quality of multiple windows is evaluated in real time based on the monitored resource status data, and the evaluation results are obtained. Based on the evaluation results, resource adjustment instructions can be dynamically generated to adjust the image quality and running resources of the changed target window and the changed background window.

[0117] The operation that changes the target window can be any operation that can change the window. This can be a direct interactive operation, a keyboard or keyboard shortcut operation, a gesture or voice operation, etc.

[0118] For example, direct interactive operations can include click or touch operations, drag-and-drop window operations, and double-click activation operations. Specifically, a click or touch operation can include a user clicking or touching a background window on the screen, triggering an operation to change the target window. Similarly, a drag-and-drop window operation can include a user dragging a window to the focal area of ​​the screen (such as the central area), triggering an operation to change the target window.

[0119] For example, keyboard or shortcut operations can include using the arrow keys + confirmation key to change windows. For instance, in a split-screen layout, the keyboard arrow keys can be used to select a window, and the confirmation key can be pressed to switch the target window.

[0120] For example, gestures or voice operations can include multi-finger swipes, voice commands, etc. For instance, multi-finger swipes can include a user swiping horizontally across a touchpad or touchscreen to switch active windows. Similarly, voice commands can be triggered by a user using a voice command (such as "switch to browser window") to change the target window.

[0121] According to embodiments of this disclosure, the target window change operation includes at least one of the following: window size change, target window change, and window quantity change.

[0122] Changing the window size can include increasing or decreasing the size. Changing the target window can include changing the target window from window A to window B. Changing the number of windows can include increasing or decreasing the number of windows.

[0123] Resizing operations can include any operation that can adjust the window size. For example, resizing operations can include mouse operations: dragging the window edge to adjust, double-clicking the window edge to automatically adjust to a preset size, clicking the window's maximize or minimize buttons, etc. They can also include touch screen gesture operations: pinching to zoom the window, dragging the edge to adjust, and specific swipe gestures to trigger adjustments.

[0124] Changing the size of the target window will cause changes to resource status data, and a larger size can easily lead to resource overload. When users change the window size by dragging the edge, maximizing / minimizing, or using touch gestures, the system automatically detects the electronic device's resource status data and dynamically adjusts resource allocation, which helps improve system smoothness and user experience.

[0125] According to embodiments of this disclosure, the method further includes: determining the window to be changed based on the window change operation; and displaying the window to be changed using a preset window display mode.

[0126] Displaying the window to be changed using preset window display methods can include: changing the window background of the window to be changed to a preset color; adding shadows or 3D effects to the window to be changed; highlighting the border of the window to be changed; changing the transparency of the window to be changed; and displaying a preset icon at a preset position on the window to be changed.

[0127] Figure 5 is a schematic diagram of the display method after the target window is changed according to an embodiment of the present disclosure.

[0128] As shown in Figure 5, in this embodiment 500, the display device 520 currently displays a target window 521 and a background window 522. When the user 510 performs an operation to change the target window on the display device 520, for example, the user 510 clicks on the background window 522 on the screen of the display device 520, triggering the operation to change the target window; at this time, in response to this operation, the background window 522 is changed to the changed target window 532, and the target window 521 is changed to the changed background window 531; then, resource status data 540 for the electronic device is obtained, and a resource adjustment instruction 550 is generated based on the resource status data. At this time, the resource adjustment instruction includes a resource extension instruction 551 used to adjust the resources of the changed target window 532. The resource extension instruction may include a thread priority instruction. Adjusting the resources of the changed target window 532 may include, for example, switching the rendering pipeline of the changed target window from a shared rendering pipeline to an independent rendering pipeline, and increasing the resolution and running memory of the changed target window 532. Resource compression instruction 552 is used to adjust the resources of the modified background window 531, for example, to switch the rendering pipeline of the modified background window from an independent rendering pipeline to a shared rendering pipeline, and to compress the resolution and running memory of the modified background window 531.

[0129] Changing the target window can alter resource status data, potentially leading to resource overload. After a user action triggers a change in the target window, the system can automatically acquire the electronic device's resource status data and dynamically generate resource adjustment commands. This allows for pipeline switching between the changed target window and the changed background window, enabling real-time switching of rendering pipelines and memory usage between different windows. This helps improve system smoothness and user experience.

[0130] Resizing operations can include any operation that can adjust the window size. For example, resizing operations can include mouse operations: dragging the window edge to adjust, double-clicking the window edge to automatically adjust to a preset size, clicking the window's maximize or minimize buttons, etc. They can also include touch screen gesture operations: pinching to zoom the window, dragging the edge to adjust, and specific swipe gestures to trigger adjustments.

[0131] Changing the size of the target window will cause changes to resource status data, and a larger size can easily lead to resource overload. When users change the window size by dragging the edge, maximizing / minimizing, or using touch gestures, the system automatically detects the electronic device's resource status data and dynamically adjusts resource allocation, which helps improve system smoothness and user experience.

[0132] According to embodiments of this disclosure, the resource adjustment instruction is determined by the following operations: monitoring resource status data for the electronic device; evaluating the rendering quality of the display effect of multiple windows in real time based on the resource status data to obtain an evaluation result, wherein the resource status data includes at least one of the following: chip temperature, memory bandwidth utilization, CPU load rate, and graphics processor utilization; and generating a resource adjustment instruction based on the evaluation result and resource allocation strategy.

[0133] Each metric in the resource status data is configured with a threshold. Based on the resource status data, the rendering quality assessment of the display effect of multiple windows can include evaluating whether each metric exceeds the threshold.

[0134] For example, the threshold values ​​for chip temperature, memory bandwidth utilization, CPU load, and graphics processor utilization are all preset. It should be noted that the specific values ​​for preset temperature, preset utilization, preset load, and preset utilization can be determined according to actual needs, and this disclosure does not limit them. For example, the preset temperature can be 80°C, the preset utilization can be 10M / s, and the preset load can be 80%.

[0135] Rendering quality assessment of multiple windows based on resource status data can include: evaluating the rendering quality of multiple windows based on resource status data and threshold values ​​for each metric. For example, evaluating the rendering quality of multiple windows can include: assessing whether the chip temperature exceeds a preset temperature, whether the memory bandwidth utilization exceeds a preset utilization rate, whether the CPU load rate exceeds a preset load rate, and whether the graphics processor utilization exceeds a preset utilization rate.

[0136] The evaluation results can include the evaluation results for each metric. For example, the evaluation results can include the evaluation results for chip temperature, memory bandwidth utilization, CPU load, and graphics processor utilization. The evaluation result for each metric can be "yes" or "no". For example, the evaluation result for chip temperature can be "yes", while the evaluation result for memory bandwidth utilization can be "no".

[0137] Resource allocation strategies can include resolution allocation strategies and memory allocation strategies.

[0138] Resolution allocation strategies are used to configure different resolution parameter values ​​for different levels of windows. Resolution parameter values ​​can be absolute resolution values ​​or resolution percentages. For example, a resolution allocation strategy can include resolution parameter values ​​for a target window and resolution parameter values ​​for a background window. The target window can have one or more resolution parameter values, and the background window can have one or more resolution parameter values. The resolution parameter value for the target window is greater than the resolution parameter value for the background window.

[0139] For example, a resolution allocation strategy may include partitioning based on window priority. For high-priority windows: maintain native resolution or use dynamic super-resolution to ensure optimal image quality. For medium-priority windows, scale proportionally (e.g., to 70% of native resolution) to balance performance and sharpness. For low-priority windows, force down to the lowest available resolution (e.g., 480p) to reduce GPU load. By monitoring and evaluating the resource status data of electronic devices in real time, resource adjustment instructions can be generated reasonably based on the device's resource status, dynamically adjusting and optimizing window resources.

[0140] Memory allocation strategies can be used to configure different memory parameter values ​​for different window levels. Memory parameter values ​​can be fixed amounts of memory or memory percentages. For example, a memory allocation strategy can include memory parameter values ​​for a target window and memory parameter values ​​for a background window. The target window can have one or more memory parameter values, and the background window can have one or more memory parameter values. The memory parameter value for the target window is greater than the memory parameter value for the background window.

[0141] According to embodiments of this disclosure, a resource adjustment instruction is generated based on the evaluation results and resource allocation strategy, including: in response to the evaluation results indicating that the display effect meets preset conditions, generating a resource expansion instruction for the target window based on resource status data and resource allocation strategy, and determining the resource expansion instruction as a resource adjustment instruction; or in response to the evaluation results indicating that the display effect does not meet preset conditions, generating a resource expansion instruction for the target window and a resource compression instruction for the background window based on resource status data and resource allocation strategy.

[0142] Resource adjustment instructions are generated based on resource expansion instructions and resource compression instructions.

[0143] The preset condition can be that the evaluation result for each indicator is "no". That is, if all indicators do not exceed their respective thresholds, the evaluation result meets the preset condition. If any indicator exceeds its threshold, the evaluation result does not meet the preset condition.

[0144] When the evaluation results meet the preset conditions, indicating low resource load, only the resources of the target window need to be increased; there is no need to compress the background window resources. In this case, only resource expansion instructions for the target window need to be generated; that is, the resource adjustment instructions only contain resource expansion instructions.

[0145] When the evaluation results do not meet the preset conditions, it indicates a high resource load. In this case, it is necessary to increase the resources of the target window and compress the resources of the background window. Therefore, it is necessary to generate resource expansion instructions for the target window and resource compression instructions for the background window; that is, the resource adjustment instructions include both resource expansion and resource compression instructions.

[0146] Figure 6 is a schematic diagram of generating resource adjustment instructions according to an embodiment of the present disclosure.

[0147] As shown in Figure 6, in this embodiment 600, the resource status data 620 of the electronic device 610 is monitored in real time. Rendering quality is evaluated based on the resource status data 620, resulting in an evaluation result 630. Then, operation S640 is executed to determine whether the evaluation result 630 meets preset conditions. If it does, operation S650 is executed to generate resource expansion instructions based on the resource status data and resource allocation strategy, resulting in a resource adjustment instruction. This resource adjustment instruction includes the resource expansion instructions. If the evaluation result does not meet the preset conditions, operation S660 is executed to generate resource expansion instructions and resource compression instructions based on the resource status data and resource allocation strategy, resulting in a resource adjustment instruction. This resource adjustment instruction also includes both resource expansion and resource compression instructions.

[0148] Figure 7 is a schematic diagram of generating resource extension instructions according to an embodiment of the present disclosure.

[0149] As shown in Figure 7, the resource allocation strategy in this embodiment 700 includes a resolution allocation strategy 720 and a memory allocation strategy 710. First, based on the resource status data 730 and the resolution allocation strategy 720, the target window resolution 750 for the target window is determined; then, based on the resource status data 730 and the memory allocation strategy 710, the target window runtime memory 740 for the target window is determined; subsequently, based on the target window resolution 750 and the target window runtime memory 740, a resource expansion instruction 760 is generated.

[0150] If the evaluation results do not meet the preset conditions, it is also necessary to generate resource compression instructions for the background window. For example, generating resource compression instructions for the background window based on resource status data and resource allocation strategy may include: determining the compression resolution for the background window based on resource status data and resolution allocation strategy; determining the bandwidth for the background window based on resource status data and memory allocation strategy; and generating resource compression instructions based on compression resolution and background window bandwidth.

[0151] According to embodiments of this disclosure, the resource allocation strategy includes at least one strategy parameter; the operation further includes: acquiring monitoring indicator data of multiple windows according to preset monitoring indicators; analyzing the display effect of the multiple windows according to the monitoring indicator data to obtain analysis results; in response to the analysis results indicating that the display effect of at least one window is unreasonable, adjusting the parameter value of at least one target strategy parameter among the at least one strategy parameter according to the preset parameter adjustment strategy to obtain the adjusted resource allocation strategy.

[0152] The strategy parameters can be the resolution and memory parameters mentioned above.

[0153] Preset monitoring metrics can include any metric that reflects the status of resources. Examples of preset monitoring metrics include hardware performance metrics, temperature and power consumption metrics. Hardware performance metrics can include CPU load rate, GPU utilization, memory bandwidth utilization, etc.; temperature and power consumption metrics can include chip temperature, battery power consumption, etc.

[0154] Preset parameter adjustment strategies can include resolution scaling factor, memory quota, rendering priority weight, etc.

[0155] By acquiring real-time performance data for each window based on preset monitoring indicators and analyzing the data, the system determines whether there are any display issues (such as stuttering, tearing, or blurry images) and generates quantitative analysis results. Then, based on the analysis results and preset parameter adjustment strategies, the system flexibly adjusts resource allocation strategies, achieving automatic optimization and thus intelligent optimization of the multi-window display effect.

[0156] Figure 8 is a schematic diagram of the system architecture of the display method according to an embodiment of the present disclosure.

[0157] As shown in Figure 8, the system architecture 800 of this embodiment includes a Window Management 810 and a hardware processing unit 820. The Window Management 810 includes a window manager 811 and a focus controller 812. The window manager 811 is used to control multi-window splitting, and the focus controller 812 is used to control the focus window. The focus window here is the target window described above.

[0158] After the system starts split-screen mode, it is divided into two or more windows, such as window 1, window 2, ..., window n. There is only one focused window, and the focused application is considered the current main application. Each individual window can play independent audio and video resources.

[0159] The hardware processing unit 820 is responsible for dynamic pipeline reconstruction 821 and rendering quality evaluation 822. Specifically, it collects SoC temperature, CPU frequency, and DDR bandwidth, organizes the collected information into a hardware sensor data set, and uses texture memory management algorithms, rendering quality evaluation, and dynamic rendering pipeline reconstruction logic to interact with Window Management in real time to change the memory state of individual windows in real time.

[0160] Figure 9 is a schematic diagram of a display method according to an embodiment of the present disclosure.

[0161] As shown in Figure 9, the display method of this embodiment 900 is mainly implemented by a hardware layer 910, a system service layer 920, a rendering decision layer 930, and a display output layer 940. Specifically, it includes:

[0162] The hardware layer 910 includes a temperature sensor 911, a CPU load counter 912, a DDR bandwidth utilization detector 913, and a GPU utilization counter 914. Resource status data 915 is generated using the data collected by the temperature sensor 911, CPU load counter 912, DDR bandwidth utilization detector 913, and GPU utilization counter 914.

[0163] The system service layer 920 reads the resource status data 915 collected by the hardware layer 910 in real time. The dynamic resource adjustment module 921 performs matrix preprocessing on the resource status data 915 to generate a hardware status matrix 922. Then, based on the hardware status matrix 922, frame rate, and response time, a rendering quality assessment is performed, and an assessment result 923 is output. Afterward, resources are dynamically allocated based on the assessment result 923 and the resource allocation strategy, resulting in a resource allocation result 924. The resource allocation result 924 may include, for example, a focus window of 4K@60Hz, a non-focus window (background window) of 1080p@30Hz, and adjustments to the ASTC compression level.

[0164] The dynamic scheduling decision engine 931 of the rendering decision layer 930 obtains the resource allocation result 924 and sends a resource adjustment instruction to the rendering pipeline controller 932. The rendering pipeline controller 932 includes a shader program controller 932-1 and a thread priority controller 932-2. The shader program controller 932-1 loads the shader program according to the shader preloading strategy, obtains the shader resource pool 933, and determines that the rendering pipeline of the focus window 941 is an independent rendering pipeline, such as a high-performance pipeline, and determines that the rendering pipeline of the non-focus window 942 is a shared rendering pipeline, such as a lightweight pipeline. The thread priority controller 932-2 adjusts the thread priority in the rendering thread pool 934 in real time according to the thread priority instruction. Then, it requests the required shaders from the shader resource pool 933 and binds the obtained shaders to the corresponding rendering pipelines for rendering. For example, the shader for the focus window 941 is bound to the high-performance pipeline, and the shader for the focus window 941 is bound to the lightweight pipeline for rendering to obtain the display screen. During processing, memory usage of the relevant applications in the focused window is prioritized to improve screen smoothness.

[0165] The display output layer 940 displays the rendered image in the focus window 941 and the non-focus window 942 to achieve multi-window display.

[0166] It should be noted that the hardware state matrix is ​​a composite of data from multiple windows, including temperature and frame rate. The hardware state matrix is ​​a matrix formed by a single set of data. This data does not need to be stored; it is a temporary data set placed on DDR and maintained in a dynamic queue in the rendering library.

[0167] The hardware status matrix can include the following fields: time (microseconds), SoC temperature (°C), multi-core CPU load rate (%), DDR bandwidth utilization (GB / s), and GPU utilization (%).

[0168] Figure 10 is a schematic diagram of the display method after the focus window is changed according to an embodiment of the present disclosure.

[0169] As shown in Figure 10, the display method of this embodiment 1000 is the implementation flow of the display method after the focus window is changed. It is mainly implemented by the focus detection layer 1010, the resource scheduling layer 1020, the rendering decision layer 1030, and the display output layer 1040. Specifically, it includes:

[0170] The focus detection layer 1010 includes a focus window recognition module 1012. The focus window recognition module 1012 recognizes the touch events of the user 1011. When the touch event is a window change, it matches the application ID of each window to determine the focus window 1013 and updates the state of the focus window, such as adding a mark to the current focus window.

[0171] After determining the focus window 1013, the dynamic scheduling module 1021 in the resource scheduling layer 1020 obtains the resource status data 1022 and performs matrix preprocessing on the resource status data 1022 to generate a hardware status matrix; the resource allocation calculator 1023 generates a resource allocation scheme 1204 based on the hardware status matrix. The resource allocation scheme 1024 may include the focus window: 4K@60Hz, the non-focus window (background window): 1080p@30Hz, and ASTC compression level adjustment.

[0172] The decision rendering layer 1031 first executes the surface compositor 1031 reconfiguration step, a crucial processing step in the process that lays the groundwork for subsequent operations. After the surface compositor 1031 reconfiguration, it enters the texture resource manager 1032. The texture resource manager 1032 performs different processing depending on whether the window is the focus window: it allocates a high-precision texture 1033 to the focus window to ensure good display quality. For non-focus windows, it performs texture compression 1034, saving resources while maintaining basic display quality. After the texture resource manager 1032 completes its processing, it enters the rendering pipeline switching step 1035, switching the rendering pipeline of the focus window to an independent rendering pipeline and the rendering pipeline of the non-focus windows to a shared rendering pipeline, and then rendering is performed to obtain the displayed image.

[0173] The display output layer 1040 first uses the display compositing engine 1041 to composite the display image. After processing by the display compositing engine 1041, the image enters the display controller 1042, which controls and adjusts the display signal to ensure that the display content is output correctly. Then, the display signal is transmitted to the display device 1043 for display. The area above the display device 1043 marked "4K@60Hz" represents the focus window; "4K@60Hz" indicates support for 4K resolution and a 60Hz refresh rate. Below this, there are three areas marked "1080," each representing a non-focus window. "1080" indicates a resolution of 1080P.

[0174] It should be noted that the embodiments of this disclosure solve the problem of pipeline state synchronization and resource conflicts when switching rendering pipelines in real time between different windows (e.g., using the 8K pipeline for the focus window and the ASTC compressed pipeline for the non-focus window) by preloading multiple sets of shader programs in the GPU memory, thus avoiding recompilation overhead. Furthermore, by using the memory-mapped file system / dev / shm to share the hardware state matrix and coordinating with atomic operations to ensure data consistency, the conflict between multi-threaded collaboration and low-latency communication is resolved. Specifically, the hardware monitoring thread, dynamic scheduling thread, and rendering thread need millisecond-level collaboration, but the IPC (e.g., pipes, sockets) latency in related examples is too high. By adopting a method of predicting hardware state trends, multi-dimensional real-time data acquisition is achieved.

[0175] According to embodiments of this disclosure, the method further includes: in response to a change operation of a target window among a plurality of windows displayed by the master device, determining a changed target window, wherein the plurality of windows display the output screen of the slave device; generating a control command based on an operation on the input device of the master device; and controlling the display screen of the slave device using the control command.

[0176] The operation of changing the target window among the multiple windows displayed on the main device can include any operation that can change the window. This can be a direct interactive operation, keyboard or shortcut operation, gesture or voice operation, etc.

[0177] For example, direct interactive actions may include cursor movement actions, click or touch actions. For instance, a click or touch action may involve a user clicking or touching one of multiple windows to trigger a change action.

[0178] For example, voice operation may include voice commands. For instance, a voice command may be a user triggering a change in the target window by giving a voice command such as "switch to window A".

[0179] Input devices can be mice, keyboards, touchpads, etc. Different operations, such as mouse clicks and keyboard key presses, can be translated into corresponding control commands. For example, when a user clicks on a specific area with the mouse or enters a specific shortcut key on the keyboard, the host device can recognize these operations and generate corresponding control commands, such as scrolling the page or zooming in and out of the window.

[0180] Generating control commands based on the operation of the input device for the master device may include: converting the input signal into a slave device-compatible command format to generate control commands based on the operation of the input device for the master device.

[0181] Controlling the display screen of the slave device using control commands can include: establishing a control channel between the input device of the master device and the slave device corresponding to the modified target window through a physical interface; transmitting control commands to the slave device in real time through the control channel, so that the input device of the master device can directly control the slave device.

[0182] The control channel can be a bidirectional input control channel. The physical interface can include a Type-C interface, and the slave device can be an external monitor, projector, laptop, or other similar device.

[0183] Based on the above display method, this disclosure also provides a display device. The device will be described in detail below with reference to FIG11.

[0184] Figure 11 is a structural block diagram of a display device according to an embodiment of the present disclosure.

[0185] As shown in FIG11, the display device 1100 of this embodiment includes a first determining module 1110 and a first adjusting module 1120.

[0186] The first determining module 1110 is configured to determine, in response to a multi-window display operation for a display device in an electronic device, a target window among a plurality of windows to be displayed, wherein the target window represents the window currently being interacted with by the user.

[0187] The first adjustment module 1120 is used to respond to a resource adjustment instruction for the electronic device and adjust at least one of the display quality and operating resources of the target window according to the resource adjustment instruction, so as to control the display screen of the target window according to the adjusted display quality and at least one of the adjusted operating resources. The resource adjustment instruction is dynamically determined according to the resource status data and resource allocation strategy of the electronic device.

[0188] According to embodiments of this disclosure, the display screens of at least two of the plurality of windows include video screens.

[0189] According to embodiments of this disclosure, the electronic device includes an electronic device with video conferencing capabilities.

[0190] According to embodiments of this disclosure, the electronic device includes a smart TV.

[0191] According to embodiments of this disclosure, the display area size of the display device is larger than a preset size.

[0192] According to embodiments of this disclosure, the electronic device includes an independent rendering pipeline, and resource adjustment instructions include a target window resolution for the target window.

[0193] According to an embodiment of this disclosure, the first adjustment module includes: a first rendering submodule.

[0194] The first rendering submodule is used to render the display screen of the target window to the target window resolution using an independent rendering pipeline, based on resource adjustment instructions.

[0195] According to embodiments of this disclosure, resource adjustment instructions include thread priority instructions.

[0196] According to embodiments of this disclosure, the first adjustment module includes: an adjustment submodule and a second rendering submodule.

[0197] The adjustment submodule is used to adjust the thread priority of the rendering thread pool for the target viewport according to the thread priority instruction.

[0198] The second rendering submodule is used to render the display screen of the target window to the target window resolution based on the independent rendering pipeline and the adjusted threads.

[0199] According to embodiments of this disclosure, the resource adjustment instructions also include memory expansion instructions.

[0200] According to embodiments of this disclosure, the first adjustment module further includes an allocation submodule.

[0201] The allocation submodule is used to allocate the target window's runtime memory contained in the memory expansion instruction to the target window. According to embodiments of this disclosure, the display device further includes: a second adjustment module.

[0202] The second adjustment module is used to adjust at least one of the display quality and running resources of the background window according to the resource compression instruction when the resource adjustment instruction includes a resource compression instruction for the background window, wherein the background window represents the window after removing the target window from a plurality of windows.

[0203] According to embodiments of this disclosure, the electronic device includes a shared rendering pipeline, and resource compression instructions include compressed resolution.

[0204] According to an embodiment of this disclosure, the second adjustment instruction includes: a third rendering submodule.

[0205] The third rendering submodule is used to render the background window's display to a compressed resolution using a shared rendering pipeline, according to resource compression instructions.

[0206] According to embodiments of this disclosure, the resource compression instruction further includes a bandwidth limiting instruction.

[0207] According to embodiments of this disclosure, the second adjustment module further includes a restriction submodule.

[0208] The limiting submodule is used to limit the bandwidth of the background window to the bandwidth included in the bandwidth limiting instruction, based on the bandwidth limiting instruction.

[0209] According to embodiments of this disclosure, the shared rendering pipeline includes at least one first shared rendering pipeline and at least one second shared rendering pipeline; the background window includes at least one first-level window and at least one second-level window.

[0210] According to embodiments of this disclosure, the third rendering submodule includes: a first rendering unit.

[0211] The first rendering unit is used to render the display screen of the first-level window to the first compressed resolution using the first shared rendering pipeline according to the resource compression instructions.

[0212] According to embodiments of this disclosure, the resource compression instructions include a first compression instruction for a first-level window and a second compression instruction for a second-level window.

[0213] According to embodiments of this disclosure, the third rendering submodule further includes a second rendering unit and a third rendering unit.

[0214] The second rendering unit is used to render the display screen of the first level window to the first compression resolution using the first shared rendering pipeline according to the first compression instruction.

[0215] The third rendering unit is used to render the display screen of the second-level window to the second compression resolution using the second shared rendering pipeline according to the second compression instruction, wherein the first compression resolution is different from the second compression resolution.

[0216] According to embodiments of this disclosure, the above-described apparatus further includes: a first monitoring module.

[0217] The first monitoring module is used to monitor the resource status data of electronic devices in response to window change operations.

[0218] According to embodiments of this disclosure, the target window change operation includes at least one of the following: window size change, target window change, and window quantity change.

[0219] According to embodiments of this disclosure, the above-described apparatus further includes a second determining module and a display module.

[0220] The second determination module is used to determine the window to be changed based on the window change operation.

[0221] The display module is used to display the window to be changed using a preset window display mode.

[0222] According to embodiments of this disclosure, the resource adjustment instruction is determined using the following module:

[0223] The second monitoring module is used to monitor the resource status data of electronic devices.

[0224] The evaluation module is used to evaluate the rendering quality of the display effect of multiple windows in real time based on resource status data, and obtain the evaluation results. The resource status data includes at least one of the following: chip temperature, memory bandwidth utilization, CPU load rate, and graphics processor utilization.

[0225] The generation module is used to generate resource adjustment instructions based on the evaluation results and resource allocation strategy.

[0226] According to embodiments of this disclosure, the generation module includes: a first generation submodule, a second generation submodule, and a third generation submodule.

[0227] The first generation submodule is used to generate resource expansion instructions for the target window based on resource status data and resource allocation strategy in response to the evaluation result characterization display effect meeting preset conditions, and to determine the resource expansion instructions as resource adjustment instructions.

[0228] The second generation submodule is used to generate resource expansion instructions for the target window and resource compression instructions for the background window in response to the evaluation result indicating that the display effect does not meet the preset conditions, based on resource status data and resource allocation strategy.

[0229] The third generation submodule is used to generate resource adjustment instructions based on resource expansion instructions and resource compression instructions.

[0230] According to embodiments of this disclosure, the resource allocation strategy includes a resolution allocation strategy and a memory allocation strategy.

[0231] According to embodiments of this disclosure, the first generation submodule includes: a first determining unit, a second determining unit, and a first generation unit.

[0232] The first determining unit is used to determine the target window resolution for the target window based on resource status data and resolution allocation strategy.

[0233] The second determining unit is used to determine the target window's runtime memory based on resource status data and memory allocation strategy.

[0234] The first generation unit is used to generate resource expansion instructions based on the target window resolution and the target window's running memory.

[0235] According to embodiments of this disclosure, the second generation submodule includes: a third determining unit, a fourth determining unit, and a second generation unit.

[0236] The third determining unit is used to determine the compression resolution for the background window based on resource status data and resolution allocation strategy.

[0237] The fourth determining unit is used to determine the bandwidth for the background window based on resource status data and memory allocation strategy.

[0238] The second generation unit is used to generate resource compression instructions based on the compression resolution and the bandwidth of the background window.

[0239] According to embodiments of this disclosure, the resource allocation strategy includes at least one strategy parameter.

[0240] According to embodiments of this disclosure, the display device further includes an acquisition module, an analysis module, and a sixth adjustment module.

[0241] The acquisition module is used to acquire monitoring indicator data from multiple windows based on preset monitoring indicators.

[0242] The analysis module is used to analyze the display effects of multiple windows based on monitoring indicator data and obtain analysis results.

[0243] The sixth adjustment module is used to respond to the unreasonable display effect of at least one window representing the analysis results, and adjust the parameter value of at least one target strategy parameter among at least one strategy parameter according to the preset parameter adjustment strategy to obtain the adjusted resource allocation strategy.

[0244] According to embodiments of this disclosure, it further includes: a third determining module, an instruction generating module, and a control module.

[0245] The third determination module is used to determine the changed target window in response to the change operation of the target window among the multiple windows displayed by the main device, wherein the multiple windows display the output screen of the slave device.

[0246] The instruction generation module is used to generate control instructions based on the operation of the input device of the master device.

[0247] The control module is used to control the display screen of the slave device using control commands.

[0248] According to embodiments of this disclosure, any plurality of modules in the first determining module 1110 and the first adjusting module 1120 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the first determining module 1110 and the first adjusting module 1120 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these. Alternatively, at least one of the first determining module 1110 and the first adjusting module 1120 may be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0249] It should be noted that the display device part in the embodiments of this disclosure corresponds to the display method part in the embodiments of this disclosure. For a detailed description of the display device part, please refer to the display method part, which will not be repeated here.

[0250] Figure 12 is a block diagram of an electronic device suitable for implementing a display method according to an embodiment of the present disclosure.

[0251] As shown in FIG12, an electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0252] RAM 1203 stores various programs and data required for the operation of electronic device 1200. Processor 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. Processor 1201 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1202 and / or RAM 1203. It should be noted that the programs may also be stored in one or more memories other than ROM 1202 and RAM 1203. Processor 1201 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0253] According to embodiments of this disclosure, the electronic device 1200 may further include an input / output (I / O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1210 as needed so that computer programs read from it can be installed into the storage section 1208 as needed.

[0254] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0255] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1202 and / or RAM 1203 and / or one or more memories other than ROM 1202 and RAM 1203 described above.

[0256] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the monitoring data processing method provided in the embodiments of this disclosure.

[0257] When the computer program is executed by the processor 1201, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0258] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1209, and / or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0259] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0260] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0261] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0262] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0263] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A display method, the method comprising: In response to a multi-window display operation for a display device in an electronic device, a target window among a plurality of windows to be displayed is determined, the target window representing the window currently being interacted with by the user; as well as In response to a resource adjustment instruction for the electronic device, at least one of the display quality and operating resources of the target window is adjusted according to the resource adjustment instruction, so as to control the display screen of the target window according to at least one of the adjusted display quality and adjusted operating resources, wherein the resource adjustment instruction is dynamically determined based on the resource status data and resource allocation strategy of the electronic device.

2. The method of claim 1, wherein, At least two of the plurality of windows display video feeds.

3. The method of claim 1, wherein, The electronic devices include those with video conferencing capabilities.

4. The method of claim 2, wherein, The electronic device includes a smart TV.

5. The method according to any one of claims 1 to 4, wherein, The display area of ​​the display device is larger than the preset size.

6. The method of claim 1, wherein, The electronic device includes an independent rendering pipeline, and the resource adjustment instructions include a target window resolution for the target window. The step of adjusting at least one of the display quality and operating resources for the target window according to the resource adjustment instruction includes: According to the resource adjustment instructions, the display screen of the target window is rendered to the target window resolution using an independent rendering pipeline.

7. The method of claim 1, wherein, The resource adjustment instructions include thread priority instructions; The step of adjusting at least one of the display quality and operating resources for the target window according to the resource adjustment instruction includes: Adjust the thread priority for the target window in the rendering thread pool according to the thread priority instruction; Render the target window's display to the target window resolution using an independent rendering pipeline and adjusted threads.

8. The method according to claim 6, wherein the resource adjustment instruction further includes a memory expansion instruction; The method further includes: The memory allocated to the target window in the memory expansion instruction is allocated to the target window.

9. The method according to claim 1, further comprising: When the resource adjustment instruction includes a resource compression instruction for a background window, at least one of the display quality and operating resources for the background window is adjusted according to the resource compression instruction, wherein the background window represents the window among the plurality of windows after removing the target window.

10. The method of claim 9, wherein, The electronic device includes a shared rendering pipeline, and the resource compression instructions include compressed resolution; Adjusting at least one of the display quality and runtime resources for the background window according to the resource compression instruction includes: According to the resource compression instructions, the background window's display is rendered to the compressed resolution using a shared rendering pipeline.

11. The method of claim 10, wherein, The resource compression command also includes: bandwidth limiting command; The method further includes: According to the bandwidth limiting instruction, the bandwidth of the background window is limited to the bandwidth included in the bandwidth limiting instruction.

12. The method of claim 10, the shared rendering pipelines comprising at least one first shared rendering pipeline and at least one second shared rendering pipeline; The background window includes at least one first-level window and at least one second-level window; The step of rendering the background window's display image to the compressed resolution using a shared rendering pipeline according to the resource compression instruction includes: According to the resource compression instruction, the first shared rendering pipeline is used to render the display screen of the first level window to the compressed resolution.

13. The method of claim 12, wherein, The resource compression instructions include a first compression instruction for the first level window and a second compression instruction for the second level window; The step of rendering the background window's display image to the compressed resolution using a shared rendering pipeline according to the resource compression instruction includes: According to the first compression instruction, the first shared rendering pipeline is used to render the display screen of the first level window to the first compression resolution; According to the second compression instruction, the second shared rendering pipeline is used to render the display screen of the second level window to the second compression resolution, wherein the first compression resolution is different from the second compression resolution.

14. The method according to claim 1, further comprising: In response to window change operations, monitor resource status data for the electronic device.

15. The method of claim 14, wherein, The target window change operation includes at least one of the following: window size change, target window change, and window quantity change.

16. The method according to claim 14 or 15, further comprising: Based on the window change operation, determine the window to be changed; The window to be changed is displayed using a preset window display mode.

17. The method of claim 1, wherein, The resource adjustment instruction is determined using the following operation: Monitor the resource status data of the electronic device; Based on the resource status data, the rendering quality of the display effect of the multiple windows is evaluated in real time to obtain the evaluation result. The resource status data includes at least one of the following: chip temperature, memory bandwidth utilization, central processing unit load rate, and graphics processing unit utilization. Based on the evaluation results and the resource allocation strategy, the resource adjustment instruction is generated.

18. The method of claim 17, wherein, The step of generating the resource adjustment instruction based on the evaluation results and the resource allocation strategy includes: In response to the evaluation result indicating that the display effect meets preset conditions, a resource expansion instruction for the target window is generated based on the resource status data and the resource allocation strategy, and the resource expansion instruction is determined to be the resource adjustment instruction; or In response to the evaluation result indicating that the display effect does not meet the preset conditions, a resource expansion instruction for the target window and a resource compression instruction for the background window are generated based on the resource status data and the resource allocation strategy. The resource adjustment instruction is generated based on the resource expansion instruction and the resource compression instruction.

19. The method of claim 18, wherein, The resource allocation strategy includes a resolution allocation strategy and a memory allocation strategy; The step of generating resource extension instructions for the target window based on the resource status data and the resource allocation strategy includes: Based on the resource status data and the resolution allocation strategy, determine the target window resolution for the target window; Based on the resource status data and the memory allocation strategy, determine the target window's runtime memory for the target window; The resource expansion instruction is generated based on the target window resolution and the target window's running memory.

20. The method of claim 19, wherein, Generate resource compression instructions for the background window based on the resource status data and the resource allocation strategy, including: Based on the resource status data and the resolution allocation strategy, determine the compression resolution for the background window; Based on the resource status data and the memory allocation strategy, determine the bandwidth for the background window; The resource compression instruction is generated based on the compression resolution and the bandwidth of the background window.

21. The method of claim 19, wherein, The resource allocation strategy includes at least one strategy parameter; The operation also includes: Based on preset monitoring indicators, obtain monitoring indicator data for the multiple windows; The display effect of the multiple windows is analyzed based on the monitoring index data to obtain the analysis results; In response to the analysis results indicating that the display effect of at least one window is unreasonable, the parameter value of at least one target strategy parameter in the at least one strategy parameter is adjusted according to the preset parameter adjustment strategy to obtain the adjusted resource allocation strategy.

22. The method according to claim 1, further comprising: In response to a change operation of the target window among multiple windows displayed on the main device, the changed target window is determined, wherein the multiple windows display the output screen of the slave device; Control commands are generated based on the operation of the input device for the main device; The control commands are used to control the display screen of the slave device.

23. An electronic device, comprising: One or more processors; Memory, used to store one or more programs; as well as Display device; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 22.

24. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 22.

25. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 22.