Method and apparatus for optimizing android container display system, electronic device, computer-readable storage medium and computer program product

Replace the HWC hardware module and generate vsync signals through the HWC program, combined with hardware rendering of the GPU hardware driver layer, the complexity and cost problems of the Android container display system in multiple concurrent scenarios are solved, and a smoother display experience is achieved.

WO2024235049A1PCT designated stage expired Publication Date: 2024-11-21MOORE THREADS TECH CO LTD

Patent Information

Application Number
PCT/CN2024/091410
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-15
Filing Date
2024-05-07
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

The existing Android container display system has problems such as increased system integration complexity, redundancy of software solutions, increased cost, poor display and poor experience in multiple concurrent scenarios, and lacks solutions to optimize standard graphics rendering interfaces from the hardware driver layer.

Method used

Replace the HWC hardware module through the HWC program, generate vsync signals, and use the GPU hardware driver layer to perform hardware rendering, reduce module complexity, save hardware costs, and directly output the initial surface rendering data in full-screen rendering scenarios to avoid unnecessary synthesis operations.

Benefits of technology

It effectively reduces the module complexity of the Android container display system, saves hardware costs, improves the smoothness of the display screen and data output efficiency, and solves the problems of poor display and poor experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024091410_21112024_PF_FP_ABST
    Figure CN2024091410_21112024_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of computers, and provides a method and apparatus for optimizing an Android container display system, an electronic device, a computer-readable storage medium and a computer program product. The method for optimizing the Android container display system comprises: a HWC program generating a vsync signal, the vsync signal comprising an app-vsync signal; a target application program sending a rendering instruction in response to the app-vsync signal, the rendering instruction being used to instruct performing graphic rendering on a surface in the target application program using hardware rendering means; in response to the rendering instruction, a GPU hardware driving layer calling a hardware GPU connected to the Android container display system to perform graphic rendering, to obtain initial surface rendering data; and a Surfaceflinger module outputting target surface rendering data according to the initial surface rendering data.
Need to check novelty before this filing date? Find Prior Art

Description

Optimization method, device, electronic device, computer-readable storage medium, and computer program product for Android container display system

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The embodiments of this application are based on the Chinese patent application with application number 202310544843.7, application date May 15, 2023, and application name “An optimization method and device for an Android container display system, and an electronic device”, and claim the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to, but is not limited to, the field of computer technology, and in particular to an optimization method, device, electronic device, computer-readable storage medium, and computer program product for an Android container display system. Background Art

[0004] Due to its openness, customizability, and widespread use, the Android system has been adopted by a wide range of embedded devices, such as mobile phones, tablets, and car computers. Both individual users and cloud computing platform vendors use multiple login accounts and utilize high-performance computing applications on low-end hardware resources. If each account requires a dedicated hardware device or requires high-performance hardware, this would result in a significant waste of resources. Container virtualization technology enables multiple systems to run simultaneously on a single device, effectively addressing this issue by offloading high-performance computing to the cloud. Given the unique characteristics of Android containers, a method for optimizing the rendering of Android container display systems is urgently needed.

[0005] Summary of the Invention

[0006] Embodiments of the present application provide an optimization method, device, electronic device, computer-readable storage medium, and computer program product for an Android container display system.

[0007] An embodiment of the present application provides an optimization method for an Android container display system, wherein the Android container display system corresponds to: an Android container application layer, an Android system framework layer, and a graphics processing unit (GPU) hardware driver layer, wherein a target application runs in the Android container application layer, and the Android system framework layer includes: a Surfaceflinger module and a hardware synthesizer (HWC) program; the method includes: the HWC program generating a vsync signal, wherein the vsync signal includes: an app-vsync signal; the target application issues a rendering instruction in response to the app-vsync signal, wherein the rendering instruction is used to instruct the use of hardware rendering to perform graphics rendering on the surface in the target application; the GPU hardware driver layer responds to the rendering instruction and calls the hardware GPU connected to the Android container display system to perform graphics rendering to obtain initial surface rendering data; the Surfaceflinger module outputs target surface rendering data based on the initial surface rendering data.

[0008] Since the Android container display system does not have an actual hardware display module, the embodiment of the present application uses an HWC program to replace the HWC hardware module to generate a vsync signal, thereby effectively reducing the module complexity of the Android container display system and saving the hardware cost of the Android container display system.

[0009] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application or in the background technology will be described below.

[0011] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0012] FIG1 shows a schematic diagram of an Android container display system according to an embodiment of the present application.

[0013] FIG2 shows a flowchart of a method for optimizing an Android container display system according to an embodiment of the present application.

[0014] FIG3 shows a rendering diagram of an Android container display system according to an embodiment of the present application.

[0015] FIG4 shows a schematic diagram of video memory application of an Android container display system according to an embodiment of the present application.

[0016] FIG5 shows a schematic diagram of video memory optimization of an Android container display system according to an embodiment of the present application.

[0017] FIG6 shows a block diagram of an optimization device for an Android container display system according to an embodiment of the present application.

[0018] FIG7 shows a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0020] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0021] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of three situations: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0022] In addition, numerous specific details are provided in the following detailed description to better illustrate the present application. Those skilled in the art will appreciate that the present application can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present application.

[0023] Related technologies primarily focus on software-level optimization based on the application of the Android container display system, failing to optimize the entire Android container display system from a hardware perspective. Furthermore, while commonly used optimization methods utilize GPU rendering, they only utilize standard graphics rendering interfaces. Currently, there is no solution for optimizing the standard graphics rendering interface and the Android container display system from the hardware driver layer. Therefore, using the existing native Android container display system to connect to containers, especially multi-concurrent Android containers, increases the complexity of system integration and creates redundancy in software solutions. High concurrency can lead to choppy display and a poor user experience; from a cost perspective, redundant hardware modules in the Android container display system can lead to increased costs.

[0024] This application provides an optimization method for an Android container display system, which can solve the above problems and optimize the Android container display system. The optimization method for an Android container display system provided in an embodiment of this application is described in detail below.

[0025] Figure 1 shows a schematic diagram of an Android container display system according to an embodiment of the present application. As shown in Figure 1, the Android container display system corresponds to: an Android container application layer 11, an Android system framework layer 12, and a graphics processing unit (GPU) hardware driver layer 13. The top layer is the Android container application layer 11, where all user operations are performed. This layer maintains the native implementation of the Android system, making it easy to follow Android system version upgrades without the need for adaptation. In addition, all operation instructions in the Android container are issued from this layer. The Android system framework layer 12 is located in the middle layer and serves as a link between the upper and lower layers. It connects to the Android container application layer 11 upward and the graphics processing unit (GPU) hardware driver layer 13 downward. The last layer is the graphics processing unit (GPU) hardware driver layer 13, which provides an operating interface for hardware GPU rendering. The Android container application layer 11 and the Android system framework layer 12 are software layers that directly face the platform and users; the graphics processing unit (GPU) hardware driver layer 13 is the driver layer, maintained by the GPU manufacturer and responding to various instructions issued by the software layer.

[0026] Figure 2 shows a flowchart of an optimization method for an Android container display system according to an embodiment of the present application. The method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable instruction stored in a memory. Alternatively, the method can be executed by a server. The method is used to optimize the Android container display system shown in Figure 1. In the Android container display system, a target application runs in the Android container application layer 11, and the Android system framework layer 12 includes: a display synthesis (Surfaceflinger) module and a hardware synthesis (HWC) program. Since there is no actual hardware display device in the Android container display system, the embodiment of the present application uses the HWC program to replace the HWC hardware module in the related art.

[0027] As shown in FIG2 , the method includes: in step S21 , a hardware compositor (HWC) program generates a vertical synchronization (vsync) signal, wherein the vertical synchronization (vsync) signal includes an application-vertical synchronization (app-vsync) signal.

[0028] In the Android container display system, there's no actual hardware display device. Therefore, the Android container display system doesn't require an HWC hardware module. Instead, the HWC program can be used to replace the HWC hardware module. The HWC program generates the app-vsync signal to update the data that the target application needs to render.

[0029] In step S22 , the target application sends a rendering instruction in response to an application-vertical synchronization (app-vsync) signal, wherein the rendering instruction is used to instruct to perform graphics rendering on a graphical interface (surface) in the target application using a hardware rendering method.

[0030] In one example, the target application responds to the app-vsync signal and calls the HWUI module based on the surface in the target application to issue a rendering instruction, indicating that graphics rendering is performed on data that needs to be rendered using a hardware rendering method.

[0031] In step S23, the graphics processing unit (GPU) hardware driver layer 13 responds to the rendering instruction and calls the hardware graphics processing unit (GPU) connected to the Android container display system to perform graphics rendering to obtain initial graphics interface (surface) rendering data.

[0032] The graphics processing unit (GPU) hardware driver layer 13 responds to the rendering instruction, uses a standard graphics rendering interface, calls the hardware GPU connected to the Android container display system to perform graphics rendering, and obtains initial surface rendering data.

[0033] Figure 3 shows a rendering diagram of the Android container display system according to an embodiment of the present application. As shown in Figure 3, step 1: the target application 301 calls the HWUI module 302; steps 2 to 4: the HWUI module 302 uses the standard graphics rendering interface 303 to update the various status information corresponding to the graphics rendering to the hardware GPU 405 through the graphics rendering driver 304, and then performs the graphics rendering operation in the hardware graphics processor (GPU) 305, and returns the initial surface rendering data corresponding to the target application 301. In the Android system, standard graphics rendering interfaces include: OpenGL ES and Vulkan, and other interfaces may also be included according to actual conditions. This application does not specifically limit this.

[0034] As shown in step 5 to step 6 of FIG3 : the initial surface rendering data is passed to the display synthesis (Surfaceflinger) module 307 through the buffer queue (Buffer Queue) 306. In the Android system of the related art, there are steps 7 to step 10 as shown in FIG3 : after receiving the initial surface rendering data, the display synthesis module 307 queries whether to use the hardware graphics processor (GPU) 305 for synthesis. If the hardware graphics processor (GPU) 305 is used for synthesis, the synthesized data will be passed to the hardware synthesizer (HWC) 308. This process requires the hardware synthesizer (HWC) 308 to receive data through the Buffer Queue, as shown in step 9 of FIG3 , and then the synthesized data is sent to the hardware display device (display) 310 through the display driver (display driver) 309 for hardware display.

[0035] However, in the Android container display system, since there is no corresponding hardware display device, that is, in the Android container display system, the display driver 309 and hardware display device 310 indicated by the dashed box in Figure 3 do not exist. Therefore, the Android container display system does not require the HWC hardware module and can use the HWC program to replace the HWC hardware module. In other words, in the related art, the HWC shown in Figure 3 is the HWC hardware module; in the embodiment of the present application, the HWC shown in Figure 3 is the HWC program. The HWC program is used to generate the vsync signal.

[0036] Since there is no display driver and hardware display device, there is no need to perform step 8 and step 10 on the synthesized display data for downward transmission, thereby reducing the application and use of the Buffer Queue shown in step 9.

[0037] In step S24 , the display synthesis (Surfaceflinger) module outputs target graphic interface (surface) rendering data according to the initial graphic interface (surface) rendering data.

[0038] The following text will describe in detail the specific process of how the Surfaceflinger module outputs target surface rendering data based on the initial surface rendering data in combination with possible implementation methods of the embodiments of the present application.

[0039] In the embodiment of the present application, since the Android container display system does not have an actual hardware display device, the HWC program is used to replace the HWC hardware module to generate the vsync signal, thereby effectively reducing the module complexity of the Android container display system and saving the hardware cost of the Android container display system.

[0040] In a possible implementation, the vsync signal includes: an sf-vsync signal; the Surfaceflinger module processes the initial surface rendering data in response to the sf-vsync signal, and then outputs the target surface rendering data.

[0041] The sf-vsync signal is used to instruct the Surfaceflinger module to process the initial surface rendering data to obtain the target surface rendering data to update the screen display information.

[0042] In a possible implementation, the HWC program generates a vsync signal through a timer trigger.

[0043] The HWC program generates the app-vsync signal and the sf-vsync signal through a timer trigger. The timer trigger interval of the app-vsync signal and the sf-vsync signal can be configured according to the actual display requirements of the Android container display system, and this application does not specifically limit this. As shown in Figure 3, the HWC generates the application-vertical synchronization (app-vsync) signal and the graphics interface-vertical synchronization (sf-vsync) signal.

[0044] In one possible implementation, the Surfaceflinger module sends a request to the HWC program whether to perform a synthesis operation, and the request is used to request whether to call the hardware GPU to perform a synthesis operation; after the Surfaceflinger module receives the instruction to perform the synthesis operation returned by the HWC program, it responds to the sf-vsync signal and calls the hardware GPU to perform a synthesis operation on the initial surface rendering data, and obtains and outputs the target surface rendering data.

[0045] The Surfaceflinger module sends a request to the HWC program to request whether to call the hardware GPU to perform the synthesis operation.

[0046] In one example, the HWC program can instruct the hardware GPU to perform a compositing operation (execute a compositing operation instruction) by returning a first flag bit. For example, when the HWC program returns a first flag bit of 0, it instructs the hardware GPU to perform a compositing operation. In addition to being in the form of 0, the first flag bit can also be in other forms, which is not specifically limited in this application. In addition to instructing the hardware GPU to perform a compositing operation by returning a first flag bit, the HWC program can also use other implementable methods to instruct the hardware GPU to perform a compositing operation, which is not specifically limited in this application.

[0047] When it is necessary to call the hardware GPU for synthesis operation, the Surfaceflinger module responds to the sf-vsync signal, calls the hardware GPU to perform synthesis operation on the initial surface rendering data, and obtains and outputs the target surface rendering data.

[0048] In a possible implementation, in a full-screen rendering scenario, the Surfaceflinger module directly outputs the initial surface rendering data as the target surface rendering data.

[0049] The full-screen rendering scene can be a game scene running in an Android container, or other scenes that require full-screen display. This application does not make specific limitations on this.

[0050] In full-screen rendering scenarios, there's no need to render and update the desktop's status bar, navigation bar, desktop background, etc. However, related technologies require the Surfaceflinger module to perform compositing operations before output, even in full-screen rendering scenarios. This redundant compositing operation lengthens the processing flow and reduces data output efficiency. This can also cause display interruptions and reduce smoothness when Android containers are launched concurrently.

[0051] Therefore, in the embodiment of the present application, in order to optimize the Android container display system in the full-screen rendering scenario, in the full-screen rendering scenario, the Surfaceflinger module directly outputs the initial surface rendering data as the target surface rendering data without performing a compositing operation, thereby improving data output efficiency in the full-screen rendering scenario and improving the smoothness of the display screen when the Android container is started concurrently. At this time, the refresh rate of the display screen is updated only by the app-vsync signal, without the need to perform a compositing operation through the sf-vsync signal.

[0052] In one possible implementation, the Surfaceflinger module sends a request to the HWC program to determine whether to perform a synthesis operation. The request is used to request whether to call the hardware GPU for the synthesis operation. After the Surfaceflinger module receives the instruction not to perform the synthesis operation returned by the HWC program, it directly outputs the initial surface rendering data as the target surface rendering data.

[0053] The Surfaceflinger module sends a request to the HWC program to request whether to call the hardware GPU to perform the synthesis operation.

[0054] In one example, the HWC program can indicate that it is not necessary to call the hardware GPU for the synthesis operation (not execute the synthesis operation instruction) by returning the second flag bit. For example, when the HWC program returns the second flag bit 1, it indicates that it is not necessary to call the hardware GPU for the synthesis operation. In addition to taking the form of 1, the second flag bit can also take other forms, and this application does not specifically limit this. In addition to indicating that it is not necessary to call the hardware GPU for the synthesis operation by returning the second flag bit, the HWC program can also use other implementable methods to indicate that it is not necessary to call the hardware GPU for the synthesis operation, and this application does not specifically limit this.

[0055] Without calling the hardware GPU for synthesis operations, the Surfaceflinger module directly outputs the initial surface rendering data as the target surface rendering data.

[0056] Figure 4 shows a schematic diagram of video memory application for an Android container display system according to an embodiment of the present application. As shown in Figure 4, the target application 401 corresponding to the Android container display system applies for video memory based on a buffer queue 402. The buffer queue uses a graphics buffer 403 to describe a graphics buffer for cross-process transmission. The underlying implementation of the Graphics Buffer is completed by the GPU manufacturer, with the focus on connecting to the Gralloc module 405 in the GPU driver layer through the Gralloc hardware abstraction layer (HAL) 404. The Gralloc module is used to apply for and manage video memory.

[0057] In one possible implementation, the GPU hardware driver layer includes: a Gralloclock module and a DMA module. When the Android container display system accesses video memory data in the hardware GPU based on the Gralloclock module, the DMA module is used to transfer the video memory data to the system memory corresponding to the Android container display system. The Android container display system reads the video memory data from the system memory.

[0058] FIG5 shows a schematic diagram of video memory optimization of an Android container display system according to an embodiment of the present application. As shown in FIG5 , when the Android container display system accesses the video memory data in the hardware GPU based on the graphics allocation (Grallo) module 5031, step 1: the Android container display system sends a video memory read instruction to the graphics processing unit (GPU) driver layer 503 based on the direct read and write tool (devmem) 501. In the related art, step 3 is adopted: the video memory data in the hardware GPU is mapped to the Android system 504 by means of a memory mapping file such as mmap. For example, as shown in FIG3 , the video memory data in the hardware GPU is mapped to the software video encoder (Soft Video Encoder) 5041 in the media module in the Android system 504 by means of a memory mapping file such as mmap. However, reading video memory data by means of a memory mapping file such as mmap will cause the CPU occupancy rate of the central processing unit (CPU) corresponding to the Android container display system to increase, and the data reading speed is slow, resulting in the display screen corresponding to the Android system 504 being stuck and not smooth. The CPU runs an Android container and applications within the container, and interfaces with the Surfaceflinger module and HWC program within the Android system 504. The Gralloclock module within the GPU driver layer implements the application and release of video memory within the Android system.

[0059] In the embodiment of the present application, step 3 is canceled and step 2 is adopted: the DMA module is used to transfer the video memory data to the system memory 502 corresponding to the Android container display system. In this way, step 4 can be executed: the CPU corresponding to the Android container display system directly reads the video memory data from the system memory 502, thereby effectively improving the video memory data reading rate. As shown in Figure 3, the DMA module is used to transfer the video memory data to the system memory 502 corresponding to the Android container display system, and the Soft Video Encoder in the media module of the Android container display system can directly read the video memory data from the system memory 502. The embodiment of the present application provides a fast and efficient way to use video memory from the hardware driver level, and optimizes the Android container display system at the hardware driver level.

[0060] In a possible implementation, the host operating system that executes the optimization method for the Android container display system is an operating system based on a Linux distribution.

[0061] Since Android system 504 is an operating system based on a Linux distribution, the host operating system that executes the optimization method for the Android container display system is also an operating system based on a Linux distribution. The Linux distribution can be Debian, Fedora, SUSE, Gentoo, Ubuntu, or other Linux distributions, which are not specifically limited in this application.

[0062] It is understood that the various method embodiments mentioned in this application can be combined with each other to form a combined embodiment without violating the principle logic. Those skilled in the art will understand that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0063] In addition, the present application also provides an optimization device, electronic device, computer-readable storage medium, and program for an Android container display system, all of which can be used to implement any Android container display system optimization method provided in the present application. For corresponding technical solutions and descriptions, please refer to the corresponding records in the method section.

[0064] FIG6 shows a block diagram of an optimization device for an Android container display system according to an embodiment of the present application. The Android container display system corresponds to: an Android container application layer 11, an Android system framework layer 12, and a graphics processing unit (GPU) hardware driver layer 13. The target application runs in the Android container application layer 11, and the Android system framework layer 12 includes: a Surfaceflinger module and an HWC program. As shown in FIG6, the optimization device 60 for the Android container display system includes: a signal generating part 61, which is configured to control the HWC program to generate a vsync signal, wherein the vsync signal includes: an app-vsync signal;

[0065] The instruction part 62 is configured to control the target application to issue a rendering instruction in response to the app-vsync signal, wherein the rendering instruction is used to instruct the use of hardware rendering to perform graphics rendering on the surface in the target application;

[0066] The rendering part 63 is configured to control the GPU hardware driver layer to respond to the rendering instruction, call the hardware GPU connected to the Android container display system to perform graphics rendering, and obtain initial surface rendering data;

[0067] The output part 64 is configured to control the Surfaceflinger module to output the target surface rendering data according to the initial surface rendering data.

[0068] In one possible implementation, the vsync signal further includes: an sf-vsync signal; and an output portion 64 configured to:

[0069] The Surfaceflinger module is controlled to respond to the sf-vsync signal, process the initial surface rendering data, and output the target surface rendering data.

[0070] In one possible implementation, the output portion 64 is configured to: control the Surfaceflinger module to send a request to the HWC program whether to perform a synthesis operation, where the request is used to request whether to call the hardware GPU to perform the synthesis operation;

[0071] After the Surfaceflinger module receives the execution synthesis operation instruction returned by the HWC program, it responds to the sf-vsync signal and calls the hardware GPU to perform a synthesis operation on the initial surface rendering data to obtain and output the target surface rendering data.

[0072] In a possible implementation, in a full-screen rendering scenario, the output portion 64 is configured to control the Surfaceflinger module to directly output the initial surface rendering data as the target surface rendering data.

[0073] In one possible implementation, the output portion 64 is configured to: control the Surfaceflinger module to send a request to the HWC program whether to perform a synthesis operation, where the request is used to request whether to call the hardware GPU to perform the synthesis operation;

[0074] After the Surfaceflinger module receives the instruction not to perform the synthesis operation returned by the HWC program, it directly outputs the initial surface rendering data as the target surface rendering data.

[0075] In a possible implementation, the signal generating portion 61 is configured to control the HWC program to generate a vsync signal by a timer trigger.

[0076] In one possible implementation, the GPU hardware driver layer includes: a Gralloclock module and a DMA module; the Android container display system optimization device 60 also includes:

[0077] The transmission part is configured to use the DMA module to transfer the video memory data to the system memory corresponding to the Android container display system when the Android container display system accesses the video memory data in the hardware GPU based on the Gralloclock module;

[0078] The reading part is configured to control the Android container display system to read the video memory data from the system memory.

[0079] In the embodiments of the present application and other embodiments, "part" can be part of a circuit, part of a processor, part of a program or software, etc., and of course it can also be a unit, a module, or a non-modular one.

[0080] In a possible implementation, the host operating system corresponding to the optimization device of the Android container display system is an operating system based on a Linux distribution version.

[0081] This method has a specific technical connection with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware computing efficiency or execution effect (including reducing the amount of data storage, reducing the amount of data transmission, increasing the hardware processing speed, etc.), thereby obtaining the technical effect of improving the internal performance of the computer system in accordance with the laws of nature.

[0082] In some embodiments, the functions or modules included in the apparatus provided in the embodiments of the present application can be used to execute the method described in the above method embodiments, and its specific implementation can refer to the description of the above method embodiments.

[0083] The present application also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0084] An embodiment of the present application further proposes an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0085] An embodiment of the present application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0086] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0087] FIG7 shows a block diagram of an electronic device according to an embodiment of the present application. Referring to FIG7 , the electronic device 1900 can be provided as a server or a terminal device. Referring to FIG7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions that can be executed by the processing component 1922, such as an application. The application stored in the memory 1932 can include one or more modules, each of which corresponds to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0088] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Microsoft's server operating system (Windows Server™), Apple's graphical user interface-based operating system (Mac OS X™), a multi-user multi-process computer operating system (Unix™), a free and open source Unix-like operating system (Linux™), an open source Unix-like operating system (FreeBSD™), or the like.

[0089] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0090] The present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present application.

[0091] A computer-readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove having instructions stored thereon, and any suitable combination thereof. The computer-readable storage medium used herein is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses through a fiber optic cable), or an electrical signal transmitted through wires.

[0092] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0093] The computer program instructions for performing the operations of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, by utilizing the state information of computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present application.

[0094] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0095] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that, when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowcharts and / or block diagrams. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowcharts and / or block diagrams.

[0096] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0097] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a special hardware-based system that performs the function or action of the specification, or can be implemented by a combination of special hardware and computer instructions.

[0098] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0099] The above description of the various embodiments tends to emphasize the differences between the various embodiments, and reference can be made to the same or similar aspects thereof.

[0100] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0101] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0102] The embodiments of the present application have been described above. The above description is illustrative and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to better explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein. Industrial Applicability

[0103] The embodiments of the present application relate to the field of computer technology and provide an optimization method, device, electronic device, computer-readable storage medium, and computer program product for an Android container display system. The optimization method for the Android container display system includes: an HWC program generates a vsync signal, wherein the vsync signal includes: an app-vsync signal; the target application responds to the app-vsync signal and issues a rendering instruction, wherein the rendering instruction is used to instruct the use of hardware rendering to perform graphics rendering on the surface in the target application; the GPU hardware driver layer responds to the rendering instruction and calls the hardware GPU connected to the Android container display system for graphics rendering to obtain initial surface rendering data; the Surfaceflinger module outputs target surface rendering data based on the initial surface rendering data. The embodiments of the present application can effectively reduce the module complexity of the Android container display system and save hardware costs.

Claims

1. An optimization method for an Android container display system, applied to an electronic device, wherein the Android container display system corresponds to: an Android container application layer, an Android system framework layer, and a graphics processor GPU hardware driver layer, wherein a target application is running in the Android container application layer, and the Android system framework layer includes: Display synthesis Surfaceflinger module and hardware synthesis HWC program; The method comprises: The HWC program generates a vertical synchronization vsync signal, wherein the vsync signal includes: an application-vertical synchronization app-vsync signal; The target application program issues a rendering instruction in response to the app-vsync signal, wherein the rendering instruction is used to instruct to perform graphics rendering on a graphical interface surface in the target application program in a hardware rendering manner; The GPU hardware driver layer responds to the rendering instruction and calls the hardware GPU connected to the Android container display system to perform graphics rendering to obtain initial surface rendering data; The Surfaceflinger module outputs target surface rendering data according to the initial surface rendering data.

2. The method according to claim 1, wherein: The vsync signal also includes: a graphics interface-vertical synchronization sf-vsync signal; The Surfaceflinger module responds to the sf-vsync signal, processes the initial surface rendering data, and outputs the target surface rendering data.

3. The method according to claim 2, wherein: The Surfaceflinger module sends a request to the HWC program whether to perform a synthesis operation, wherein the request is used to request whether to call the hardware GPU to perform a synthesis operation; After receiving the execution synthesis operation instruction returned by the HWC program, the Surfaceflinger module responds to the sf-vsync signal and calls the hardware GPU to perform a synthesis operation on the initial surface rendering data to obtain and output the target surface rendering data.

4. The method according to claim 1, wherein: In a full-screen rendering scenario, the Surfaceflinger module directly outputs the initial surface rendering data as the target surface rendering data.

5. The method according to claim 4, wherein: The Surfaceflinger module sends a request to the HWC program whether to perform a synthesis operation, wherein the request is used to request whether to call the hardware GPU to perform a synthesis operation; After receiving the instruction not to execute the synthesis operation returned by the HWC program, the Surfaceflinger module directly outputs the initial surface rendering data as the target surface rendering data.

6. The method according to any one of claims 1 to 5, wherein: The HWC program generates the vsync signal by timer triggering.

7. The method according to any one of claims 1 to 5, wherein: The GPU hardware driver layer includes: a graphics allocation Gralloclock module and a direct memory access DMA module; When the Android container display system accesses the video memory data in the hardware GPU based on the Gralloclock module, the video memory data is transmitted to the system memory corresponding to the Android container display system by using the DMA module; The Android container display system reads the display memory data from the system memory.

8. The method according to any one of claims 1 to 5, wherein: The host operating system that executes the optimization method of the Android container display system is an operating system based on a Linux distribution version.

9. An optimization device for an Android container display system, wherein: The Android container display system corresponds to: an Android container application layer, an Android system framework layer and a GPU hardware driver layer, the Android container application layer runs a target application, and the Android system framework layer includes: a Surfaceflinger module and a HWC program; The device comprises: The signal generating part is configured to control the HWC program to generate a vsync signal, wherein the vsync signal includes: an app-vsync signal; an instruction portion configured to control the target application to respond to the app-vsync signal, Issuing a rendering instruction, wherein the rendering instruction is used to instruct to perform graphics rendering on the surface in the target application using a hardware rendering method; The rendering part is configured to control the GPU hardware driver layer to respond to the rendering instruction, call the hardware GPU connected to the Android container display system to perform graphics rendering, and obtain initial surface rendering data; The output part is configured to control the Surfaceflinger module to output the target surface rendering data according to the initial surface rendering data.

10. The device according to claim 9, wherein: The vsync signal also includes: a graphics interface-vertical synchronization sf-vsync signal; the output part is also configured as: The Surfaceflinger module is controlled to respond to the sf-vsync signal, perform data processing on the initial surface rendering data, and then output the target surface rendering data.

11. The device according to claim 10, wherein: The output section is also configured as: Control the Surfaceflinger module to send a request to the HWC program whether to perform a synthesis operation, wherein the request is used to request whether to call the hardware GPU to perform a synthesis operation; After the Surfaceflinger module receives the execution synthesis operation instruction returned by the HWC program, in response to the sf-vsync signal, the hardware GPU is called to perform a synthesis operation on the initial surface rendering data to obtain and output the target surface rendering data.

12. The device according to claim 9, wherein: In the full screen rendering scenario, the output part is also configured as: The Surfaceflinger module is controlled to directly output the initial surface rendering data as the target surface rendering data.

13. The device according to claim 12, wherein: The output section is also configured as: Control the Surfaceflinger module to send a request to the HWC program whether to perform a synthesis operation, wherein the request is used to request whether to call the hardware GPU to perform a synthesis operation; After receiving the instruction not to execute the synthesis operation returned by the HWC program, the Surfaceflinger module directly outputs the initial surface rendering data as the target surface rendering data.

14. The device according to any one of claims 9 to 13, wherein: The signal generating section is further configured to: Control the HWC program to generate the vsync signal through a timer triggering method.

15. The device according to any one of claims 9 to 13, wherein: The GPU hardware driver layer includes: a graphics allocation Gralloclock module and a direct memory access DMA module; the device also includes: The transmission part is configured to use the DMA module to transmit the video memory data to the system memory corresponding to the Android container display system when the Android container display system accesses the video memory data in the hardware GPU based on the Gralloclock module; The reading part is configured to control the Android container display system to read the display memory data from the system memory.

16. The device according to any one of claims 9 to 13, wherein: The host operating system that executes the optimization method of the Android container display system is an operating system based on a Linux distribution version.

17. An electronic device comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 8.

18. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method according to any one of claims 1 to 8.

19. A computer program product, comprising a computer program or instructions, which, when executed on an electronic device, enables the electronic device to execute the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Android container implementation method, system and equipment

    CN113110910A

  • Display picture rendering method and device and terminal equipment

    CN115988265A

  • Method and device for optimizing Android container display system and electronic equipment

    CN116360928A

  • Device and method for constructing secure containers executable by android applications, and computer-readable record medium storing program therefor

    KR102235556B1

Cited By

  • Graph rendering method and device, electronic equipment and storage medium

    CN119693216A