Container graphical display method, device, equipment and storage medium

Abstract

This application discloses a container graphics display method, apparatus, device, and storage medium, relating to the field of graphics display technology. The container graphics display method includes: in response to a drawing request from an application, rendering and generating multiple graphics layers, each graphics layer being associated with a graphics buffer handle; creating a drawing surface for the graphics layers; parsing the graphics buffer handles and extracting buffer attribute information of the graphics layers; converting the graphics buffers into shared memory buffers based on the buffer attribute information and obtaining a shared memory buffer identifier corresponding to the shared memory buffer; and sending the drawing surface and the shared memory buffer identifier to a host display service, so that the host display service can composite the graphics layers into a complete display image and drive the display screen to display. This application enables the coexistence of container and host displays under limited display resources.

Description

Technical Field

[0001] This application relates to the field of graphics display technology, and in particular to a container graphics display method, apparatus, device, and storage medium. Background Technology

[0002] With the development of smart cockpit technology, in-vehicle entertainment systems have become an important component for enhancing the driving experience. To balance cost control and cross-screen interaction, some manufacturers have adopted a solution of running Android containers on Linux hosts to achieve collaboration between entertainment functions and the main cockpit control domain. However, in scenarios with limited display hardware resources, the display controller device file can only be exclusively used by one display service. If the Android container display service exclusively uses this resource to ensure its display performance, the Linux host will be unable to use display functions, resulting in the container and host being unable to coexist. If the Linux host display service exclusively uses this resource, the Android container can only resort to using graphics processor compositing, which not only consumes graphics processor resources but also leads to a decrease in system display performance and an increase in overall power consumption. Therefore, how to achieve coexistence of container and host displays under limited display resources remains a problem that needs to be solved.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a container graphics display method, apparatus, device, and storage medium, aiming to solve the technical problem of how to achieve coexistence of container and host display under limited display resources.

[0005] To achieve the above objectives, this application proposes a container graphics display method, the method comprising:

[0006] In response to the application's drawing request, multiple graphics layers are rendered, each of which is associated with a graphics buffer handle; Create a drawing surface for the graphics layer; Parse the graphics buffer handle and extract the buffer attribute information of the graphics layer; Based on the buffer attribute information, the graphics buffer corresponding to the graphics buffer handle is converted into a shared memory buffer, and the shared memory buffer identifier corresponding to the shared memory buffer is obtained; The drawing surface and the shared memory buffer identifier are sent to the host display service, so that the host display service can combine the graphics layer into a complete display screen and drive the display screen to display based on the received drawing surface and the shared memory buffer identifier.

[0007] In one embodiment, the step of creating a drawing surface for the graphics layer includes: Based on the identification information of the graphic layers, multiple graphic layers belonging to the same application interface are grouped into the same drawing window; Create an initial drawing surface for the drawing window; Based on the display position and size information of the graphic layer, a corresponding sub-drawing surface is created in the initial drawing surface to obtain a drawing surface containing the sub-drawing surface.

[0008] In one embodiment, the step of parsing the graphics buffer handle and extracting the buffer attribute information of the graphics layer includes: Obtain the file descriptor contained in the graphics buffer handle; Read buffer attribute information from the graphics buffer handle.

[0009] In one embodiment, the step of converting the graphics buffer handle corresponding to the graphics buffer handle into a shared memory buffer based on the buffer attribute information, and obtaining the shared memory buffer identifier corresponding to the shared memory buffer, includes: Obtain the file descriptor corresponding to the graphics buffer handle from the buffer attribute information; The storage parameters are determined based on the buffer attribute information; Based on the file descriptor and the storage parameters, the graphics buffer corresponding to the graphics buffer handle is converted into a shared memory buffer; Obtain the file descriptor corresponding to the shared memory buffer, and use it as the identifier of the shared memory buffer.

[0010] In one embodiment, prior to the step of rendering to generate multiple graphics layers, the method further includes: Initiate a connection request to the host display service via a socket; A handshake negotiation is performed with the host display service to establish an inter-process communication connection.

[0011] In one embodiment, the step of sending the drawing surface and the shared memory buffer identifier to the host display service includes: The drawing surface and the shared memory buffer identifier are encapsulated to generate a data set to be synthesized. The data set to be synthesized is sent to the host display service via inter-process communication.

[0012] In one embodiment, the step of rendering and generating multiple graphics layers in response to a drawing request from an application includes: Receive a drawing request submitted by the application, the drawing request including drawing instructions and materials to be processed; The graphics processor is invoked according to the drawing instructions to draw the material to be processed in the allocated graphics buffer and generate the corresponding graphics layer. Generate a corresponding graphics buffer handle for the graphics layer to obtain a graphics layer associated with a graphics buffer handle.

[0013] Furthermore, to achieve the above objectives, this application also proposes a container graphic display device, the container graphic display device comprising: The rendering module is used to render and generate multiple graphics layers in response to the application's drawing requests, each of which is associated with a graphics buffer handle; A module is created to create a drawing surface for the graphics layer; The extraction module is used to parse the graphics buffer handle and extract the buffer attribute information of the graphics layer; The conversion module is used to convert the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer according to the buffer attribute information, and to obtain the shared memory buffer identifier corresponding to the shared memory buffer. The sending module is used to send the drawing surface and the shared memory buffer identifier to the host display service, so that the host display service can combine the graphic layer into a complete display screen and drive the display screen to display based on the received drawing surface and the shared memory buffer identifier.

[0014] In addition, to achieve the above objectives, this application also proposes a container graphics display device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the container graphics display method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the container graphics display method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the container graphics display method described above.

[0017] This application provides a container graphics display method. Responding to a drawing request from an application, this application renders and generates multiple graphics layers, each associated with a graphics buffer handle; creates a drawing surface for each graphics layer; parses the graphics buffer handle, extracts the buffer attribute information of the graphics layer; converts the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer based on the buffer attribute information, and obtains the shared memory buffer identifier corresponding to the shared memory buffer; and sends the drawing surface and the shared memory buffer identifier to a host display service, so that the host display service, based on the received drawing surface and shared memory buffer identifier, composites the graphics layers into a complete display image and drives the display screen to display it. This application enables the container to independently complete the rendering and generation of graphics layers and the creation of drawing surfaces, parses and converts the graphics buffer handles, and sends the drawing surface and the converted shared memory buffer identifier to the host display service, allowing the host to complete hardware compositing using the display processing unit. Compared to the container performing layer compositing itself, this application transfers the compositing task to the host side, avoiding performance loss caused by the container calling the graphics processor for compositing, and simultaneously achieving coexistence of display functions between the container and the host under limited display resources. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the container graphic display method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the container graphic display method of this application; Figure 3 A simplified flowchart illustrating the container graphic display method provided in Embodiment 1 of this application; Figure 4 This is a schematic diagram of the module structure of the container graphics display device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the container graphics display method in this application embodiment.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] In response to a drawing request from an application, this application renders and generates multiple graphics layers, each associated with a graphics buffer handle; creates a drawing surface for each graphics layer; parses the graphics buffer handle and extracts the buffer attribute information of the graphics layer; converts the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer based on the buffer attribute information, and obtains the shared memory buffer identifier corresponding to the shared memory buffer; and sends the drawing surface and the shared memory buffer identifier to a host display service, so that the host display service, based on the received drawing surface and shared memory buffer identifier, combines the graphics layers into a complete display image and drives the display screen to display it.

[0025] With the development of smart cockpit technology, in-vehicle entertainment systems have become an important component for enhancing the driving experience. To balance cost control and cross-screen interaction, some manufacturers have adopted a solution of running Android containers on Linux hosts to achieve collaboration between entertainment functions and the main cockpit control domain. However, in scenarios with limited display hardware resources, the display controller device file can only be exclusively used by one display service. If the Android container display service exclusively uses this resource to ensure its display performance, the Linux host will be unable to use display functions, resulting in the container and host being unable to coexist. If the Linux host display service exclusively uses this resource, the Android container can only resort to using graphics processor compositing, which not only consumes graphics processor resources but also leads to a decrease in system display performance and an increase in overall power consumption. Therefore, how to achieve coexistence of container and host displays under limited display resources remains a problem that needs to be solved.

[0026] This application enables a container to independently render and generate graphics layers and create drawing surfaces. It also parses and converts the graphics buffer handle, sending the drawing surface and the converted shared memory buffer identifier to the host display service. This allows the host to utilize the display processing unit to complete hardware compositing. Compared to containers performing layer compositing themselves, this application offloads the compositing task to the host, avoiding performance losses caused by containers calling the graphics processor for compositing. Simultaneously, it achieves coexistence of display functions between the container and the host under limited display resources.

[0027] Based on this, embodiments of this application provide a method for displaying container graphics, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the container graphics display method of this application.

[0028] In this embodiment, the container graphic display method includes steps S10 to S50: Step S10: In response to the application's drawing request, render and generate multiple graphics layers, each of which is associated with a graphics buffer handle; It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or container graphics display device capable of performing the above functions. The following description uses a container graphics display device as an example to illustrate this embodiment and the subsequent embodiments.

[0029] It should be noted that before rendering, the host graphics processing unit (GPU) related device nodes need to be mapped into the Android container, and the graphics memory allocation module hardware abstraction layer (HAL) and GPU HAL software need to be adapted.

[0030] Specifically, when a user launches an application (such as a video app, music app, or system settings app) through the in-vehicle entertainment screen, the application submits a drawing request to the display system within the container. This drawing request contains the content to be drawn on the screen and related instructions.

[0031] The Android container's display service (SurfaceFlinger) is responsible for receiving graphical user interface rendering requests from upper-layer applications, including various entertainment and system application interfaces. These are rendered onto their respective graphical user interface layers. Each layer is associated with a graphics buffer, a memory region allocated by the Graphics Memory Allocator (gralloc). The layer's buffer is passed internally by the Android graphics system as a buffer handle, which is an array of fields containing specific information about that graphics buffer. Finally, the Android display service SurfaceFlinger passes the layer to the Hardware Composer (hwcomposer).

[0032] In one feasible approach, the step of rendering and generating multiple graphics layers in response to a drawing request from an application includes: receiving a drawing request submitted by the application, the drawing request including drawing instructions and materials to be processed; invoking a graphics processor according to the drawing instructions to perform drawing processing on the materials to be processed in an allocated graphics buffer to generate corresponding graphics layers; generating a corresponding graphics buffer handle for the graphics layers to obtain graphics layers associated with graphics buffer handles.

[0033] It's important to note that when a user interacts with an application on the in-vehicle entertainment screen, the application generates interface rendering requests. These requests are submitted to the display compositing service within the container via the Android system's graphics framework. Rendering requests contain two core parts: rendering instructions and raw data resources. The first is the raw data resources used during rendering. Examples include: application icon resources (PNG images), images downloaded from the network, video frame data, and text strings to be displayed. These resources are processed by the graphics processor and transformed into the final pixel content. The display compositing service then analyzes the received rendering requests and allocates an independent graphics layer for each request. For each layer, the display compositing service first allocates a suitable graphics buffer through the graphics memory allocation module (gralloc). The size and format of the graphics buffer are determined based on the output requirements described in the drawing request. For example, a full-screen video layer might require a buffer of 1920×1080 pixels in RGBA_8888 format. Next, the display compositing service submits the drawing instructions and the material to be processed to the graphics processor. The graphics processor performs the actual rendering operations, transforming, compositing, and coloring the material according to the coordinates, shape, and color information in the drawing instructions, and finally writing it into the graphics buffer. For example, for text drawing, the graphics processor converts the font outline into pixels; for image drawing, the graphics processor scales, rotates, and transforms the image before writing it into the buffer. After the drawing process is complete, the graphics buffer stores the complete pixel data of the layer, and the system creates a corresponding graphics layer object that records the layer's metadata (such as the application it belongs to, Z-axis order, etc.). The display compositing service generates a graphics buffer handle for each graphics layer's corresponding graphics buffer. The generation process of the graphics buffer handle first encapsulates the physical memory page information of the graphics buffer into a kernel-manageable object. Then, a file descriptor is created pointing to this kernel object. Next, the buffer's attribute information (width, height, pixel format, stride, etc.) is collected and packaged together with the file descriptor into a field array, namely the graphics buffer handle. After generating the graphics buffer handle, the display compositing service associates it with the corresponding graphics layer object. At this point, each graphics layer not only contains the layer's own metadata but also is associated with the graphics buffer storing pixel data through a handle. Finally, the display compositing service submits all graphics layers associated with graphics buffer handles to the hardware compositing module as input for subsequent steps.

[0034] In one feasible approach, prior to the step of rendering and generating multiple graphics layers, the method further includes: initiating a connection request to the host display service via a socket; and conducting a handshake negotiation with the host display service to establish an inter-process communication connection.

[0035] It should be noted that the Android container needs to establish a communication channel with the host display service. In this embodiment, sockets are used as the inter-process communication mechanism. Specifically, the Linux display service communication socket device node is mapped into the Android container. The Android container's hardware synthesis module can act as a client of the Linux display service to communicate with it and deliver the Android container's graphical interface.

[0036] Step S20: Create a drawing surface for the graphic layer; It's important to note that after obtaining the graphics layers, corresponding drawing entities need to be created for these layers so that the host can recognize and process them. First, a connection is established between the socket device node and the Linux display service, Wayland server. Then, Wayland server creates a corresponding drawing window, a drawing surface (surface), and a subsurface for each layer in the Android container. Layers are grouped into the same window based on their taskid or app ID (e.g., multiple layers in an application constitute the complete screen of the current application). The subsurface determines the coordinate position and size of the layer being drawn. It's crucial that the drawing window be set to the entire screen size so that the subsurface can correctly position the layer's coordinates on the screen.

[0037] Step S30: Parse the graphics buffer handle and extract the buffer attribute information of the graphics layer; It should be noted that the container's hardware composition module needs to parse the buffer handle and extract key layer buffer information, including the layer handle file descriptor (fd), the width, height, pixel format, pixel step size, and pixel modifier of the graphics buffer.

[0038] In one feasible approach, the step of parsing the graphics buffer handle and extracting the buffer attribute information of the graphics layer includes: obtaining the file descriptor contained in the graphics buffer handle; and reading the buffer attribute information from the graphics buffer handle.

[0039] It's important to note that after receiving the graphics buffer handle, the container's hardware compositing module first parses the handle's data structure. In Linux / Android systems, the graphics buffer handle is typically implemented as a structure containing a file descriptor (fd). A file descriptor is a non-negative integer that serves as a handle in Linux systems for processes to access kernel objects (such as open files, sockets, memory regions, etc.). Here, the file descriptor points to a memory object in the kernel specifically for the graphics buffer; through this file descriptor, the process can obtain access to the actual physical memory. The hardware compositing module extracts this file descriptor from the graphics buffer handle by calling an interface provided by the kernel. This file descriptor becomes the key credential for accessing and transforming the graphics buffer in subsequent steps. Next, it continues to read other attribute information from the graphics buffer handle. The graphics buffer handle, as an array of fields, contains not only the file descriptor but also encapsulates various metadata describing the graphics buffer. The hardware compositing module sequentially reads attribute information including the buffer's width, height, pixel format, pixel stride, pixel modifiers, etc. These attribute information collectively constitute a complete description of the graphics buffer and are parameters required for subsequent buffer transformations. The hardware compositing module temporarily stores this information in the form of a data structure for use in later steps.

[0040] Step S40: Convert the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer according to the buffer attribute information, and obtain the shared memory buffer identifier corresponding to the shared memory buffer; It should be noted that the container converts the Android display system format graphics buffer into a Linux Direct Memory Access Buffer (dmabuf) and passes the direct memory access buffer handle (dmabuf handle) to the Linux display service. This avoids the performance loss of memory copying when the Android container passes multiple layers to the Linux host.

[0041] In one feasible approach, the file descriptor corresponding to the graphics buffer handle is obtained from the buffer attribute information; storage parameters are determined based on the buffer attribute information; the graphics buffer corresponding to the graphics buffer handle is converted into a shared memory buffer based on the file descriptor and the storage parameters; and the file descriptor corresponding to the shared memory buffer is obtained as the shared memory buffer identifier.

[0042] It's important to note that before starting the conversion operation, the container's hardware compositing module retrieves the file descriptor again from the already extracted and stored buffer attribute information. This file descriptor serves as the bridge connecting the Android graphics buffer and subsequent shared memory mapping. The hardware compositing module analyzes the buffer attribute information, extracting the key storage parameters required for the conversion. Then, based on these parameters, it calculates the required storage size and memory alignment for the shared memory buffer, forming a complete set of storage parameters. These parameters are used when calling kernel interfaces to ensure that the converted shared memory buffer correctly reflects the data structure of the original buffer. Then, it calls the Direct Memory Access Buffer (dmabuf) related interfaces provided by the Linux kernel to perform the buffer conversion operation. Essentially, the conversion process exports the physical memory page pointed to by the Android graphics buffer as a Direct Memory Access Buffer object. The hardware compositing module passes the file descriptor and storage parameters to the kernel, which creates a new Direct Memory Access Buffer based on this information and associates it with the original physical memory page. This process does not involve data copying; it simply creates a new file descriptor in the kernel pointing to the same physical memory. After the conversion is complete, the contents of the original Android graphics buffer can now be accessed via the Direct Memory Access Buffer. This memory region retains the original pixel data while also enabling cross-process sharing and supporting direct hardware access. After successfully creating the direct memory access buffer, a new file descriptor is returned to the hardware compositing module. This new file descriptor is the unique identifier for accessing the shared memory buffer. The hardware compositing module obtains this file descriptor and stores it as the identifier for the shared memory buffer.

[0043] Step S50: Send the drawing surface and the shared memory buffer identifier to the host display service, so that the host display service can combine the graphic layers into a complete display screen and drive the display screen to display based on the received drawing surface and the shared memory buffer identifier.

[0044] It should be noted that the created drawing surface and the acquired shared memory buffer identifier are associated and sent to the display service on the Linux host through the established inter-process communication connection. Then, the Linux display service (waylandserver) associates the received drawing surface with the dmabuf and submits it to the window compositor (wayland compositor). The wayland compositor accesses the display processing unit (DPU) through the DirectRendering Manager Card (DRM Card) and uses hardware compositing to combine the graphics drawing surfaces passed from the Android container into a complete display screen. Finally, the complete display screen is pushed to the display screen for display.

[0045] In one feasible approach, the step of sending the drawing surface and the shared memory buffer identifier to the host display service includes: encapsulating the drawing surface and the shared memory buffer identifier to generate a data set to be synthesized; and sending the data set to be synthesized to the host display service via inter-process communication.

[0046] It should be noted that after completing the creation of drawing surfaces and buffer conversions for all graphics layers, the container's hardware compositing module needs to organize the relevant data into a format suitable for one-time transmission. For each graphics layer, the hardware compositing module associates its corresponding drawing surface information (including the drawing surface identifier, the position and size of sub-drawing surfaces, etc.) with the shared memory buffer identifier. Then, the hardware compositing module aggregates the associated data of all layers to form a set of data to be composited. Finally, the encapsulated set of data to be composited is sent to the host display service via inter-process communication.

[0047] In response to a drawing request from an application, this embodiment renders and generates multiple graphics layers, each associated with a graphics buffer handle. A drawing surface is created for each graphics layer. The graphics buffer handle is parsed to extract the buffer attribute information of the graphics layer. Based on the buffer attribute information, the graphics buffer corresponding to the graphics buffer handle is converted into a shared memory buffer, and a shared memory buffer identifier is obtained. The drawing surface and the shared memory buffer identifier are sent to the host display service, enabling the host display service to composite the graphics layers into a complete display image and drive the display screen to display it, based on the received drawing surface and shared memory buffer identifier. This embodiment independently completes the rendering and generation of graphics layers and the creation of drawing surfaces through a container, and parses and converts the graphics buffer handles, sending the drawing surface and the converted shared memory buffer identifier to the host display service, enabling the host to complete hardware compositing using the display processing unit. Compared to the container performing layer compositing itself, this embodiment transfers the compositing task to the host side, avoiding performance loss caused by the container calling the graphics processor for compositing, while achieving coexistence of display functions between the container and the host under limited display resources.

[0048] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S20 also includes steps S201 to S203: Step S201: Based on the identification information of the graphic layers, group multiple graphic layers belonging to the same application interface into the same drawing window; Upon receiving multiple graphics layers, the first step is to analyze the identification information of each layer. In the Android system, each graphics layer carries a task identifier (taskid) or application identifier (app ID), which is used to distinguish the application and interface to which the layer belongs. Based on this identification information, multiple layers belonging to the same application interface are grouped together. For example, in a video playback application, its video frame layer, control bar layer, and subtitle layer, although with different content, all belong to the same application interface, so these layers will be grouped together. For each such layer group, the hardware compositing module will create an independent drawing window for it, serving as a container for subsequent drawing surface creation.

[0049] Step S202: Create an initial drawing surface for the drawing window; It's important to note that for each created drawing window, the host display service is notified via an established inter-process communication connection to create an initial drawing surface for that window. The size of this initial drawing surface is typically set to the entire screen size, as the application interface may need to occupy the entire screen, and sub-layers need to be positioned within that area. Upon receiving the request, the host display service creates the corresponding drawing surface object in its internal data structure and returns the identifier of that drawing surface to the hardware compositing module. This initial drawing surface becomes the common canvas for all sub-layers.

[0050] Step S203: Based on the display position and size information of the graphic layer, create a corresponding sub-drawing surface in the initial drawing surface to obtain a drawing surface containing the sub-drawing surface.

[0051] It should be noted that the display parameters for each graphics layer are then obtained, including the layer's coordinate position on the screen (X-axis offset, Y-axis offset) and the layer's width and size information. This information is obtained from the layer metadata submitted by the display compositing service. For each graphics layer, the hardware compositing module requests the host display service to create a corresponding sub-drawing surface from the initial drawing surface created in step S22. The sub-drawing surface is a derived surface of the main drawing surface; it inherits the drawing attributes of the main drawing surface but can independently set its own position and size. The coordinate position and size information of each layer are configured to the corresponding sub-drawing surface. For example, the video frame layer may be located in the center of the screen with a size of 1920×1080; the control bar layer may be located at the bottom of the screen with a size of 1920×200. This information is precisely set in the sub-drawing surfaces. Finally, a drawing surface containing multiple sub-drawing surfaces is obtained.

[0052] This embodiment categorizes multiple graphic layers belonging to the same application interface into the same drawing window based on the identification information of the graphic layers; creates an initial drawing surface for the drawing window; and creates corresponding sub-drawing surfaces within the initial drawing surface according to the display position and size information of the graphic layers, thus obtaining a drawing surface containing the sub-drawing surfaces. This embodiment categorizes multiple graphic layers belonging to the same application interface into the same drawing window and creates a corresponding sub-drawing surface for each layer, achieving precise mapping between layers within the container and the host drawing surface.

[0053] For example, to help understand the implementation flow of the container graphic display method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 3 , Figure 3A simplified flowchart of a container graphics display method is provided. Specifically: An Android container application (entertainment application and system application) initiates a rendering request. The Android display service SurfaceFlinger receives the request and, after rendering, passes the graphical interface layers to the hardware composition module HWComposer. The hardware composition module generates three layers based on the task identifier (taskid): task1 includes layers layer1_1 and layer1_2, task2 includes layer layer2, and task3 includes layer layer3. The hardware composition module sends the drawn layer information and the direct memory access buffer (DMABUF) to the display service (Wayland server) on the Linux host. The Linux host's display service maps and associates the received drawing surfaces and DMABUFs one-to-one, generating three windows: windows1 corresponds to task1, windows2 corresponds to task2, and windows3 corresponds to task3. Then it is submitted to the Wayland compositor, which accesses the display controller hardware DPU through the DRM Card and uses hardware compositing to combine the graphics drawing surface passed by the Android container into a complete display screen. Finally, the complete display screen is pushed to the display screen for display.

[0054] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the container graphics display method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0055] This application also provides a container graphic display device, please refer to... Figure 4 The container graphic display device includes: Rendering module 10 is used to render and generate multiple graphics layers in response to a drawing request from an application, each of which is associated with a graphics buffer handle. Module 20 is used to create a drawing surface for the graphics layer; Extraction module 30 is used to parse the graphics buffer handle and extract the buffer attribute information of the graphics layer; The conversion module 40 is used to convert the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer according to the buffer attribute information, and to obtain the shared memory buffer identifier corresponding to the shared memory buffer. The sending module 50 is used to send the drawing surface and the shared memory buffer identifier to the host display service, so that the host display service can combine the graphic layer into a complete display screen and drive the display screen to display based on the received drawing surface and the shared memory buffer identifier.

[0056] This application, in response to a rendering request from an application, renders and generates multiple graphics layers, each associated with a graphics buffer handle; creates a drawing surface for each graphics layer; parses the graphics buffer handle and extracts the buffer attribute information of the graphics layer; converts the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer based on the buffer attribute information, and obtains the shared memory buffer identifier corresponding to the shared memory buffer; and sends the drawing surface and the shared memory buffer identifier to a host display service, so that the host display service, based on the received drawing surface and shared memory buffer identifier, composites the graphics layers into a complete display image and drives the display screen to display it. This application independently completes the rendering and generation of graphics layers and the creation of drawing surfaces through a container, and parses and converts the graphics buffer handle, sending the drawing surface and the converted shared memory buffer identifier to the host display service, enabling the host to complete hardware compositing using the display processing unit. Compared to the container's own layer compositing method, this application transfers the compositing task to the host side, avoiding the performance loss caused by the container calling the graphics processor for compositing, and simultaneously achieving the coexistence of display functions between the container and the host under limited display resources.

[0057] In one embodiment, the creation module 20 is further configured to classify multiple graphic layers belonging to the same application interface into the same drawing window according to the identification information of the graphic layers; create an initial drawing surface for the drawing window; and create corresponding sub-drawing surfaces in the initial drawing surface according to the display position and size information of the graphic layers, thereby obtaining a drawing surface containing the sub-drawing surfaces.

[0058] In one embodiment, the extraction module 30 is further configured to obtain the file descriptor contained in the graphics buffer handle; and read buffer attribute information from the graphics buffer handle.

[0059] In one embodiment, the conversion module 40 is further configured to obtain the file descriptor corresponding to the graphics buffer handle from the buffer attribute information; determine storage parameters according to the buffer attribute information; convert the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer based on the file descriptor and the storage parameters; and obtain the file descriptor corresponding to the shared memory buffer as a shared memory buffer identifier.

[0060] In one embodiment, the rendering module 10 is further configured to initiate a connection request to the host display service via a socket; and to perform a handshake negotiation with the host display service to complete the establishment of an inter-process communication connection.

[0061] In one embodiment, the sending module 50 is further configured to encapsulate the drawing surface and the shared memory buffer identifier to generate a data set to be synthesized; and send the data set to be synthesized to the host display service through inter-process communication.

[0062] In one embodiment, the rendering module 10 is further configured to receive a drawing request submitted by an application, the drawing request including drawing instructions and materials to be processed; call the graphics processor according to the drawing instructions to perform drawing processing on the materials to be processed in the allocated graphics buffer to generate a corresponding graphics layer; generate a corresponding graphics buffer handle for the graphics layer to obtain a graphics layer associated with a graphics buffer handle.

[0063] The container graphics display device provided in this application, employing the container graphics display method in the above embodiments, can solve the technical problem of how to achieve coexistence of container and host displays under limited display resources. Compared with the prior art, the beneficial effects of the container graphics display device provided in this application are the same as those of the container graphics display method provided in the above embodiments, and other technical features in the container graphics display device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0064] This application provides a container graphics display device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the container graphics display method in Embodiment 1 above.

[0065] The following is for reference. Figure 5 This document illustrates a structural schematic diagram of a container graphics display device suitable for implementing embodiments of this application. The container graphics display device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5The container graphics display device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0066] like Figure 5 As shown, the container graphics display device may include a processing unit 1001 (e.g., a central processing unit, a graphics processor, etc.) that can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the container graphics display device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the container graphics display device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show container graphics display devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0067] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0068] The container graphics display device provided in this application, employing the container graphics display method in the above embodiments, can solve the technical problem of how to achieve coexistence of container and host displays under limited display resources. Compared with the prior art, the beneficial effects of the container graphics display device provided in this application are the same as those of the container graphics display method provided in the above embodiments, and other technical features in this container graphics display device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0069] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0070] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0071] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the container graphics display method in the above embodiments.

[0072] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0073] The aforementioned computer-readable storage medium may be included in the container graphics display device; or it may exist independently and not assembled into the container graphics display device.

[0074] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a container graphics display device, cause the container graphics display device to: in response to a drawing request from an application, render and generate multiple graphics layers, each graphics layer being associated with a graphics buffer handle; create drawing surfaces for the graphics layers; parse the graphics buffer handles and extract buffer attribute information of the graphics layers; convert the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer based on the buffer attribute information, and obtain a shared memory buffer identifier corresponding to the shared memory buffer; and send the drawing surfaces and the shared memory buffer identifier to a host display service, so that the host display service, based on the received drawing surfaces and the shared memory buffer identifier, composites the graphics layers into a complete display image and drives the display screen to display.

[0075] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0076] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0077] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0078] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described container graphics display method, thereby solving the technical problem of how to achieve coexistence of container and host displays under limited display resources. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the container graphics display method provided in the above embodiments, and will not be repeated here.

[0079] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the container graphics display method described above.

[0080] The computer program product provided in this application can solve the technical problem of how to achieve coexistence of container and host display under limited display resources. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the container graphics display method provided in the above embodiments, and will not be repeated here.

[0081] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method of displaying a graphic on a container, characterized by, The method includes: In response to the application's drawing request, multiple graphics layers are rendered, each of which is associated with a graphics buffer handle; Create a drawing surface for the graphics layer; Parse the graphics buffer handle and extract the buffer attribute information of the graphics layer; Based on the buffer attribute information, the graphics buffer corresponding to the graphics buffer handle is converted into a shared memory buffer, and the shared memory buffer identifier corresponding to the shared memory buffer is obtained; The drawing surface and the shared memory buffer identifier are sent to the host display service, so that the host display service can combine the graphics layer into a complete display screen and drive the display screen to display based on the received drawing surface and the shared memory buffer identifier.

2. The method of claim 1, wherein, The step of creating a drawing surface for the graphic layer includes: Based on the identification information of the graphic layers, multiple graphic layers belonging to the same application interface are grouped into the same drawing window; Create an initial drawing surface for the drawing window; Based on the display position and size information of the graphic layer, a corresponding sub-drawing surface is created in the initial drawing surface to obtain a drawing surface containing the sub-drawing surface.

3. The method of claim 1, wherein, The step of parsing the graphics buffer handle and extracting the buffer attribute information of the graphics layer includes: Obtain the file descriptor contained in the graphics buffer handle; Read buffer attribute information from the graphics buffer handle.

4. The method of claim 1, wherein, The steps of converting the graphics buffer handle corresponding to the graphics buffer into a shared memory buffer based on the buffer attribute information, and obtaining the shared memory buffer identifier corresponding to the shared memory buffer, include: Obtain the file descriptor corresponding to the graphics buffer handle from the buffer attribute information; The storage parameters are determined based on the buffer attribute information; Based on the file descriptor and the storage parameters, the graphics buffer corresponding to the graphics buffer handle is converted into a shared memory buffer; Obtain the file descriptor corresponding to the shared memory buffer, and use it as the identifier of the shared memory buffer.

5. The method of claim 1, wherein, Before the step of rendering and generating multiple graphic layers, the method further includes: Initiate a connection request to the host display service via a socket; A handshake negotiation is performed with the host display service to establish an inter-process communication connection.

6. The method of claim 1, wherein, The step of sending the drawing surface and the shared memory buffer identifier to the host display service includes: The drawing surface and the shared memory buffer identifier are encapsulated to generate a data set to be synthesized. The data set to be synthesized is sent to the host display service via inter-process communication.

7. The method as described in claim 1, characterized in that, The step of rendering and generating multiple graphics layers in response to a drawing request from an application includes: Receive a drawing request submitted by the application, the drawing request including drawing instructions and materials to be processed; The graphics processor is invoked according to the drawing instructions to draw the material to be processed in the allocated graphics buffer and generate the corresponding graphics layer. Generate a corresponding graphics buffer handle for the graphics layer to obtain a graphics layer associated with a graphics buffer handle.

8. A container graphic display device, characterized in that, The device includes: The rendering module is used to render and generate multiple graphics layers in response to the application's drawing requests, each of which is associated with a graphics buffer handle; A module is created to create a drawing surface for the graphics layer; The extraction module is used to parse the graphics buffer handle and extract the buffer attribute information of the graphics layer; The conversion module is used to convert the graphics buffer corresponding to the graphics buffer handle into a shared memory buffer according to the buffer attribute information, and to obtain the shared memory buffer identifier corresponding to the shared memory buffer. The sending module is used to send the drawing surface and the shared memory buffer identifier to the host display service, so that the host display service can combine the graphic layer into a complete display screen and drive the display screen to display based on the received drawing surface and the shared memory buffer identifier.

9. A container graphic display device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the container graphics display method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the container graphics display method as described in any one of claims 1 to 7.

Images