Display method and device and vehicle

Abstract

The invention relates to a display method, device and system and a vehicle. The method relates to the technical field of intelligent cabin display, and comprises the following steps: acquiring DRM frame cache data; a display pipeline of first virtual DRM hardware is determined, the first virtual DRM hardware is one of multiple pieces of virtual DRM hardware registered in a system kernel by entity DRM hardware, and the system kernel supports operation of the multiple pieces of virtual DRM hardware and the entity DRM hardware; and displaying the DRM frame cache data through a display pipeline of the first virtual DRM hardware by using the first virtual DRM hardware. Through the technical scheme of the invention, the problems of high software and hardware cost and stability and safety of a display link in multi-screen display in related technologies are solved. The software and hardware cost of multi-screen display is reduced, and the stability and safety of the display link are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent cockpit display, and in particular to a display method, device, system and vehicle. BACKGROUND

[0002] With the development of the automobile industry, especially the rapid development of new energy vehicles, the automobile intelligent cockpit system basically uses multiple screens, such as instrument screens, central control screens, head-up displays, entertainment screens, etc., and there is a trend of more and more screens in the automobile intelligent cockpit. In related technologies, in order to ensure the stability of different display screens, the hardware resources are increased, which has the problem of high software and hardware cost. At the same time, the current mainstream vehicle-mounted system uses an Android Automotive operating system, and if all the screens run on the Android system, the display link will face the problems of stability and security. SUMMARY

[0003] In order to overcome the problems in the related art, the present disclosure provides a kind of. Only one SOC chip (hardware cost can be saved) can be used, and the DPU hardware in SOC is split through pipeline and registered as multiple virtual drm cards. The instrument screen and the screen managed in other Android systems open different drm cards respectively, so that the instrument screen can be displayed independently and safely, and the security and stability are high.

[0004] According to a first aspect of an embodiment of the present disclosure, a display method is provided, comprising:

[0005] obtaining DRM frame buffer data;

[0006] determining a display pipeline of a first virtual DRM hardware, wherein the first virtual DRM hardware is one of a plurality of virtual DRM hardware registered in a system kernel by an entity DRM hardware, and the system kernel supports the running of the plurality of virtual DRM hardware and the entity DRM hardware;

[0007] displaying the DRM frame buffer data through the display pipeline of the first virtual DRM hardware using the first virtual DRM hardware.

[0008] In some embodiments of the present disclosure, determining the display pipeline of the first virtual DRM hardware comprises:

[0009] in response to an opening instruction of the first virtual DRM hardware, checking whether the first DRM management authority exists in the authority list to obtain a checking result, the first DRM management authority being an authority of an opening process of the first virtual DRM hardware to access the entity DRM hardware;

[0010] According to the check result, a display pipeline of the first virtual DRM hardware is determined.

[0011] In some embodiments of the present disclosure, according to the check result, determining the display pipeline of the first virtual DRM hardware comprises:

[0012] If the check result is that the first DRM management permission does not exist in the permission list, a display pipeline of the first virtual DRM hardware is created using the second DRM management permission, the second DRM management permission being a permission of an opening process of the entity DRM hardware;

[0013] If the check result is that the first DRM management permission exists in the permission list, a part of the display pipeline of the entity DRM hardware is leased using the first DRM management permission as the display pipeline of the first virtual DRM hardware.

[0014] In some embodiments of the present disclosure, the DRM frame buffer data is displayed by the first virtual DRM hardware through the display pipeline of the first virtual DRM hardware, comprising:

[0015] The composition result of the layers of the DRM frame buffer data is displayed on at least one display screen corresponding to the first virtual DRM hardware by the first virtual DRM hardware through the display pipeline of the first virtual DRM hardware,

[0016] The correspondence between the first virtual DRM hardware and the at least one display screen is set in a DTSI configuration file.

[0017] In some embodiments of the present disclosure, after the DRM frame buffer data is displayed, the method further comprises:

[0018] If the check result is that the first DRM management permission does not exist in the permission list, the first DRM management permission is added to the permission list.

[0019] In some embodiments of the present disclosure, the method further comprises:

[0020] The number of the plurality of virtual DRM hardware, the display pipeline of the entity DRM hardware, and the display pipeline of the entity DRM hardware being split into the display pipelines of the plurality of virtual DRM hardware are obtained from a DTSI configuration file of a system kernel.

[0021] In some embodiments of the present disclosure, the method further comprises:

[0022] In the system kernel, a first priority is allocated to a first container and a second priority is allocated to a second container, the first container being a system or a software environment supporting running of the first virtual DRM hardware, and the second container being a system or a software environment supporting running of a second virtual DRM hardware in the plurality of virtual DRM hardware;

[0023] If the first priority is higher than the second priority, a first resource of a system kernel is allocated to the first container, and a second resource of the system kernel is allocated to the second container, the first resource being greater than the second resource;

[0024] Based on the first resource, a first virtual DRM hardware is used to display a composition result of the layers of the DRM frame buffer data in at least one display screen corresponding to the first virtual DRM hardware;

[0025] Based on the second resource, a second virtual DRM hardware is used to display a composition result of the layers of the DRM frame buffer data in at least one display screen corresponding to the second virtual DRM hardware.

[0026] According to a second aspect of the embodiments of the present disclosure, a display device is provided, including:

[0027] An obtaining module is configured to obtain DRM frame buffer data.

[0028] A determining module is configured to determine a display pipeline of a first virtual DRM hardware, the first virtual DRM hardware being one of a plurality of virtual DRM hardware registered in a system kernel by an entity DRM hardware, the system kernel supporting running of the plurality of virtual DRM hardware and the entity DRM hardware.

[0029] A display module is configured to display the DRM frame buffer data by using the first virtual DRM hardware and the display pipeline of the first virtual DRM hardware.

[0030] According to a third aspect of the embodiments of the present disclosure, a vehicle is provided, including a processor, and a memory for storing processor-executable instructions, wherein the processor is configured to implement any of the methods in the first aspect.

[0031] According to a fourth aspect of the embodiments of the present disclosure, a terminal is provided, including a processor, and a memory for storing processor-executable instructions, wherein the processor is configured to implement any of the methods in the first aspect.

[0032] According to a fifth aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to perform the method in any of the first aspect.

[0033] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: acquiring DRM frame buffer data, providing a data source for display using virtual DRM hardware; determining the display pipeline of a first virtual DRM hardware, wherein the first virtual DRM hardware is one of multiple virtual DRM hardwares registered in the system kernel, the system kernel supports the operation of multiple virtual DRM hardwares and physical DRM hardware, providing display pipelines for multiple virtual DRM hardwares for displaying DRM frame buffer data, and supporting independent display of multiple displays; using the first virtual DRM hardware, displaying DRM frame buffer data through the display pipeline of the first virtual DRM hardware, and displaying DRM frame buffer data through the display pipeline of the virtual DRM hardware. This reduces the hardware and software costs for supporting different display screens, while improving the stability and security of the display link.

[0034] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0036] Figure 1 This diagram illustrates the correspondence between objects in the DPU hardware pipeline and the DRM software architecture in related technologies.

[0037] Figure 2 This is a flowchart illustrating a display method according to some embodiments of the present disclosure;

[0038] Figure 3 This is a flowchart illustrating a method for determining the display pipeline of a first virtual DRM hardware according to some embodiments of the present disclosure;

[0039] Figure 4 This is a flowchart illustrating a method for determining the display pipeline of a first virtual DRM hardware based on inspection results, according to some embodiments of the present disclosure;

[0040] Figure 5 This is a flowchart illustrating a method for displaying DRM frame buffer data using a first virtual DRM hardware through the display pipeline of the first virtual DRM hardware, according to some embodiments of the present disclosure.

[0041] Figure 6 This is a flowchart illustrating a method for adding a first DRM management permission to a permission list, according to some embodiments of this disclosure;

[0042] Figure 7This is a flowchart illustrating a method for obtaining configuration file data according to some embodiments of this disclosure;

[0043] Figure 8 This is a flowchart illustrating a method for displaying the composite result of layers of DRM frame buffer data on at least one display screen according to different priorities, based on some embodiments of this disclosure.

[0044] Figure 9 This is a schematic diagram of the virtual DRM hardware and display pipeline after the DPU hardware display pipeline is split according to an embodiment of the present disclosure.

[0045] Figure 10 This is a schematic diagram illustrating a multi-DRM card display system applied in a car cabin according to an embodiment of the present disclosure;

[0046] Figure 11 This is a flowchart illustrating a method for displaying using a virtual DRM hardware display pipeline according to an embodiment of this disclosure;

[0047] Figure 12 This is a block diagram of a display device according to some embodiments of the present disclosure;

[0048] Figure 13 This is a block diagram illustrating a vehicle according to an exemplary embodiment. Detailed Implementation

[0049] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0050] First, a brief introduction to the relevant terms that have been disclosed:

[0051] Direct Rendering Manager (DRM): In this disclosure, it is a Linux kernel module responsible for managing and coordinating access to GPU resources. It provides an abstraction of the graphics hardware, enabling the operating system and applications to efficiently utilize these hardware resources for graphics rendering. It allows multiple applications to share the same Graphics Processing Unit (GPU) resources without interfering with each other.

[0052] DRM card: Abbreviated as DRM module, in this disclosure, refers to the kernel module in the operating system that manages direct rendering resources. It acts as a bridge between hardware and software, ensuring that user-space programs can effectively communicate with GPU hardware. When an application needs to use graphics functions, it sends a request to the DRM card through the interface provided by the DRM. The DRM card then translates these requests into an instruction set that the GPU can understand, thereby achieving efficient execution.

[0053] Display Processing Unit (DPU): In this disclosure, it refers to a unit in a modern computing architecture specifically designed for data processing and graphics rendering.

[0054] Display pipeline: In this disclosure, it refers to a series of sequentially executed processing steps used to ultimately display layers on the screen, including the Cathode Ray Tube Controller (CRTC), i.e., the display controller, the encoder, and the connector. The encoder is responsible for converting the data processed by the CRTC into signals suitable for a specific output format, such as HDMI or VGA. The connector describes how the physical interface connects to the display device, and is responsible for identifying the connection status and obtaining device information.

[0055] Layer: In this disclosure, it refers to the way an image is displayed on the screen, including position, flipping and color mixing. Each image has one or more associated layers, through which complex display effects, such as multi-layer compositing, can be achieved.

[0056] Container: In this disclosure, it refers to a lightweight, executable software package that contains an application and its dependencies.

[0057] Figure 1This diagram illustrates the correspondence between the DPU hardware pipeline and the DRM software architecture in related technologies. As shown, the Source Surface Processor Pipes (SSPP) of the DPU hardware pipeline supports plane layer composition in the DRM framebuffer. After processing on the SSPP, CRTC processing is performed on the Layer Mixer and Destination Surface Processor Pipes (DSPP). The Encoder function is implemented on the interface (INTF) module, which processes the encoded data through the Display Serial Interface (DSI) into data that is connected to the display device via the Connector.

[0058] In the field of automotive intelligent cockpit display technology, automobiles have specific requirements for the stability and safety of display devices, especially the instrument cluster, which needs to display important information in real time, such as vehicle speed, engine status, and engine temperature. Therefore, automobiles have a unique solution for the instrument cluster to ensure its stability.

[0059] In related technologies, multiple system-on-chips (SOCs) are used to support multi-screen displays in the car cabin. For instrument panels, a separate SOC chip and a separate system are used for normal display, which has very high hardware and software costs.

[0060] In related technologies, the in-vehicle system uses the Android automotive operating system. Because multiple screens in the cabin run on the Android display system, and the Android display system has inherent limitations in multi-screen display, specifically in terms of stability, security, and smoothness, the system suffers from several issues. Regarding stability, a crash in any link of the Android display chain can cause the entire display system to freeze and display a black screen. In terms of security, the instrument cluster and the in-vehicle infotainment (IVI) system share a process space and cannot be securely isolated. Regarding smoothness, the Android display system's Surfac eFlinger display serial mechanism and the time-sharing multiplexing of multi-screen refreshes are prone to stuttering.

[0061] To address the aforementioned technical challenges, this disclosure proposes a display method that decomposes the display pipeline of physical DRM hardware into multiple virtual DRM hardware display pipelines. This method can be applied to multi-screen display scenarios in automotive cockpits, as well as multi-screen display scenarios on terminal devices such as mobile phones and tablets. Since this disclosure is a system kernel-level method, it is also compatible with displays on different operating systems. The application scenarios are not limited in the embodiments of this disclosure.

[0062] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0063] To address the above-mentioned problems or one of the above-mentioned problems, this disclosure proposes the following method.

[0064] Figure 2 This is a flowchart illustrating a display method according to some embodiments of the present disclosure. For example... Figure 2 As shown, this method can be executed in automotive smart cockpits or on terminals such as mobile phones and tablets, and specifically includes the following steps.

[0065] In step 201, DRM frame buffer data is obtained.

[0066] In this embodiment, DRM frame cache data refers to the content to be displayed by the user-space application, such as the user interface (UI). When performing the display method of this disclosure, it is necessary to read the DRM frame cache data to be displayed from the memory.

[0067] In step 202, the display pipeline of the first virtual DRM hardware is determined, wherein the first virtual DRM hardware is one of multiple virtual DRM hardwares registered in the system kernel, and the system kernel supports the operation of multiple virtual DRM hardwares and physical DRM hardware.

[0068] In this embodiment, physical DRM hardware refers to the hardware resources actually used for graphics processing by the DRM architecture. In the system kernel, physical DRM hardware is typically drm card0. The first virtual DRM hardware refers to a DRM hardware virtualized based on the physical DRM hardware; this DRM hardware is one of multiple virtual DRM hardware registered in the system kernel. The system kernel supports the operation of multiple virtual DRM hardware and physical DRM hardware. To display DRM frame buffer data, a display pipeline for the first virtual DRM hardware is determined to carry the displayed data. Furthermore, multiple virtual DRM hardwares can be virtualized from the physical DRM hardware, and the display pipeline of each virtual DRM hardware can support the display on at least one display device.

[0069] In step 203, the DRM frame buffer data is displayed using the first virtual DRM hardware through the display pipeline of the first virtual DRM hardware.

[0070] In this embodiment, a first virtual DRM hardware is used. The driver for the first virtual DRM hardware is opened, and the display pipeline of the first virtual DRM hardware starts working, displaying the DRM frame buffer data through the display pipeline.

[0071] The display method provided in this disclosure acquires DRM frame buffer data, providing a data source for display using virtual DRM hardware. It determines the display pipeline of a first virtual DRM hardware, which is one of multiple virtual DRM hardwares registered in the system kernel. The system kernel supports the operation of multiple virtual DRM hardwares and the physical DRM hardware, providing display pipelines for multiple virtual DRM hardwares to display the DRM frame buffer data, supporting independent display on multiple screens. Using the first virtual DRM hardware, the DRM frame buffer data is displayed through its display pipeline. This reduces the hardware and software costs for supporting different screens while improving the stability and security of the display link.

[0072] Figure 3 This is a flowchart illustrating a method for determining the display pipeline of a first virtual DRM hardware according to some embodiments of this disclosure. Figure 3 As shown, it includes the following steps:

[0073] In step 301, in response to the opening command of the first virtual DRM hardware, the system checks whether the first DRM management permission exists in the permission list and obtains the check result. The first DRM management permission is the permission of the opening process of the first virtual DRM hardware to access the physical DRM hardware.

[0074] In this embodiment, the permission list refers to a list that records the permissions of virtual DRM hardware opening processes to access physical DRM hardware. It can record the access permissions of multiple virtual DRM hardwares to the physical DRM hardware. The first DRM management permission refers to the permission that the opening process of the first virtual DRM hardware has to access the physical DRM hardware. When the first virtual DRM hardware receives an instruction to be opened, the system checks whether the first DRM management permission of the virtual DRM hardware already exists in the permission list, and obtains the check result.

[0075] In step 302, the display pipeline of the first virtual DRM hardware is determined based on the inspection results.

[0076] In this embodiment, based on the inspection results, the display pipeline of the first virtual DRM hardware is determined to display DRM frame cache data. For example, if the inspection result shows that a first DRM management permission exists in the permission list, then display pipeline resources are directly leased from the physical DRM hardware based on this first DRM management permission. If the inspection result shows that a first DRM management permission does not exist in the permission list, then a first DRM management permission is created, thereby obtaining the right to use the physical DRM display pipeline resources, and the display pipeline of the physical DRM is split to form the display pipeline of the virtual DRM hardware.

[0077] In this embodiment, when the first virtual DRM hardware is enabled, the display pipeline of the first virtual DRM hardware is determined according to whether the first DRM management permission exists in the permission list, for different situations. This provides display pipeline resources for displays based on the virtual DRM hardware.

[0078] Figure 4 This is a flowchart illustrating a method for determining the display pipeline of a first virtual DRM hardware based on inspection results, according to some embodiments of this disclosure. Figure 4 As shown, it includes the following steps.

[0079] In step 401, if the check result shows that the first DRM management permission does not exist in the permission list, the display pipeline of the first virtual DRM hardware is created using the second DRM management permission. The second DRM management permission is the permission for the opening process of the physical DRM hardware.

[0080] In this embodiment, the second DRM management permission is the permission to open the process of the physical DRM hardware. If the check result shows that the first DRM management permission does not exist in the permission list, then the display pipeline of the first virtual DRM hardware is created using the second DRM management permission. That is, based on the second DRM management permission, the physical DRM hardware creates a copy of the DRM display pipeline file as the display pipeline of the first virtual DRM hardware.

[0081] In step 402, if the check result shows that the first DRM management permission exists in the permission list, the first DRM management permission is used to rent part of the display pipeline of the physical DRM hardware as the display pipeline of the first virtual DRM hardware.

[0082] In this embodiment, if the check result shows that the first DRM management permission exists in the permission list, it means that the current virtual DRM hardware has the permission to access the physical DRM hardware. Based on the first DRM management permission, a portion of the display pipeline resources are leased from the physical DRM hardware as the display pipeline of the first virtual DRM hardware.

[0083] In this embodiment, the display pipeline of the virtual DRM hardware is determined based on different inspection results, and a display pipeline is provided for the display DRM frame cache data of the virtual DRM hardware.

[0084] Figure 5 This is a flowchart illustrating a method for displaying DRM frame buffer data using a first virtual DRM hardware through the display pipeline of the first virtual DRM hardware, according to some embodiments of this disclosure. Figure 5 As shown, it includes the following steps.

[0085] In step 501, a first virtual DRM hardware is used, and the composite result of the DRM frame buffer data layer is displayed on at least one display screen corresponding to the first virtual DRM hardware through the display pipeline of the first virtual DRM hardware. The correspondence between the first virtual DRM hardware and the at least one display screen is set in the DTSI configuration file.

[0086] In this embodiment, the display screen is a display device used to display the frame buffer content. Optionally, the display screen can be any of the central control screen, instrument panel screen, or entertainment screen in the car cabin, or it can be the screen of a mobile phone or tablet computer. When the first virtual DRM hardware already has first DRM management permissions in the permission list, the first virtual DRM hardware is used. Through the display pipeline of the first virtual DRM hardware, the composite result of the DRM frame buffer data layers is displayed on at least one display screen corresponding to the first virtual DRM hardware. This completes the multi-screen display of DRM frame buffer data using the virtual DRM display pipeline when the permission list includes the first DRM management permissions of the current virtual DRM hardware. The correspondence between the first virtual DRM hardware and at least one display screen is set in the DTSI configuration file of the system kernel. The DTSI configuration file can also set the number of virtual DRM hardware and the resources of the DPU display pipeline as needed. For example, it can set the display pipeline resources such as CRTC, Connector, and Plane of the physical DRM hardware for the virtual DRM hardware to rent.

[0087] Figure 6 This is a flowchart illustrating a method for adding a first DRM management permission to a permission list, according to some embodiments of this disclosure. Figure 6 As shown, it includes the following steps.

[0088] In step 601, if the check result shows that the first DRM management permission does not exist in the permission list, the first DRM management permission is added to the permission list.

[0089] In this embodiment, if the check result shows that the first DRM management permission does not exist in the permission list, it means that the current virtual DRM hardware does not have permission to access the physical DRM hardware. In this case, the first DRM management permission is added to the virtual DRM hardware and added to the permission list, thereby providing permission for the subsequent use of the virtual DRM hardware.

[0090] Figure 7 This is a flowchart illustrating a method for obtaining configuration file data according to some embodiments of this disclosure. Figure 7 As shown, it includes the following steps.

[0091] In step 701, the number of virtual DRM hardware devices and the display pipeline of physical DRM hardware devices are obtained from the DTSI configuration file of the system kernel. The display pipeline of physical DRM hardware devices is split into the display pipeline of multiple virtual DRM hardware devices.

[0092] In this embodiment, before displaying DRM frame buffer data through the display pipeline of the virtual DRM hardware using the display method of this disclosure, it is necessary to read the number of virtual DRM hardware devices and the display pipeline of the physical DRM hardware from the DTSI configuration file of the system kernel. The display pipeline of the virtual DRM hardware is obtained by splitting the pipeline resources of the physical DRM hardware. Optionally, the display pipeline of the physical DRM hardware can be split into multiple display pipelines of virtual DRM hardware. This achieves the virtualization of the DRM device. Through hardware virtualization technology, multiple display pipelines are added, providing multiple data display paths for applications or systems, facilitating the display output of multiple display contents. Compared to providing display hardware resources and corresponding software independently for each display device, this reduces hardware and software costs.

[0093] Figure 8 This is a flowchart illustrating a method for displaying the composite result of layers of DRM frame buffer data on at least one display screen according to different priorities, based on some embodiments of this disclosure. Figure 8 As shown, it includes the following steps.

[0094] In step 801, in the system kernel, a first priority is assigned to the first container and a second priority is assigned to the second container. The first container is a system or software environment that supports the operation of the first virtual DRM hardware, and the second container is a system or software environment that supports the operation of the second virtual DRM hardware among multiple virtual DRM hardware.

[0095] In this embodiment, the second virtual DRM hardware is a DRM hardware virtualized based on the physical DRM hardware, and is different from the first virtual DRM hardware. The first container refers to the system or software environment that supports the operation of the first virtual DRM hardware, and the second container refers to the system or software environment that supports the operation of the second virtual DRM hardware among multiple virtual DRM hardware devices. The first priority refers to the priority assigned to the process running in the first container by the scheduler in the system kernel responsible for managing processes, and the second priority refers to the priority assigned to the process running in the second container by the scheduler in the system kernel responsible for managing processes. In the system kernel, the process scheduler assigns different priorities to containers running virtual DRM hardware.

[0096] In step 802, if the first priority is higher than the second priority, the first resource of the system kernel is allocated to the first container, and the second resource of the system kernel is allocated to the second container, wherein the first resource is greater than the second resource.

[0097] In this embodiment, the first resource refers to the resources allocated by the system kernel to the first container. The second resource refers to the resources allocated by the system kernel to the second container. Both the first and second resources include computing resources, storage resources, or network resources, etc. If the first priority is higher than the second priority, it means that the priority of the process running the first container is higher than the priority of the process running the second container. To ensure the stable and secure operation of the first container, it is necessary to allocate the first resource of the system kernel to the first container and the second resource of the system kernel to the second container. The amount of the first resource is greater than the amount of the second resource. For example, the CPU unutilized rate and available memory space in the first resource are both greater than those in the second resource.

[0098] In step 803, based on the first resource, using the first virtual DRM hardware, the composite result of the DRM frame buffer data layer is displayed on at least one display screen corresponding to the first virtual DRM hardware.

[0099] In this embodiment, based on the first resource, a first virtual DRM hardware is used in the first container to display the composite result of the DRM frame buffer data layer on at least one display screen corresponding to the first virtual DRM hardware. That is, for the first container with higher priority, more resources are allocated to the display pipeline of the first virtual DRM hardware to ensure its normal operation and safe and stable display of the multimedia data stream in the display pipeline of the virtual DRM hardware.

[0100] In step 804, based on the second resource, using the second virtual DRM hardware, the composite result of the DRM frame buffer data layer is displayed on at least one display screen corresponding to the second virtual DRM hardware.

[0101] In this embodiment, based on the second resource, a second virtual DRM hardware is used in the second container to display the composite result of the DRM frame cache data layer on at least one display screen corresponding to the second virtual DRM hardware. That is, for the second container, which has a lower priority than the first container, fewer resources are used to display the composite result of the application's DRM frame cache data layer based on the display pipeline of the second virtual DRM hardware.

[0102] In one embodiment of this example,

[0103] Figure 9 This is a schematic diagram of the virtual DRM hardware and display pipeline after the DPU hardware display pipeline is split according to an embodiment of this disclosure. Figure 9 As shown, the DPU hardware display pipeline is split into two virtual DRM hardware components: the first virtual DRM hardware (Virtualdrm card1) and the second virtual DRM hardware (Virtualdrm card2). The first virtual DRM hardware receives the rendered DRM frame buffer data and performs layer composition on the Planes layer using its display pipeline. After processing by the CRTC, Encoder, and Connector of the DRM device's display pipeline, the content of the DRM frame buffer is displayed on the connected display screen. The second virtual DRM hardware contains multiple DRM frame buffers, which are combined and composited into four DRM frame buffer data sets in the Planes layer. These contents are then displayed on their respective connected display screens via four parallel display pipelines.

[0104] Figure 10 This is a schematic diagram illustrating a multi-DRM card display system applied in a car cabin, according to an embodiment of this disclosure. Figure 10As shown, the display system can be divided into user space, kernel space, and SOC space. Multiple systems or containers exist in user space. In the IVI container, the central control screen IVI-UI runs on the Android system. In the Cluster container, the instrument panel screen Cluster-UI runs on an XX system or container. In the Other container, other displays' Other-UI runs. The instrument panel display is based on DRM card1 in kernel space, the central control screen display is based on DRM card2 provided by kernel space, and the other displays' displays are based on other DRMcardX in kernel space. DRM card1, DRM card2, and DRMcardX are all virtualized by the kernel graphics system layer (KGSL) in the kernel. Furthermore, KGSL supports different displays based on the graphics resources provided by the 3D Adreno GPU, virtualizing the GPU's DPU display pipeline resources into DRM card1, DRM card2, and DRM cardX. In the SOC space, DRM card1 is supported by DPU CRTC1 of the display pipeline, DRM card2 is supported by the five DPU CRTCs of the display pipeline, and DRM cardX is supported by DPU CRTCX of the display pipeline. For different systems, the image rendering and compositing process is completed according to the system and obtains frame buffer data. The display of this frame buffer data is developed using the libdrm library, which supports the DRM architecture. For the IVI-UI central control screen running the Android system, the Android application implements the interface display function through the SurfaceFlinger and HWUI libraries. After SurfaceFlinger forms the interface, HWC performs layer compositing to obtain frame buffer data (IVI Image Rendering). This frame buffer data is processed by the Libdrm library, which supports the DRM architecture, and displayed as IVIDisplay by the virtual DRM hardware DRM card2. For HWUI, the UI rendering is completed by calling the Graphics System layer (GSL) based on OpenGL or Vulkan. GSL renders the graphics system layer based on the hardware resources provided by the GPU.For the Cluster-UI instrument panel running in the XX system or container, the interface display is drawn using a suitable rendering engine according to actual needs, such as Wayland, Weston, Flutter, Unity, etc. The drawn instrument panel frame cache data (Cluster Image Rendering) is displayed as a Cluster Display based on the virtual DRM hardware display pipeline implemented by the Libdrm function library.

[0105] Figure 11 This is a flowchart illustrating a method for displaying data using a virtual DRM hardware display pipeline, according to an embodiment of this disclosure. Figure 11 As shown, when the virtual DRM hardware (virtual card) is opened, it checks whether the virtual card has DRM management permissions (DRM master) of the physical DRM hardware (real card). If the virtual card has the real card's DRM master, the real card's driver is opened. Based on this driver, the real card's DRM master is obtained. According to the obtained real card's DRM master, the real card's DRM file is obtained to complete the splitting of the physical DRM hardware's display pipeline. The DRM modeset feature of the split display pipeline is checked, that is, the parameters such as display output resolution, color depth, and refresh rate are verified. If the verification fails, an error code is returned. If the verification passes, the virtual DRM hardware initializes the physical DRM hardware's display pipeline by leasing its resources, and then creates the virtual card's DRM master by leasing the physical DRM hardware's display pipeline resources. Based on the DRM management permissions of the virtual DRM hardware, the system checks the validity of the pipeline resources currently leased from the physical DRM hardware. If the resource is invalid, an error code is returned; otherwise, it indicates that the leased pipeline resources are valid, meaning the display pipeline resources of the physical DRM hardware have been successfully leased. The system then lists all pipeline resources of the physical DRM hardware and adds the virtual card's DRM master to the real card's permission list of DRM masters. This successfully establishes the display pipeline for the virtual DRM hardware, allowing framebuffer data to be displayed and output through this virtual DRM hardware display pipeline.

[0106] This disclosure proposes a display method that acquires DRM frame buffer data, providing a data source for display using virtual DRM hardware. It determines the display pipeline of a first virtual DRM hardware, which is one of multiple virtual DRM hardwares registered in the system kernel. The system kernel supports the operation of multiple virtual DRM hardwares and the physical DRM hardware, providing display pipelines for multiple virtual DRM hardwares to display the DRM frame buffer data, supporting independent display on multiple screens. Using the first virtual DRM hardware, the DRM frame buffer data is displayed through its display pipeline. This reduces the hardware and software costs for supporting different screens while improving the stability and security of the display link.

[0107] Figure 12 This is a block diagram of a display device according to some embodiments of the present disclosure. (Refer to...) Figure 12 The device includes an acquisition module 121, a determination module 122, and a display module 123.

[0108] The acquisition module 121 is configured to acquire DRM frame buffer data;

[0109] The determining module 122 is configured to determine the display pipeline of the first virtual DRM hardware, wherein the first virtual DRM hardware is one of multiple virtual DRM hardwares registered in the system kernel, and the system kernel supports the operation of multiple virtual DRM hardwares and physical DRM hardware.

[0110] The display module 123 is configured to use the first virtual DRM hardware to display DRM frame buffer data through the display pipeline of the first virtual DRM hardware.

[0111] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0112] Figure 13 This is a block diagram illustrating a vehicle 1300 according to an exemplary embodiment. For example, vehicle 1300 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicle. Vehicle 1300 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0113] Reference Figure 13The vehicle 1300 may include various subsystems, such as an infotainment system 1310, a perception system 1320, a decision control system 1330, a drive system 1340, and a computing platform 1350. The vehicle 1300 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the vehicle 1300 can be interconnected via wired or wireless means.

[0114] In some embodiments, the infotainment system 1310 may include a communication system, an entertainment system, and a navigation system, etc.

[0115] The perception system 1320 may include several sensors for sensing information about the environment surrounding the vehicle 1300. For example, the perception system 1320 may include a global positioning system (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU), a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device.

[0116] The decision control system 1330 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0117] The drive system 1340 may include components that provide powered motion to the vehicle 1300. In one embodiment, the drive system 1340 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0118] Some or all of the functions of the vehicle 1300 are controlled by a computing platform 1350. The computing platform 1350 may include at least one processor 1351 and a memory 1352, the processor 1351 being able to execute instructions 1353 stored in the memory 1352.

[0119] Processor 1351 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0120] The memory 1352 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0121] In addition to instruction 1353, memory 1352 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 1352 can be used by computing platform 1350.

[0122] In this embodiment of the disclosure, the processor 1351 may execute instructions 1353 to complete all or part of the steps of the above-described display method.

[0123] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the display method provided in this disclosure.

[0124] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0125] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0126] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0127] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0128] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term “and / or” includes any one of the relevant listed items and any combination of any two or more; similarly, “at least one of…” includes any one of the relevant listed items and any combination of any two or more.

Claims

1. A display method, characterized in that, include: Retrieve DRM frame cache data; The display pipeline of the first virtual DRM hardware is determined, wherein the first virtual DRM hardware is one of a plurality of virtual DRM hardware registered in the system kernel, and the system kernel supports the operation of the plurality of virtual DRM hardware and the physical DRM hardware. Using the first virtual DRM hardware, the DRM frame buffer data is displayed through the display pipeline of the first virtual DRM hardware.

2. The method according to claim 1, characterized in that, The display pipeline for determining the first virtual DRM hardware includes: In response to the opening command of the first virtual DRM hardware, check whether the first DRM management permission exists in the permission list, and obtain the check result. The first DRM management permission is that the opening process of the first virtual DRM hardware has the permission to access the physical DRM hardware. Based on the inspection results, the display pipeline of the first virtual DRM hardware is determined.

3. The method according to claim 2, characterized in that, The step of determining the display pipeline of the first virtual DRM hardware based on the inspection results includes: If the check result indicates that the first DRM management permission does not exist in the permission list, the second DRM management permission is used to create the display pipeline of the first virtual DRM hardware. The second DRM management permission is the permission for the opening process of the physical DRM hardware. If the check result indicates that the first DRM management permission exists in the permission list, the first DRM management permission is used to rent a portion of the display pipeline of the physical DRM hardware as the display pipeline of the first virtual DRM hardware.

4. The method according to claim 1, characterized in that, The step of using the first virtual DRM hardware and displaying the DRM frame buffer data through the display pipeline of the first virtual DRM hardware includes: Using the first virtual DRM hardware, the composite result of the DRM frame buffer data layers is displayed on at least one display screen corresponding to the first virtual DRM hardware through the display pipeline of the first virtual DRM hardware. The correspondence between the first virtual DRM hardware and the at least one display screen is set in the DTSI configuration file.

5. The method according to any one of claims 2 to 4, characterized in that, After displaying the DRM frame cache data, the method further includes: If the check result indicates that the first DRM management permission does not exist in the permission list, then the first DRM management permission is added to the permission list.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The number of the multiple virtual DRM hardware and the display pipeline of the physical DRM hardware are obtained from the DTSI configuration file of the system kernel. The display pipeline of the physical DRM hardware is split into the display pipelines of the multiple virtual DRM hardware.

7. The method according to claim 1, characterized in that, The method further includes: In the system kernel, a first priority is assigned to a first container and a second priority is assigned to a second container. The first container is a system or software environment that supports the operation of the first virtual DRM hardware, and the second container is a system or software environment that supports the operation of the second virtual DRM hardware among the plurality of virtual DRM hardware. If the first priority is higher than the second priority, the first resource of the system kernel is allocated to the first container, and the second resource of the system kernel is allocated to the second container, wherein the first resource is greater than the second resource; Based on the first resource, using the first virtual DRM hardware, the composite result of the layer of DRM frame buffer data is displayed on at least one display screen corresponding to the first virtual DRM hardware. Based on the second resource, using the second virtual DRM hardware, the composite result of the layer of DRM frame buffer data is displayed on at least one display screen corresponding to the second virtual DRM hardware.

8. A display device, characterized in that, include: The acquisition module is used to acquire DRM frame cache data; A determination module is used to determine the display pipeline of the first virtual DRM hardware, wherein the first virtual DRM hardware is one of a plurality of virtual DRM hardware registered in the system kernel, and the system kernel supports the operation of the plurality of virtual DRM hardware and the physical DRM hardware. The display module is used to display the DRM frame buffer data through the display pipeline of the first virtual DRM hardware.

9. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Implement the method according to any one of claims 1 to 7.

10. A terminal, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: The method for implementing any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to perform the method as claimed in any one of claims 1 to 7.

Images