Display controller and method for displaying data

Abstract

A display controller includes: a first manager unit configured to receive first display data from a first data source via a service manager unit, and generate first composite data using the first display data; and a second manager unit configured to receive second display data directly from a second data source, and generate second synthesized data using the second display data. The display controller is configured to provide the first composite data and the second composite data to the image processing unit. The image processing unit is configured to perform a rendering operation using the first composite data and the second composite data, and generate rendered display data provided to the display device.

Description

Technical Field

[0001] The present invention relates to a display controller, a method for displaying data, and a method for receiving display data. Background Art

[0002] The operating system includes a Direct Rendering Manager (DRM) for interfacing with a display controller. Programs in user space access the DRM to send data to the display controller and perform necessary configurations and settings. Typically, at a given time, the DRM provides an interface that is accessible by only one user space program, which may be called the DRM Master. Attempts by other user space programs, or user-space processes, other than the DRM Master to access the DRM will fail. There is a need to access the DRM from more than one process. Summary of the Invention

[0003] This Summary of the Invention is provided to introduce a selected simplified subset of concepts that are further described in the Detailed Description below. This Summary of the Invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0004] According to one embodiment, a display controller includes:

[0005] A first manager unit configured to receive first display data from a first data source via a service manager unit and generate first composite data using the first display data; and

[0006] A second manager unit configured to directly receive second display data from a second data source and generate second composite data using the second display data;

[0007] Wherein the display controller is configured to provide the first composite data and the second composite data to an image processing unit, and the image processing unit is configured to perform a rendering operation using the first composite data and the second composite data and generate rendered display data provided to a display device.

[0008] In one or more embodiments, the first manager unit and the second manager unit are direct rendering manager modules.

[0009] In one or more embodiments,

[0010] The first manager unit includes a first kernel mode setting unit configured to configure display parameters and update display content using the received first display data; and

[0011] The second display unit includes a second kernel mode setting unit configured to configure display parameters and update display content using the received second display data.

[0012] In one or more embodiments,

[0013] The first kernel mode setting unit includes at least one first DRM buffer each configured to receive first display data, and the first kernel mode setting unit is configured to synthesize the first display data with first plane data to generate first synthesized data; and

[0014] The second kernel mode setting unit includes at least one second DRM buffer each configured to receive second display data, and the second kernel mode setting unit is configured to synthesize the second display data with second plane data to generate second synthesized data.

[0015] In one or more embodiments,

[0016] The display controller is in the kernel area of the operating system; and

[0017] The service manager unit is in the user area of the operating system.

[0018] In one or more embodiments, the operating system is the Android operating system and the service manager unit is the Surfaceflinger unit of the Android operating system.

[0019] In one or more embodiments, the operating system is the Linux operating system and the service manager unit is the Weston unit of the Linux operating system.

[0020] In one or more embodiments, the service manager unit is configured to allow only one data source to access the first manager unit.

[0021] In one or more embodiments,

[0022] The first manager unit includes a first status indicator, and the first manager unit is configured to receive the first display data in response to the status of the first status indicator being available; and

[0023] The second manager unit includes a second status indicator, and the second manager unit is configured to receive the second display data in response to the status of the second status indicator being available.

[0024] According to another embodiment, a method for displaying display data includes:

[0025] Register a manager unit including a status indicator as a target for receiving display data;

[0026] At the manager unit, in response to the status indicator indicating that the manager unit is available, directly receive display data from a data source;

[0027] Generate synthetic data by the manager unit using the received display data; and

[0028] Provide the synthetic data for rendering and display.

[0029] In one or more embodiments, the method further includes:

[0030] Register another manager unit including an additional status indicator as another target for receiving display data;

[0031] At the other manager unit, in response to the additional status indicator indicating that the other manager unit is available, receive additional display data from an additional data source via a service manager unit, and have the service manager unit block other data sources from accessing the other manager unit; and

[0032] At the other manager unit, generate additional synthetic data using the received additional display data.

[0033] In one or more embodiments, registering the manager unit as a target for receiving display data and registering the other manager unit as another target for receiving display data includes connecting the manager unit and the other manager unit together to a display subsystem node in a device tree by:

[0034] Initialize the other manager unit by binding components of the other manager unit using the binding function of the other manager unit;

[0035] Initialize the manager unit by binding components of the manager unit using the binding function of the other manager unit;

[0036] Decrement the plane count of the other manager unit by one; and

[0037] The manager unit inherits a pixel clock from the additional manager unit.

[0038] In one or more embodiments, the service manager unit is one of the Surfaceflinger module of the Android operating system and the Weston module of the Linux operating system.

[0039] In one or more embodiments, initializing the manager unit further includes: the device tree provides timing and zpos configurations for the manager unit.

[0040] In one or more embodiments,

[0041] Generating the composite data includes synthesizing the received display data with plane data to generate the composite data; and

[0042] Generating the additional composite data includes synthesizing the received additional display data with additional plane data to generate the additional composite data.

[0043] According to another embodiment, in the kernel region, a method of receiving display data includes:

[0044] Receiving first display data from a first user process via a service manager unit and rejecting receipt of additional data from another user process; wherein the first user process and the service manager unit are located in a user region different from the kernel region; and

[0045] Receiving second display data directly from a second user process.

[0046] In one or more embodiments, receiving the first display data from the first user process via the service manager unit includes: in response to a status indicator of a first manager unit located in the kernel region indicating that the first manager unit is available, the first manager unit receives the first display data.

[0047] In one or more embodiments, receiving second display data directly from the second user process includes: in response to a status indicator of a second manager unit indicating that the second manager unit is available, the second manager unit receives the second display data.

[0048] In one or more embodiments, the kernel region is the Android kernel, wherein the service manager unit is the Surfaceflinger module of the Android operating system.

[0049] In one or more embodiments, the kernel region is the Linux kernel, wherein the service manager unit is the Weston module of the Linux operating system. Description of the Drawings

[0050] For the foregoing content of the present invention to be understood in a more specific manner, a further detailed description of the present invention can be obtained by referring to the embodiments, some of which are shown by the accompanying legends. The accompanying legends only show typical embodiments of the present invention, and since the present invention can have other equally effective embodiments, the accompanying legends should not be construed as limiting the scope of the present invention. The drawings are drawn for ease of understanding rather than for measuring the present invention. For those skilled in the art, after reading this description and in combination with the accompanying legends, the benefits of the claimed inventive subject matter will be readily understood. In the drawings, like reference numerals are used to indicate like elements, and:

[0051] Figure 1 is a block diagram of a system including a display controller according to an embodiment;

[0052] Figure 2 is a block diagram showing the structural arrangement of a system including a display controller according to an embodiment;

[0053] Figure 3 is a block diagram of a system including two DRM according to an embodiment;

[0054] Figure 4 is a flowchart of a method for displaying display data according to an embodiment; and

[0055] Figure 5 is a flowchart of a method for receiving display data according to an embodiment. Detailed Embodiments

[0056] Figure 1 is a block diagram of a system including a display controller according to an embodiment. System 100 includes a display controller 102, a plurality of data sources 104, and a display module 106. In the embodiment, the display controller 102 is implemented as a display driver module in an operating system (OS), such as the Android operating system or the Linux operating system. The display controller 102 receives display data from the data source 104, performs necessary operations on the received display data, and sends the processed display data to the display module 106. In an example where the display controller 102 is implemented as a display driver module in the operating system, the data source 104 can be a user process (or user application) of the OS, which accesses the display controller 102 through an interface. In the embodiment, the display module 106 can be a display screen having a hardware interface coupled to the OS and driven by the OS.

[0057] The display controller 102 includes a service manager unit 122, a first manager unit 124, and a second manager unit 126. The first manager unit 124 receives display data from one of the data sources 104 via the service manager unit 122. The second manager unit 126 receives display data directly from one of the data sources 104 without going through or passing through the service manager unit 122. In the illustrated embodiment, the first manager unit 124 and the second manager unit 126 are implemented as the Direct Rendering Manager (DRM) of the OS. The first manager unit 124 performs operations on each received display data, that is, on the display data it receives, to generate first composite data as a response. Similarly, the second manager unit 126 also performs operations on the display data it receives to generate second composite data as a response.

[0058] In an example implementation of the operating system OS, the data source 104 and the service manager unit 122 are in the so-called "user space zone" of the OS, while the first manager unit 124 and the second manager unit 126 are in the so-called "kernel zone" of the OS. In the example where the operating system is Android OS, the service manager unit 122 is the Surfaceflinger module of Android OS. In the example where the operating system is Linux OS, the service manager unit 122 is the Weston module of Linux OS. Both the Surfaceflinger module and the Weston module only allow one user space process to access the DRM 124, which means that when the first manager unit 124 is being accessed by one of the user space processes 104 (named DRM Master), the service manager unit 122 will reject or block attempts or requests from other data sources 104 to access the DRM 124.

[0059] The service manager unit 122 manages a buffer (not shown in the figure), which receives display data from the data source 104 and buffers the received display data. In response to a synchronization signal (usually from the display module 106) indicating that the display module 106 is about to refresh a display frame, the service manager unit 122 forwards the buffered display data to the first manager unit 124. In the case where the service manager unit 122 is implemented as the Surfaceflinger module of the Android OS, the synchronization signal is the vertical synchronization signal "vsync". As described above, forwarding the buffered display data to the DRM 124 by the service manager unit 122 will limit the access of other data sources 104 to the first manager unit 124 simultaneously.

[0060] Figure 2 is a block diagram showing the structural arrangement of a system including a display controller according to an embodiment. Figure 2As shown, system 200 implemented as an operating system includes a plurality of user processes 202, also referred to as clients, a service manager unit 204, an interface unit 206, a manager module 208 including at least two manager units 282, and a Graphic Processing Unit (GPU) 210. The plurality of clients 202 are configured to send display data for display on a display screen, which is driven by the image processing unit 210 in the hardware domain. As described above, the service manager unit 204 can be implemented as the Surfaceflinger module when system 200 is the Android OS, or as the Weston module when system 200 is the Linux OS. The interface unit 206 is a set of interface APIs necessary to access the manager module 208, i.e., DRM. Typically, the interface unit 206 can be the "libdrm" resource in the OS. One of the clients 202 is shown as being connected to the interface unit 206 and the manager module 208 by a dashed line, meaning that the client 202 that interacts with the DRM 282 without going through the service manager unit 204 can directly or through the libdrm resource 206 to call the DRM 282. In the described embodiment, the client 202, the Surfaceflinger module or the Weston module 204, and the libdrm resource 206 are all located in the user interface domain of the OS. It is understood that the Surfaceflinger module or the Weston module 204 consumes more processing resources, thus making the Central Processing Unit (CPU) heavily loaded. Direct access to the DRM 282 without the participation of the Surfaceflinger module or the Weston module 204 can reduce the CPU burden, for example, significantly reducing the CPU load by up to 90%.

[0061] In an embodiment, the DRM 282 located in the kernel domain of the OS presents a state, and the service manager unit 204 or the client 202 uses this state to determine whether to send display data to the DRM 282. For example, the state of the DRM 282 is stored (such as as a flag or a status indicator), and the service manager unit 204 or the client 202 checks this state and determines whether to send display data based on this state. Also as described above, the service manager unit 204 only allows one client 202 to access the DRM module 208 at a time. However, according to various embodiments, if one of the DRM 282s is being accessed by a client 202 through the service manager unit 204, another client 202 can still access another DRM 282 simultaneously and directly without going through or passing through the service manager unit 204.

[0062] Please refer to Figure 2 , where each in the DRM 282 of the manager module 208 generates synthesized data using the received display data from the client 202 or relayed via the service manager unit 204. Examples of the manager module 208 synthesizing and generating the synthesized data will be explained in detail below. The synthesized data is provided to the image processing unit 210 for rendering and display. In an embodiment, the image processing unit 210 is located in the hardware domain. The image processing unit 210 provides the rendered display data to a display device (not shown in the figure), such as a display screen.

[0063] Figure 3 is a block diagram of a system including a DRM 300 and another DRM 300' according to an embodiment. The DRM 300 and the DRM 300' are similar to each other and are both connected to the GPU 310, similar to that shown in Figure 2 . Only the DRM 300 will be described herein. The DRM 300' has corresponding configurations and is similarly labeled, except that its labels have superscripts. Figure 3 The DRM 300 of Figure 2 can be an implementation of the manager unit 282 of the manager module 208 in Figure 2 . The DRM 300 includes a Kernel Mode Setting (KMS) unit 302 and a status indicator 304. The KMS unit 302 receives the supplied display data and uses the display data to configure display parameters and update the display content. The KMS unit 302 includes a DRM buffer 322, a CRT Controller (CRTC) 324, and a connector 326. The DRM buffer 322 receives display data, such as directly from the client 202 or through the service manager unit 204 shown in

[0064] The connector 326 serves as an interface to provide the synthesized data to the GPU 310. In various embodiments, the KMS unit 302 may include more than one DRM buffer 322, each configured to receive and buffer display data. The CRTC 324 switches between the DRM buffers 322 to read out the corresponding display data and synthesize it with the plane data.

[0065] The DRM 300 also includes a status indicator 304. The status of the status indicator 304 indicates whether the DRM 300 is accessible, meaning whether the DRM 300 is open to receive display data. If the status of the status indicator 304 indicates that the DRM 300 is available, the DRM buffer 322 receives and buffers the display data. In various embodiments, the status indicator 304 of the DRM 300 may be a flag or a register in the OS. The status indicator 304 of the DRM 300 may also be configured to update the status indicator 304' of the DRM 300', so that the available status of the DRM is shared among the DRMs.

[0066] Figure 4 is a flowchart of a method for displaying display data according to an embodiment. Figure 4 The method of Figures 1 to 3 The system of Figure 3 will be described. In step 402, the first manager unit, the DRM 124, and the second manager unit 126 are all registered as targets for receiving display data. The registration of the DRM will be described in detail below. In step 404, the service manager unit 122 buffers the display data received from the first data source 104. In step 406, the service manager unit 122 transfers the buffered display data to the first manager unit, the DRM 124. The service manager unit 122 thus rejects other data from other user processes from being provided to the first manager unit 124 being accessed. As described above with reference to

[0067] In step 410, the second manager unit 126 directly receives display data from the user process 104 of the second data source. As described above with reference to Figure 3As described, the DRM 300’ receives display data in response to the status indicator 304’ indicating that the DRM 300’ is available. In step 412, the second manager unit 126 uses the received display data to generate second composite data. As described above, the DRM 126 generates the second composite data by synthesizing the received display data with the received plane data. In step 414, the first manager unit 124 provides the first composite data to the display module 106, and the second manager unit 126 provides the second composite data to the display module 106 for rendering and display.

[0068] Return to reference Figure 3 , when the system is implemented as an OS, the graphic sub-system node in the device tree lists all display interface resources. During system initialization, both the DRM 300 and the DRM 300’ are registered with the OS, for example, registered as / dev / dri / cardx when the OS is Linux or Android. The components of the DRM 300, including the CRTC 324, the connector 326, etc., are bound together as a DRM device in the device tree through the DRM bind function, and this process may vary according to the OS. The components of the DRM 300’, including the CRTC324’, the connector 326’, etc., are also bound together as a DRM device in the device tree through the DRM bind function of the DRM 300, so that the DRM 300 and the DRM 300’ are connected together to the display subsystem node in the device tree and the commit availability is opened to resources outside the kernel domain as the target for receiving display data. Thus, the necessary configuration of the DRM 300’ is inherited from the DRM 300, such as the pixel clock when the OS is Linux or Android. During initialization, the number of occupied planes in the DRM 300 is reduced by 1 because the DRM 300’ takes over the control of one plane. In the device tree of the OS, the corresponding timing and zpos (z order, which specifies the order of planes in the CRTC 324’) are provided to the DRM300’ to initialize the DRM 300’ for its own plane composition. After this initialization, the DRM 300’ can be accessed.

[0069] It is understandable that if access is to be made through the service manager unit, for example, through the Surfaceflinger module or the Weston module, the data source is not accessible to the DRM 300 until the service manager unit is fully initialized. In the example of Android or Linux OS, the initialization of the Surfaceflinger or Weston module so that the data source can access the DRM via this module may take up to 15 seconds. The DRM 300' according to the embodiment requires a shorter time, such as 9 seconds, to start quickly and can be used by time-critical user applications. For example, the standard ASIL X (Automotive Safety Integrity Level X) in the automotive industry requires fast response for specific applications, such as image applications like Rear View Camera (RVC) display, Surround View Camera (SVC) display, or 360-degree camera display. The DRM 300' according to the embodiment performs a quick start and completes the preparation before the initialization of the Surfaceflinger or Weston module, which can meet the requirements. The user application can quickly display the content on the screen, thus enhancing the real-time experience.

[0070] During operation, both the DRM 300 and the DRM 300' commit their DRM buffers 322, 322' (commit, through the so-called "Atomic Commit" of the Linux or Android OS) to the client in the user domain. If the status indicator of the DRM indicates that the DRM is available (Atomic Check), the client then accesses the DRM, so that the DRM buffer is updated (plane atomic update). The user domain client can independently control the display content of the plane to the DRM, and the display content from multiple clients, such as Augmented Reality (AR) user applications, Virtual Reality (VR) user applications, Extended Reality (XR) user applications, etc., will be synthesized onto the plane, and it is convenient to produce an overlay or floating effect on the display.

[0071] Figure 5 is a flowchart of a method for receiving display data according to an embodiment. This method will be described with reference to Figure 2 the system 200 and Figure 3is described with respect to the system 300. The method of receiving display data is implemented in the kernel area of the OS. In step 502, in response to the status indicator 304 of the DRM 300 in the kernel area indicating that the DRM 300 is available, the DRM 300 receives display data from a first client, such as Figure 2 the first user process 202 in, via the service manager unit 204. At the same time, the service manager unit 204, such as the Surfaceflinger module of the Android OS or the Weston module of the Linux OS, rejects other data from other user processes from being provided to the DRM being accessed. In step 504, in response to the status indicator 304' of the DRM 300' in the kernel area indicating that the DRM 300' is available, the DRM 300' directly receives display data from a second client, such as Figure 2 the second user process 202 in, without going through the service manager unit 204. Step 504 may run simultaneously with step 502.

[0072] In this regard, various exemplary embodiments have been described with reference to specific examples shown. The examples of the examples are chosen to assist those skilled in the art in forming a clear understanding of each embodiment and in implementing it. However, the scope of systems, structures, and devices that may be constructed to include one or more embodiments, and the scope of methods implemented in accordance with one or more embodiments, are not limited by the exemplary examples shown. On the contrary, those skilled in the art can understand from this specification that many other configurations, structures, and methods can be implemented in accordance with the various embodiments.

[0073] It should be understood that with respect to the various positional indications used in the foregoing description of the present invention, such as top, bottom, up, down, etc., those indications are given only with reference to the corresponding drawings, and when the orientation of the device changes during manufacturing or operation, other positional relationships may instead exist. As described above, those positional relationships are described only for clarity and are not restrictive.

[0074] The foregoing description of this specification is with reference to specific embodiments and specific drawings, but the present invention should not be limited thereto and should be given by the claims. The various drawings described are exemplary and not restrictive. In the drawings, for purposes of illustration, the dimensions of the various elements may be enlarged and may not be drawn to a specific scale. This specification should also include discontinuous transformations in tolerances and properties of the various elements and the manner of operation. It should also include various weakened embodiments of the present invention.

[0075] As used in this specification and the claims, the term "comprising" does not exclude other elements or steps. Unless specifically stated otherwise, when the singular form such as "a" or "an" is used to refer to a defined or undefined element, the plural of that element should be included. Thus, the term "comprising" should not be construed as being limited to the items listed thereafter, and should not be construed as excluding other elements or steps; the scope of the description "the device comprises items A and B" should not be limited to devices that only include elements A and B. This description means that, for the purposes of this specification, only elements A and B of the device are relevant. Although coupling generally includes inductive connection, and connection generally means connection through, for example, wires, the terms "connected", "coupled", and "coupling" as used herein all indicate that there is an electrical connection between the elements being coupled or connected, and does not mean that there are no intermediate elements therebetween. When describing transistors and their connections, the terms gate, drain, and source are interchangeable with gate electrode, drain electrode, source electrode, and gate terminal, drain terminal, source terminal.

[0076] Those skilled in the art can make various specific changes without departing from the scope of the claims of the present invention.

Claims

1. A display controller, characterized in that, Comprising: A first manager unit configured to receive first display data from a first data source via a service manager unit and generate first composite data using the first display data; And A second manager unit configured to directly receive second display data from a second data source and generate second composite data using the second display data; Wherein the display controller is configured to provide the first composite data and the second composite data to an image processing unit, and the image processing unit is configured to perform a rendering operation using the first composite data and the second composite data and generate rendered display data provided to a display device.

2. The display controller according to claim 1, characterized in that The first manager unit and the second manager unit are direct rendering manager modules.

3. The display controller according to claim 2, characterized in that: The first manager unit includes a first kernel mode setting unit configured to configure display parameters and update display content using the received first display data; And The second display unit includes a second kernel mode setting unit configured to configure display parameters and update display content using the received second display data.

4. The display controller according to claim 1, wherein The service manager unit is configured to allow only one data source to access the first manager unit.

5. A method for displaying display data, characterized in that, Comprising: Registering a manager unit including a status indicator as a target for receiving display data; At the manager unit, in response to the status indicator indicating that the manager unit is available, directly receiving display data from a data source; Generating composite data by the manager unit using the received display data; And Providing the composite data for rendering and display.

6. The method according to claim 5, wherein Further comprising: Registering another manager unit including another status indicator as another target for receiving display data; At the another manager unit, in response to the another status indicator indicating that the another manager unit is available, receiving another display data from another data source via a service manager unit and preventing other data sources from accessing the another manager unit by the service manager unit; And At the another manager unit, generating another composite data using the received another display data.

7. The method according to claim 6, characterized in that: Registering the manager unit as a target for receiving display data and registering the another manager unit as another target for receiving display data includes connecting the manager unit and the another manager unit together to a display subsystem node in a device tree by: Initializing the another manager unit by binding components of the another manager unit using the binding function of the another manager unit; Initializing the manager unit by binding components of the manager unit using the binding function of the another manager unit; Decrementing the plane count of the another manager unit; and The manager unit inherits a pixel clock from the additional manager unit.

8. A method for receiving display data in a kernel region, characterized in that, Comprising: Receiving first display data from a first user process via a service manager unit and rejecting receipt of additional data from another user process; wherein the first user process and the service manager unit are located in a user area different from the kernel area; and directly receiving second display data from a second user process.

9. The method according to claim 8, wherein Receiving the first display data from the first user process via the service manager unit includes: in response to a status indicator of a first manager unit located in the kernel area indicating that the first manager unit is available, the first manager unit receives the first display data.

10. The method according to claim 8, characterized in that Directly receiving second display data from the second user process includes: in response to a status indicator of a second manager unit indicating that the second manager unit is available, the second manager unit receives the second display data.

Images