Screen projection method in vehicle

Abstract

The invention provides a screen projection method in a vehicle, a vehicle-mounted main control screen and a vehicle-mounted instrument screen are arranged in the vehicle, the vehicle-mounted main control screen is used for providing a user cabin main operation interface based on an Android system, and the vehicle-mounted instrument screen is used for providing instrument data based on a Linux system. The method comprises the following steps: monitoring whether a screen projection request from an Android system exists or not; in response to the existence of the screen projection request, starting a working queue, and controlling the working queue to obtain a target layer attribute from driving private data of a display processing unit; and constructing primary display output based on the target layer attribute. According to the method and the device, the fluency of the projection screen image in the Linux container scene can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of vehicles, and in particular to a screen projection method in a vehicle. Background Art

[0002] In the vehicle system scenario, the vehicle instrument screen usually runs the Linux system, and the main control screen uses the Android system. For the vehicle system configured with the map projection function, when performing map navigation, the navigation information on the Android system (for example, maps and turn instructions, etc.) needs to be projected onto the vehicle instrument screen, that is, the map and the video data of the instrument layer are displayed together on the vehicle instrument screen. Since the vehicle instrument screen is generally set in front of the driver, it is more convenient for the driver to observe the navigation information after projection.

[0003] In the container scenario, when the map layer of the Android system is projected to the Linux system for display, since the Android system and the Linux system are equivalent to two processes that use the DPU (display process unit) to send and output respectively, the system scheduler will use the Android system's map layer process and the Linux system's instrument layer process to send and display in different time periods. Since the DPU's operating frequency is fixed, the display frame rate of the Android system and the Linux system is reduced, thereby affecting the smoothness of the projected image. Summary of the invention

[0004] In view of this, an embodiment of the present application provides at least one screen projection method in a vehicle to overcome at least one of the above-mentioned defects.

[0005] In a first aspect, an exemplary embodiment of the present application provides a screen projection method in a vehicle, wherein the vehicle is provided with an on-board main control screen and an on-board instrument screen, wherein the on-board main control screen is used to provide a user cockpit main operation interface based on an Android system, and the on-board instrument screen is used to provide instrument data based on a Linux system, and the method comprises: monitoring whether there is a screen projection request from the Android system; in response to the existence of the screen projection request, starting a work queue, and controlling the work queue to obtain target layer attributes from the driver private data of the display processing unit; and constructing a display output based on the target layer attributes.

[0006] In one possible implementation, the method also includes: setting a first interface identifier for the screen projection layer under the Android system, and setting a second interface identifier for the instrument layer under the Linux system, wherein the step of monitoring whether there is a screen projection request from the Android system includes: obtaining a display request monitored by a designated interface of the DRM, the display request carrying an interface identifier, and in response to the interface identifier carried in the display request being a first interface identifier, determining that a screen projection request from the Android system is monitored.

[0007] In one possible implementation, the display sending application includes a first display sending application from an Android system and / or a second display sending application from a Linux system, wherein the first display sending application includes multiple first layer attributes of a projection layer, and the second display sending application includes multiple second layer attributes of an instrument layer, wherein the multiple first layer attributes and / or the multiple second layer attributes are temporarily stored in the driver private data of the display processing unit.

[0008] In a possible implementation manner, the method further includes: in response to the first display sending application, constructing a plurality of first data items in the Linux kernel space of the display processing unit, wherein the number of the plurality of first data items is consistent with the number of the plurality of first layer attributes, and each first data item includes a first uint32 array and a first uint64 array, the first uint32 array is used to store an ID of a first layer attribute, and the first uint64 array is used to store an attribute value of a first layer attribute; temporarily storing the constructed plurality of first data items in the driver private data of the display processing unit, and / or, the method further includes: in response to the second display sending application, constructing a plurality of second data items in the Linux kernel space of the display processing unit, wherein the number of the plurality of second data items is consistent with the number of the plurality of second layer attributes, and each second data item includes a second uint32 array and a second uint64 array, the second uint32 array is used to store an ID of a second layer attribute, and the second uint64 array is used to store an attribute value of a second layer attribute; temporarily storing the constructed plurality of second data items in the driver private data of the display processing unit.

[0009] In one possible implementation, based on the target layer attributes, the step of constructing a display output includes: updating the multiple first data items and / or the multiple second data items into a common structure of the display processing unit to form display content for a display output through the common structure.

[0010] In one possible implementation, the target layer attributes include multiple first layer attributes of the projection layer under the Android system and / or multiple second layer attributes of the instrument layer under the Linux system, wherein the method further includes: using a flag bit to record the target source to which the acquired target layer attributes belong, different target sources correspond to different flag bits; after completing the display output, reading the flag bit; and notifying the current frame of the display completion amount based on the read flag bit.

[0011] In one possible implementation, the method further includes: after completing the display output, in response to reaching a delay time, controlling the work queue to obtain the driver private data of the display processing unit again, wherein the length of the delay time is adjusted according to the requirements of the upper-layer business frame rate.

[0012] In one possible implementation, the method further includes: after the work queue is started, starting a counter, the counter being used to record the number of times the designated interface of DRM monitors a second display sending request from the Linux system, wherein when the second display sending request is continuously monitored, the counter accumulates the count, and when the first display sending request from the Android system is monitored, the counter is cleared; determining whether the number of times recorded by the counter reaches a counting threshold as a basis for determining the end of screen projection; and exiting the work queue in response to reaching the counting threshold.

[0013] In one possible implementation, the screen projection request from the Android system is generated in the following manner: the user cockpit main operation interface is displayed on the vehicle-mounted main control screen, the user cockpit main operation interface includes a navigation logo, and in response to a selection operation on the navigation logo, a map interface for vehicle navigation is displayed on the vehicle-mounted main control screen, the map interface includes a screen projection button, and in response to a selection operation on the screen projection button, the screen projection request is generated, and / or the first display sending application exists when the screen projection request is triggered, the second display sending application always exists, and / or the Android system runs in an LXC container.

[0014] In one possible implementation, the display output includes multiple first layer attributes of the projection layer under the Android system and multiple second layer attributes of the instrument layer under the Linux system, the projection layer includes a map navigation layer in the Android system, and the display processing unit is configured with a first hardware layer corresponding to the projection layer and a second hardware layer corresponding to the instrument layer, wherein the method also includes: according to the display output, controlling the vehicle instrument screen to simultaneously display the display content of the instrument layer and the map navigation layer, wherein the first hardware layer is used to display the display content of the projection layer on the vehicle instrument screen based on the multiple first layer attributes in the display output, and the second hardware layer is used to display the display content of the instrument layer on the vehicle instrument screen based on the multiple second layer attributes in the display output.

[0015] The screen projection method in a vehicle provided in this application can effectively improve the smoothness of the screen projection in the Linux container scenario.

[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram showing a display structure in a vehicle provided by an exemplary embodiment of the present application;

[0019] Figure 2 A flow chart showing a screen projection method in a vehicle provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0021] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0022] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "Including A, B and / or C" means including any one, any two, or any three of A, B, and C.

[0023] It should be understood that in the embodiments of the present application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined according to A. Determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0024] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0025] In the vehicle system scenario, the vehicle instrument screen usually runs the Linux system, and the main control screen uses the Android system. For the vehicle system configured with the map projection function, when performing map navigation, the navigation information on the Android system (for example, maps and turn instructions, etc.) needs to be projected onto the vehicle instrument screen, that is, the map and the video data of the instrument layer are displayed together on the vehicle instrument screen. Since the vehicle instrument screen is generally set in front of the driver, it is more convenient for the driver to observe the navigation information after projection.

[0026] In the container scenario, when the map layer of the Android system is projected to the Linux system for display, since the Android system and the Linux system are equivalent to two processes that use the DPU (display process unit) for display output respectively, the system scheduler will time-share the Android system's map layer process and the Linux system's instrument layer process for display output. Since the DPU's operating frequency is fixed, the display frame rate of the Android system and the Linux system is reduced. For example, the display frame rate of the Android system and the Linux system is only about half of that of the exclusive use of the DPU for display output.

[0027] For Android systems with multiple displays, since its display process is to display multiple displays synchronously at one time, and then display the next frame on multiple screens after the multi-screen display is completed, the display frame rate of the entire Android system is about half of the original frame rate when the map is projected.

[0028] In response to at least one of the above problems, the present application proposes a solution to ensure the frame rate of the Android system's screen projection and display in a Linux container scenario, which can at least solve the problem of the DPU being used in a time-sharing manner by the Android system's screen projection layer and the Linux system's instrument layer.

[0029] To facilitate understanding of the present application, the specific process of the screen projection method in the vehicle provided in the embodiment of the present application is introduced in detail below.

[0030] First, the names involved in the embodiments of the present application are introduced.

[0031] Virtualization is a widely used technology. Through virtualization technology, independent client systems (such as Windows, Linux, Android, etc.) can be run in the host operating system (such as Windows, Linux).

[0032] Container (Linux Container, LXC) is a lightweight virtualization technology supported by the Linux kernel in recent years. Through container technology, a new Linux system (including other systems customized based on the Linux system, such as Android) can be run on the Linux system. LXC implements namespaces such as processes, file systems, IPC, network communications, and users, so that the client system and the host system are isolated from each other. Unlike traditional solutions based on virtual machine technology, in container technology, the client system and the host system share resources such as CPU, memory, and storage, and the resource overhead is very low. In addition, through proper configuration, the client system can also directly access the hardware resources of the host, further improving the performance of the client system.

[0033] DPU (display process unit) is a display controller used to transfer image data in the video memory to the external LCD interface. These LCD interfaces include eDP, DP, MIPI, HDMI, etc. Depending on the configuration, the DPU can output images to an external interface alone or to multiple external interfaces at the same time. There are multiple hardware layers inside the DPU to save the image data required by different output interfaces, and one hardware layer corresponds to the image data of a video data stream. According to the configuration, there is a certain mapping relationship between the hardware layer and the output interface, and multiple layers can be mapped to one output interface. In the process of DPU outputting images, operations such as color format conversion and image size change are also supported. In mobile SOCs such as the ARM architecture, the GPU is mainly responsible for image rendering, and the DPU is responsible for the display output of video image data.

[0034] DRM (Direct Rendering Manager, digital rights management) is the mainstream image display framework in Linux systems. DRM is very adaptable to modern hardware and supports GPU, DPU, 3D rendering display, etc. DRM can uniformly manage GPU and Display drivers, making the software architecture more unified and convenient for development and maintenance.

[0035] Combine the following Figure 1 and Figure 2 To introduce the specific processing process of the screen projection solution of this application.

[0036] Figure 1 A schematic diagram showing a display architecture in a vehicle provided by an exemplary embodiment of the present application.

[0037] like Figure 1As shown, the display architecture includes an LXC container and a user cockpit main operation interface Android system 10, a Linux instrument layer weston process 20, and a Linux kernel space DPU driver. In the embodiment of the present application, a vehicle is provided with an on-board main control screen and an on-board instrument screen. The on-board main control screen is used to provide a user cockpit main operation interface based on the Android system, and the on-board instrument screen is used to provide instrument data based on the Linux system.

[0038] In this example, the Android system runs in an LXC container, and the LXC container runs in the Linux operating system. There are two processes that use the DPU for display, one of which is the process that draws the projection layer from Android (taking the projection request initiated by the navigation app as an example, the projection layer can be a map layer), and the other is the weston (wayland compositer) process that draws the instrument layer from the main operating interface of the Linux user cockpit. The DPU driver runs in the Linux kernel, and the Android projection layer drawing process and the weston process share the same DPU driver.

[0039] In a preferred example, each layer can be distinguished in software using a plane id, and at the same time, there will be a corresponding hardware layer corresponding to it on the DPU hardware. That is, a hardware layer has a corresponding display output buffer corresponding to the plane id. In an embodiment of the present application, a first interface identifier is set for the projection layer under the Android system (a plane id is set), and a second interface identifier is set for the instrument layer under the Linux system (another plane id is set). Accordingly, the display processing unit DPU is configured with a first hardware layer corresponding to the projection layer and a second hardware layer corresponding to the instrument layer.

[0040] Figure 2 A flow chart showing a screen projection method in a vehicle provided by an exemplary embodiment of the present application is shown.

[0041] like Figure 2 As shown, in step S101, it is monitored whether there is a screen projection request from the Android system.

[0042] In an optional example of the present application, the above-mentioned screen projection request can be triggered based on the user's operation.

[0043] For example, a screen projection request from the Android system can be generated in the following manner: the user cockpit main operation interface of the Android system is displayed on the vehicle main control screen, and the user cockpit main operation interface may include but is not limited to a navigation logo; in response to a selection operation on the navigation logo, a map interface for vehicle navigation is displayed on the vehicle main control screen, and the map interface may include but is not limited to a screen projection button; in response to a selection operation on the screen projection button, a screen projection request is generated.

[0044] Combination Figure 1 The software module diagram of the Android container 10 shown in the figure introduces the process of forming a screen projection request under the Android system.

[0045] For example, taking the projection layer as a map layer, the Android system will draw the content of the map layer and save it in a buffer, and then call the atomic ioctl commit interface of libdrm through the Surface Flinger layer and the HWC (hardware compositer) layer.

[0046] Here, libdrm is a user space library used to access DRM devices on operating systems that support the ioctl interface. It is a wrapper based on kernel DRM and provides a more user-friendly API interface.

[0047] In this case, whether there is a screen projection request from the Android system can be monitored in the following way: obtain the display sending application monitored by the specified interface of DRM (atomic ioctl commit interface), the display sending application carries an interface identifier, and in response to the interface identifier carried in the display sending application being the first interface identifier, determine that a screen projection request from the Android system is monitored.

[0048] In step S102, in response to the presence of a screen projection request, a work queue is started, and the work queue is controlled to obtain target layer properties from the driver private data of the display processing unit.

[0049] Here, when the vehicle system is started, the screen projection service does not exist by default, and the worker queue will not run in this default scenario. Only when the screen projection service is turned on will the worker be scheduled to merge the display data of the Android system's screen projection layer and the Linux system's instrument layer.

[0050] For example, the Worker needs to be started according to the parameter information in the atomic ioctl commit. For example, the plane id of the projection layer of the Android system and the instrument layer of the Linux system can be determined in the device tree DTS, and the plane id is used in the atomic ioctl commit.

[0051] In the atomic ioctl commit processing flow driven by the DPU, if it is monitored that the object id of this commit display update is the plane id of the projection layer, the atomic ioctl commit starts the worker and leaves the subsequent display of the projection layer and the instrument layer to the worker.

[0052] In the embodiment of the present application, the display sending application monitored by the atomic ioctl commit interface of DRM may include the first display sending application (i.e., screen projection request) from the Android system and / or the second display sending application from the Linux system, that is, the object monitored for this commit display sending update may be only the screen projection layer from the Android system, or only the instrument layer from the Linux system, or may include both the screen projection layer from the Android system and the instrument layer from the Linux system. Exemplarily, the first display sending application exists when the screen projection request is triggered, and the second display sending application always exists.

[0053] Here, the first display application includes multiple first layer attributes of the projection layer, and the second display application includes multiple second layer attributes of the instrument layer. The multiple first layer attributes and / or multiple second layer attributes are temporarily stored in the driver private data of the display processing unit DPU.

[0054] Exemplarily, the plurality of second layer attributes may include, but are not limited to: information such as vehicle speed and remaining power displayed by the vehicle instrument screen for the Linux system.

[0055] Combination Figure 1 The software module diagram of the Linux user space 20 shown in the figure is used to introduce the formation process of the second display application under the Linux system.

[0056] For example, the Linux system draws the contents of the instrument layer by calling the atomic ioctl commit interface of libdrm through the wayland protocol via the wayland compositer (weston). Since the instrument display information needs to exist all the time during the operation of the vehicle system, the operation of calling the atomic ioctl commit interface for the instrument layer to send it to the display always exists, and its calling frequency (that is, the layer refresh frame rate) is not fixed according to the upper-layer business.

[0057] In the atomic ioctl commit interface of DRM, the layer properties of the projection layer of the Android system are passed from the user space of the Android system to the kernel space. Here, the atomic ioctl commit of DRM is a mechanism used in the Linux kernel to manage display devices, mainly used to submit and manage configuration changes of display devices. Atomicioctl commit is a key function in the DRM framework, which allows user space applications to submit a series of display configuration changes in an atomic manner, ensuring that the state of the display device remains consistent during the submission process.

[0058] In the first embodiment, the following data storage process may be performed for the first display application.

[0059] In response to the first display transmission application, a plurality of first data items are constructed in the Linux kernel space of the display processing unit, and the constructed plurality of first data items are temporarily stored in the driver private data of the display processing unit.

[0060] Here, the number of the above-mentioned multiple first data items is consistent with the number of the multiple first layer attributes, and the layer attributes transferred to the kernel space are stored by a uint32 array and a uint64 array.

[0061] Exemplarily, each first data item includes a first uint32 array and a first uint64 array, wherein the first uint32 array is used to store the ID of a first layer attribute, and the first uint64 array is used to store the attribute value of the first layer attribute in the corresponding first uint32 array.

[0062] Exemplarily, the layer properties may include, but are not limited to, at least one of the following items: SRC_X (current buffersourcecrop_x coordinate, i.e., the X coordinate of the source image cropping area of ​​the projection layer), SRC_Y (current buffersourcecrop_y coordinate, i.e., the Y coordinate of the source image cropping area of ​​the projection layer), SRC_W (current buffersourcecrop width, i.e., the width of the source image cropping area of ​​the projection layer), SRC_H ​​(current buffersourcecrop height, i.e., the height of the source image cropping area of ​​the projection layer), CRTC_X (X coordinate of the screen display area of ​​the projection layer), CRTC_Y (Y coordinate of the screen display area of ​​the projection layer), CRTC_W (width of the screen display area of ​​the projection layer), CRTC_H (height of the screen display area of ​​the projection layer), FB_ID (FrameBuffer Object ID bound to the current plane), CRTC_ID (CRTC ID associated with the current plane), rotation (rotation angle of the current layer), alpha (global alpha of the current layer), and zposition (z-order of the current layer).

[0063] In the embodiment of the present application, the above-mentioned stored layer attributes are all integer parameters, which can be conveniently temporarily stored in the driver private data of the DPU. Here, the layer attributes of the instrument layer can be the same as the layer attributes of the projection layer, or they can be different. For example, the layer attributes of the instrument layer can also include at least one of the above items, which will not be described in detail in this application.

[0064] In a preferred example, two arrays of uint32 and uint64 are used in the DPU driver to temporarily store the layer attributes passed by the user space. Since these temporarily stored layer attributes will be used by the worker work queue in the future, a mutex lock can be used for read-write protection.

[0065] For example, the Android process executes the operation of writing layer attributes, and the work queue executes the operation of reading layer attributes. The introduction of a mutex lock can ensure that only one operation can be performed on the data stored in the DPU driver private data. For example, when performing a write operation, a read operation cannot be performed, and correspondingly, when performing a read operation, a write operation cannot be performed.

[0066] After the operation of temporarily storing the layer properties of the projection layer is completed, in order to ensure that the semantics of the display process seen by the Android upper-layer user program does not change and to reduce the complexity of the entire process, the completion amount can be used to wait for the worker to obtain the temporarily stored layer properties and display them. After the display is completed, the worker will notify the completion amount to inform the projection layer process to continue the atomic ioctl commit subsequent operations and then return to the Android user space.

[0067] In the second embodiment, the following data storage process may be performed for the second display application.

[0068] In response to the second display transmission application, a plurality of second data items are constructed in the Linux kernel space of the display processing unit, and the constructed plurality of second data items are temporarily stored in the driver private data of the display processing unit.

[0069] Here, the number of the above-mentioned multiple second data items is consistent with the number of multiple second layer attributes. In the atomic ioctl commit interface of DRM, the layer attributes of the instrument layer are passed from the Linux user space to the kernel space. The layer attributes are saved by a uint32 array and a uint64 array.

[0070] Exemplarily, each second data item includes a second uint32 array and a second uint64 array, wherein the second uint32 array is used to store the ID of a second layer attribute, and the second uint64 array is used to store the attribute value of the second layer attribute in the corresponding second uint32 array.

[0071] Preferably, a uint32 array and a uint64 array are used in the DPU driver to store the data, and a mutex lock is used for read-write protection.

[0072] After the operation of temporarily storing the layer attributes of the instrument layer is completed, the completion amount can be used to wait for the worker to obtain the layer attributes and send them for display. After the display is completed, the worker will notify the completion amount to inform the instrument layer process to continue the atomicioctl commit subsequent operations and then return to the Linux user space.

[0073] In a preferred embodiment of the present application, after the work queue is started, the exit of the work queue can also be controlled by monitoring the number of second display requests from the Linux system.

[0074] For example, after the work queue is started, the counter is started to determine whether the number of times recorded by the counter reaches the counting threshold, which is used as the basis for determining the end of screen projection. Among them, when the number of times recorded by the counter reaches the counting threshold (for example, greater than or equal to the counting threshold), it indicates that the Android system has not initiated a screen projection request for a certain period of time. At this time, it can be determined that the screen projection is over, and the control exits the work queue to display in the default scenario.

[0075] When the number of times recorded by the counter does not reach the counting threshold (for example, less than the counting threshold), it indicates that the Android system continues to initiate screen projection requests. At this time, it can be determined that the screen projection has not ended, and the work queue continues to obtain the target layer properties from the driver private data according to the delay time.

[0076] Here, the above-mentioned counter is used to record the number of times that the designated interface of DRM monitors the second display request from the Linux system, wherein when the second display request is continuously monitored, the counter counts cumulatively, and when the first display request is monitored from the Android system, the counter is cleared.

[0077] For example, the counter in the DPU driver records the number of times only the Linux instrument layer is committed. If there is only a commit to the Linux instrument layer, the counter will continue to increase. If there is a commit to the Android map layer during the accumulation process, the counter will be cleared. If the counter does not reach the counting threshold, it means that the screen projection function is not turned off. At this time, the worker is continuously used to merge layer attributes and display. If the counter reaches the counting threshold, it means that the upper layer has turned off the screen projection function, and queue_delayer_work() is no longer used to schedule the worker, that is, the worker is exited and returned to the original conventional DRM display mode.

[0078] As mentioned above, the display sending request monitored by the designated interface of DRM may include a first display sending request from the Android system and / or a second display sending request from the Linux system. In this case, the target layer attributes obtained by the work queue from the driver private data of the DPU may include multiple first layer attributes of the projection layer under the Android system and / or multiple second layer attributes of the instrument layer under the Linux system.

[0079] In a preferred embodiment, a flag bit can be used to record the target source to which the acquired target layer attribute belongs (eg, to record which layer attribute is acquired), and different target sources correspond to different flag bits.

[0080] Here, in addition to setting different flag bits, different values ​​can be assigned to a flag bit to characterize the target source to which the target layer attribute belongs based on different values. This application does not impose any restrictions on this.

[0081] In step S103, a display output is constructed based on the target layer attributes.

[0082] After the worker is started, it can periodically obtain the layer properties of the temporarily stored projection layer and / or instrument layer from the driver privatedata driven by the DPU according to the delay time, and then the worker merges the first layer properties of the projection layer from the Android system and the second layer properties of the instrument layer from the Linux system and sends them to the hardware for display.

[0083] Here, the temporarily stored layer attributes are the layer attributes from the Android / Linux user space temporarily stored in the driver private data of the display processing unit. Since the refresh frame rate of the Android map layer and the Linux instrument layer is not fixed according to the changes in the upper-layer business, in a worker's operation of obtaining layer attributes, it is possible to obtain only the layer attributes of the instrument layer, or only the layer attributes of the projection layer, or both layer attributes, or none of them. In this case, you can use the flag bit in the worker to record which layer attribute is obtained.

[0084] In an embodiment of the present application, multiple first data items and / or multiple second data items can be updated to a common structure of a display processing unit to form display content that is output once through the common structure.

[0085] It should be understood that the present application is not limited to this. The processing of updating multiple first data items and / or multiple second data items to the common structure of the display processing unit can also be executed in the above step S102. At this time, in step S103, the layer properties of the projection layer and / or the instrument layer can be directly obtained from the common structure.

[0086] Here, the atomic ioctl commit of the DRM display driver framework has the following characteristics: it can send the update of a single layer at a time, or it can send the update of multiple layers at a time. Since each layer has an independent hardware layer corresponding to it on the DPU hardware, the contents of multiple layers can not interfere with each other when committing. Based on this, the sending and display of the respective layer data of the original two commits are merged into one commit. In this way, the original system scheduler can be modified from using the DPU in time-sharing to the worker's exclusive use of the DPU, thereby improving the display frame rate during screen projection. Due to the uncertainty of the upper-layer service sending and display frame rate, when the worker obtains the temporary layer attributes, it may only obtain the layer attributes of one of the projection layer or instrument layer. In this scenario, when the worker commits and sends, only one layer attribute of the map layer or instrument layer is updated to the drm_atomic_state structure, and the updated content of one layer is sent and displayed.

[0087] In an embodiment of the present application, for the case where a single display output includes multiple first layer attributes of the projection layer under the Android system and multiple second layer attributes of the instrument layer under the Linux system, taking the projection layer including the map navigation layer in the Android system as an example, the vehicle-mounted instrument screen can be controlled to simultaneously display the display contents of the instrument layer and the map navigation layer based on the display output.

[0088] For example, a first hardware layer is used to display the display content of the projection layer on the vehicle instrument screen based on multiple first layer attributes in the display output, and a second hardware layer is used to display the display content of the instrument layer on the vehicle instrument screen based on multiple second layer attributes in the display output, so that the vehicle instrument screen can simultaneously present the display content of the instrument layer and the map navigation layer.

[0089] In a preferred embodiment of the present application, after the display output is completed, the flag is read, and based on the read flag, it is notified whether the display of the current frame is completed. For example, after the call to the synchronous display interface returns, the worker will notify Android and / or Linux of the completion amount based on the read flag. Exemplarily, if the flag corresponding to the projection layer is read, it means that the display is completed. If the flag corresponding to the projection layer is not read, it means that the display is not completed, and no notification may be sent at this time. Here, the method for determining whether the display of the instrument layer is completed is similar to that of the projection layer, and this application will not go into details.

[0090] For example, there is a flag bit that records the type of layer attributes obtained. Therefore, when notifying the completion amount, only the layer commit recorded by the flag bit can be notified. For example, only the update of the projection layer is recorded by the flag, then when the completion amount is notified, only the process waiting for the completion amount is woken up and notified. Since the original Linux instrument layer uses the wayland compositer for asynchronous display, a drm event and drm fence will be constructed before display. Therefore, according to the recorded flag, when there is an instrument layer display commit, the worker also needs to construct the corresponding drm event and drm fence when displaying.

[0091] Above Figure 2 The screen projection scheme shown is a screen projection processing process for a display frame rate. After completing the display output of one frame, the following processing can continue: in response to the delay time being reached, the control work queue is controlled to obtain the driver private data from the display processing unit again to perform the display processing of the next frame.

[0092] Here, the data stored in the driver private data can be updated based on the display frame rate of the projection layer and / or the instrument layer, or can also be updated according to the amount of display completion. For example, when it is determined that the display of a layer is completed, the layer properties of the layer temporarily stored in the driver private data are deleted.

[0093] In a preferred embodiment of the present application, after a worker completes display delivery, the Linux kernel interface queue_delayer_work() is used to delay for a fixed time, and then the worker is scheduled again to obtain layer attributes and display delivery. Exemplarily, the duration of the above delay time can be adjusted according to the frame rate requirements of the upper-layer business. For example, when the upper-layer business requires a frame rate of 60fps, the duration of a complete display delivery should be 16.6ms (milliseconds). At this time, it can be determined based on the upper-layer business scenario experimental verification that the duration of the delay time is 12ms, which is more reasonable. This can ensure that each worker submission has layer attributes that need to be updated. However, it should be understood that the above-listed values ​​are only examples, and those skilled in the art can adjust the above values ​​according to the actual display frame rate requirements, and the present application does not limit this.

[0094] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A screen projection method in a vehicle, characterized in that: The vehicle is provided with a vehicle-mounted main control screen and a vehicle-mounted instrument screen, wherein the vehicle-mounted main control screen is used to provide a user cockpit main operation interface based on an Android system, and the vehicle-mounted instrument screen is used to provide instrument data based on a Linux system, and the method comprises: Monitor whether there is a screen projection request from the Android system; In response to the screen projection request, start a work queue, and control the work queue to obtain target layer properties from driver private data of the display processing unit; Based on the target layer attributes, construct a display output.

2. The method according to claim 1, characterized in that The method further comprises: Set the first interface logo for the projection layer under the Android system, and set the second interface logo for the instrument layer under the Linux system. The steps of monitoring whether there is a screen projection request from the Android system include: Obtain the display application monitored by the designated interface of the DRM, wherein the display application carries an interface identifier. In response to the interface identifier carried in the display application being the first interface identifier, it is determined that a screen projection request from the Android system is monitored.

3. The method according to claim 2, characterized in that The display sending application includes a first display sending application from an Android system and / or a second display sending application from a Linux system, the first display sending application includes a plurality of first layer attributes of a projection layer, and the second display sending application includes a plurality of second layer attributes of an instrument layer, The plurality of first layer attributes and / or the plurality of second layer attributes are temporarily stored in the driver private data of the display processing unit.

4. The method according to claim 3, characterized in that The method further comprises: In response to the first display transmission application, a plurality of first data items are constructed in the Linux kernel space of the display processing unit, wherein the number of the plurality of first data items is consistent with the number of the plurality of first layer attributes, and each first data item includes a first uint32 array and a first uint64 array, wherein the first uint32 array is used to store an ID of a first layer attribute, and the first uint64 array is used to store an attribute value of a first layer attribute; Temporarily storing the constructed multiple first data items in the driver private data of the display processing unit, And / or, the method further comprises: In response to the second display transmission application, a plurality of second data items are constructed in the Linux kernel space of the display processing unit, wherein the number of the plurality of second data items is consistent with the number of the plurality of second layer attributes, and each second data item includes a second uint32 array and a second uint64 array, the second uint32 array is used to store an ID of a second layer attribute, and the second uint64 array is used to store an attribute value of a second layer attribute; The constructed multiple second data items are temporarily stored in the driver private data of the display processing unit.

5. The method according to claim 4, characterized in that Based on the target layer attributes, the steps of constructing a display output include: The plurality of first data items and / or the plurality of second data items are updated into a common structure of a display processing unit, so as to form display content outputted at one time through the common structure.

6. The method according to claim 1, characterized in that The target layer attributes include multiple first layer attributes of the projection layer in the Android system and / or multiple second layer attributes of the instrument layer in the Linux system. Wherein, the method further comprises: Use the flag bit to record the target source to which the acquired target layer attribute belongs. Different target sources correspond to different flag bits. After completing the display output, reading the flag bit; According to the flag bit read, the completion amount of the display of the current frame is notified.

7. The method according to claim 1, characterized in that The method further comprises: After the display output is completed, in response to reaching the delay time, the work queue is controlled to obtain the driver private data of the display processing unit again, The duration of the delay time is adjusted according to the requirement of the upper layer service frame rate.

8. The method according to claim 3, characterized in that The method further comprises: After the work queue is started, a counter is started, the counter is used to record the number of times the designated interface of the DRM monitors the second display sending application from the Linux system, wherein when the second display sending application is continuously monitored, the counter counts cumulatively, and when the first display sending application from the Android system is monitored, the counter is cleared; Determine whether the number of times recorded by the counter reaches a counting threshold, so as to serve as a basis for determining the end of screen projection; In response to reaching the count threshold, the work queue is exited.

9. The method according to claim 3, characterized in that: Screen projection requests from the Android system are generated in the following ways: The main operation interface of the user cockpit is displayed on the vehicle main control screen, and the main operation interface of the user cockpit includes a navigation mark. In response to the selection operation for the navigation mark, a map interface for vehicle navigation is displayed on the vehicle main control screen, and the map interface includes a projection button. In response to a selection operation on the screen projection button, generating the screen projection request, and / or, the first display sending application exists when the screen projection request is triggered, and the second display sending application always exists, And / or, the Android system runs in an LXC container.

10. The method according to claim 1, characterized in that The display output includes a plurality of first layer attributes of the projection layer under the Android system and a plurality of second layer attributes of the instrument layer under the Linux system, the projection layer includes a map navigation layer in the Android system, and the display processing unit is configured with a first hardware layer corresponding to the projection layer and a second hardware layer corresponding to the instrument layer, Wherein, the method further comprises: According to the display output, the vehicle-mounted instrument screen is controlled to simultaneously display the display contents of the instrument layer and the map navigation layer, wherein a first hardware layer is used to display the display contents of the projection layer on the vehicle-mounted instrument screen based on multiple first layer attributes in the display output, and a second hardware layer is used to display the display contents of the instrument layer on the vehicle-mounted instrument screen based on multiple second layer attributes in the display output.

Images