Rendering methods, media, and electronic devices

Abstract

The present application relates to a method for rendering a picture, a medium and an electronic device. The rendering method is applied to an electronic device including a first operating system and a second operating system, wherein the second operating system is the main system of the electronic device, and the method comprises: the first operating system performs a rendering operation of the picture; the second operating system obtains a synchronization signal of the electronic device and sends it to the first operating system, wherein the synchronization signal is used to synchronize the execution of the rendering operation of the picture and the display of the picture by the electronic device; in response to the synchronization signal, the first operating system notifies the second operating system to trigger the screen display of the electronic device. The method of the present application can synchronize the rendering frequency of the rendering operation of the picture performed by the first operating system with the synchronization signal of the second operating system, thereby solving the problem of the picture being stuck when the first operating system displays the picture on the screen of the electronic device through the second operating system.

Description

Technical Field

[0001] The present application relates to virtualization technology in the field of operating systems, and more particularly to a method for rendering a picture, a medium, and an electronic device. Background Art

[0002] The screen of an electronic device has a refresh rate, which indicates the number of times the screen updates the image per second. In an electronic device installed with Android or Linux system, the operating system can obtain the vsync signal (Vertical Synchronization) corresponding to the refresh rate (for example, if the refresh rate is 60 frames per second, then the vsync signal = 1000 / 60 = 16 milliseconds).

[0003] For an application installed on an electronic device, if the application is not stuck, the process of displaying the application interface on the screen of the electronic device is as follows: Figure 1 As shown, the electronic device of the Android system obtains the first vsync signal of the screen. During the duration of the first vsync signal, the graphics processor (GPU) of the electronic device completes the rendering of the display interface of the application to generate a first image frame. When the electronic device obtains the second vsync signal, the operating system displays the first image frame on the screen. At the same time, the GPU can render and generate a second image frame within the duration of the second vsync signal. When the electronic device obtains the third vsync signal, the second image frame is displayed on the screen. At the same time, the GPU renders and generates the third image frame. This ensures that the GPU rendering display interface and the screen display display interface work according to the rhythm of the vsync signal.

[0004] In the field of Linux and Android dual-system integration, a technical solution is usually adopted in which Linux is the primary system and Android is the secondary system. That is, Android is installed on an electronic device through virtualization. In this case, the display interface of the application installed on the Android side needs to be rendered by the GPU on the Linux side and displayed on the electronic device screen.

[0005] In the above technical solution, in the process of rendering the display interface of the application installed on the Android side and displaying the display interface by the GPU on the Linux side, since the vsync signal synchronization is not achieved between the Android side and the Linux side, the Android side cannot obtain the vsync signal on the Linux side, so that the rendering of the display interface of the application and the display of the display interface of the application on the screen of the electronic device are in a state without hardware calibration. At the same time, the Android side needs to send the interface data and rendering instructions of the display interface to the Linux side. However, within the duration of a vsync signal, the amount of data and instructions that can be transmitted between the Android side and the Linux side is limited. If the display interface of the application contains a large amount of data, the process of sending the display interface data and rendering instructions from the Android side to the Linux side will produce a high delay. This will cause the Linux side to render and generate the display interface and also produce delays, which is prone to frame loss and reduces the user experience. Summary of the Invention

[0006] The purpose of this application is to provide a method for rendering a picture, a medium and an electronic device. Through the method of this application, when an electronic device is installed with dual systems (such as a Linux system and an Android system), it can be ensured that the generation and display rhythm of the application display interface between the Linux and Android dual systems are consistent, and there will be no frame loss when displaying the application display interface due to delay.

[0007] A first aspect of the present application provides a method for rendering a screen, which is applied to an electronic device including a first operating system and a second operating system, wherein the second operating system is the main system of the electronic device, and the method includes: the first operating system executes a screen rendering operation; the second operating system obtains a synchronization signal of the electronic device and sends it to the first operating system, wherein the synchronization signal is used to maintain synchronization between the execution of the screen rendering operation and the display of the screen by the electronic device; in response to the synchronization signal, the first operating system notifies the second operating system to trigger the screen display of the electronic device.

[0008] That is, in an embodiment of the present application, the electronic device may be a vehicle computer, the second operating system may be a Linux system, and the first operating system may be an Android system. That is, the Android system is installed in a virtualized manner on a vehicle computer installed with a Linux system. The synchronization signal of the electronic device may be a vsync signal of the vehicle computer, which is used to indicate the refresh rate of the vehicle computer screen. When the rendering frequency of the screen rendering operation executed by the Android system is synchronized with the vsync signal of the Linux system, the Android system displays the screen on the vehicle computer screen through the Linux system.

[0009] In a possible implementation of the first aspect above, the first operating system determines at least one storage area in a storage component of the electronic device, and the first operating system obtains the address, size and storage format of the storage area, where the storage area is used to store screen display data.

[0010] That is, in the embodiment of the present application, the storage component may be the memory of the electronic device, and the storage area may be a memory area. The first operating system may request a memory area in the memory of the electronic device through the memory management service of the second operating system, and the second operating system returns the address, size, and storage format of the memory area to the first operating system.

[0011] In a possible implementation of the first aspect above, the first operating system performs a rendering operation of the screen, including: configuring a rendering instruction library in the first operating system, the rendering instruction library is used by the first operating system to control the rendering component of the electronic device; the first operating system sends a rendering instruction to the rendering component of the electronic device through the rendering instruction library; the rendering component of the electronic device performs the rendering operation of the screen in the storage area based on the rendering instruction.

[0012] That is, in the embodiment of the present application, the rendering instruction library here can be an interface of the rendering instruction of the GPU set in the first operating system. The first operating system can execute the rendering instruction through the rendering instruction library to call the GPU to perform the rendering operation of the picture.

[0013] In a possible implementation of the first aspect above, the rendering instruction includes the display data of the picture and the address, size, and storage format of the storage area.

[0014] That is, in the embodiment of the present application, the GPU performs a rendering operation of the picture in the storage area through the address of the storage area in the rendering instruction.

[0015] In a possible implementation of the first aspect above, the second operating system obtains a synchronization signal of the electronic device and sends it to the first operating system, including: the synchronization signal is a refresh frequency corresponding to the screen of the electronic device; the second operating system sends the synchronization signal to the first operating system through process communication.

[0016] That is, in an embodiment of the present application, an interface for sending a synchronization signal to the first operating system can be set in the synthesis management service of the second operating system, and the synthesis service of the first operating system can also obtain the synchronization signal of the second operating system through the above interface. The synchronization signal here can be the vsync signal of the second operating system.

[0017] In a possible implementation of the first aspect above, in response to a synchronization signal, the first operating system notifies the second operating system to trigger the screen display of the electronic device, including: when the first operating system receives the synchronization signal and the rendering operation of the screen of the first operating system is completed, the first operating system sends a display instruction to the second operating system; in response to the display instruction, the second operating system triggers the screen display of the electronic device within the duration of the synchronization signal.

[0018] That is, in an embodiment of the present application, if the first operating system has received a synchronization signal from the second operating system and determines that the rendering frequency of the image rendering operation performed by the first operating system is synchronized with the synchronization signal, the first operating system sends an image synthesis instruction to the second operating system, and the second operating system displays the image on the screen of the electronic device.

[0019] In a possible implementation of the first aspect, after the first operating system receives the synchronization signal, the rendering frequency of the screen rendering operation of the first operating system is adjusted to be synchronized with the synchronization signal.

[0020] That is, in an embodiment of the present application, when the first operating system determines that the rendering frequency of the screen rendering operation executed by the first operating system is not synchronized with the synchronization signal, the first operating system adjusts the rendering frequency to be synchronized with the synchronization signal of the second operating system.

[0021] In a possible implementation of the first aspect, adjusting the rendering frequency of the rendering operation of the screen of the first operating system to be synchronized with the synchronization signal includes: adjusting the rendering frequency to maintain frequency synchronization and time synchronization with the synchronization signal.

[0022] In a possible implementation of the first aspect, the display instruction includes an address of the picture located in a storage area.

[0023] In a possible implementation of the first aspect above, the synchronization signal includes at least a first synchronization signal and a second synchronization signal, the screen of the first operating system includes at least a first screen and a second screen, and the first synchronization signal is adjacent to the second synchronization signal.

[0024] In a possible implementation of the first aspect above, it includes: within the duration of the first synchronization signal, the first operating system completes the rendering operation of the first picture; within the duration of the second synchronization signal, the screen of the electronic device displays the first picture.

[0025] In a possible implementation of the first aspect above, the first operating system completes the rendering operation of the second picture within the duration of the second synchronization signal.

[0026] In a possible implementation of the first aspect, when the first operating system has not completed the rendering operation of the second picture within the duration of the second synchronization signal, the screen of the electronic device displays the first picture.

[0027] That is, in the embodiment of the present application, the first screen and the second screen may be the two screens whose display content changes in the display interface of the navigation application 200. After the rendering operation of the first screen is completed within the duration of the first synchronization signal, the first screen is displayed for the duration of the second synchronization signal; at the same time, the rendering operation of the second screen is performed for the duration of the second synchronization signal. If the rendering operation of the second screen cannot be completed within the duration of the second synchronization signal, the first screen continues to be displayed.

[0028] In a possible implementation of the first aspect above, the second operating system is configured to periodically send a synchronization signal to the first operating system, or the first operating system is configured to periodically obtain a synchronization signal from the first operating system.

[0029] In a possible implementation of the first aspect above, the rendering component is a GPU of the electronic device, and the storage component is a memory of the electronic device.

[0030] In a possible implementation of the first aspect above, the first operating system is an Android system, and the second operating system is a Linux system.

[0031] A second aspect of the present application provides a readable medium, characterized in that the readable medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the screen rendering method provided by the first aspect.

[0032] A third aspect of the present application provides an electronic device, characterized by comprising:

[0033] a memory for storing instructions to be executed by one or more processors of the electronic device, and

[0034] The processor is one of the processors of the electronic device, and is used to execute the image rendering method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The present invention shows a process in which a display interface of an application of an electronic device is displayed on a screen of the electronic device in the prior art;

[0036] Figure 2 According to an embodiment of the present application, a system architecture diagram of an electronic device with dual systems of Linux and Android is shown;

[0037] Figure 3A schematic structural diagram of a vehicle computer according to an embodiment of the present application;

[0038] Figure 4 According to an embodiment of the present application, a flowchart of displaying a display interface of a navigation application on a screen of a vehicle computer on a Linux side after a navigation application installed on an Android side is started is shown;

[0039] Figure 5 According to an embodiment of the present application, a flowchart of rendering the display interface of the navigation application on the Android side is shown;

[0040] Figure 6 (a) and 6(b) show an example of a display interface of a navigation application displayed on a screen of a vehicle according to an embodiment of the present application;

[0041] Figure 7 According to an embodiment of the present application, a schematic diagram showing an Android side applying for a memory area from a Linux side is shown;

[0042] Figure 8 According to an embodiment of the present application, a schematic diagram showing the structure of a composite management service on the Linux side, a display interface of a composite management service composite application, and a display interface on a screen of a vehicle computer is shown;

[0043] Figure 9 According to an embodiment of the present application, a system architecture diagram of another electronic device with dual systems of Linux and Android is shown;

[0044] Figure 10 According to an embodiment of the present application, a schematic diagram of displaying an application display interface on a screen of an electronic device installed with an Android system is shown;

[0045] Figure 11 According to an embodiment of the present application, a schematic diagram is shown of a display interface of an application displayed on a screen of an electronic device installed with a Linux system. DETAILED DESCRIPTION

[0046] The embodiments of the present application include but are not limited to a method for rendering a picture, a medium and an electronic device. To make the purpose, technical solution and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0047] To address the issues of data and command transmission delays between the Android and Linux sides, as well as the inconsistent timing between the rendering of the application's display interface and the display of the application's display interface, in the aforementioned Linux and Android dual systems, the present application provides a screen rendering solution. In the screen rendering solution of an embodiment of the present application, the Linux side establishes a channel for sending vsync signals to the Android side. Through this channel, the Linux side sends the electronic device's vsync signal to the Android side in real time. Simultaneously, the Android side requests a memory area on the Linux side that can be shared by the two sides. Next, the Android side is provided with a rendering instruction library that calls the Linux side's GPU. Using the rendering instruction library, the Android side directly uses the Linux side's GPU to render the display interface data within this memory area. After rendering is complete, the Android side notifies the Linux side to synthesize the display interface data into a display interface and display it. Finally, within the duration of the vsync signal, the Linux side synthesizes the rendered display interface data into the application's display interface and sends it to the electronic device's screen for display. The method provided in this application can ensure that the generation and display rhythm of the application display interface are consistent between the Linux and Android dual systems, and no frame loss will occur when displaying the application display interface due to delay.

[0048] In order to facilitate understanding of the technical solution of this application, the professional terms involved in the technical solution of this application are explained below.

[0049] The dual-system integration solution for Linux and Android involves installing the Android system within a container running on a Linux-based electronic device. The Linux system here can be called the master system, and the Android system can be called the slave system. The container hosting the Android system is equivalent to a process running on the Linux system. The container hosting the Android system can be a Linux container (LXC), a virtualization solution implemented using the Linux kernel's control group (Cgroup) and namespace technologies.

[0050] Control groups are used to limit and isolate processes' use of Linux system resources. For example, they can restrict Android users from directly accessing Linux resources, such as the device's GPU and memory.

[0051] Namespaces are used to encapsulate Linux system resources, allowing each namespace to have an independent copy of those resources. Different processes within their own namespaces can use the same resources without interfering with each other. For example, Android and Linux systems can share the GPU and memory of a device.

[0052] Figure 2 The system architecture diagram of the dual system of Linux and Android is shown.

[0053] In the embodiment of the present application, the system architecture can be divided into three layers, each with a clear role and division of labor. The layers communicate with each other through interfaces, including: user layer, system kernel layer and hardware layer.

[0054] The user layer includes an Android system 300 and a Linux system 400 .

[0055] The Android system 300 includes an Android application, a window management service 301 , a rendering service 302 , a memory service 303 , a composition service 304 (Hardware Composer, HWComposer) and a rendering instruction library 305 .

[0056] Android applications are applications that run on the Android system and have a display interface.

[0057] The window management service 301 is used to manage the windows of Android applications. The window management service 301 can allocate a canvas for each window of the application and control the window's position, size, Z order, and window animation. The Z order here refers to the position of the window in the stack of overlapping windows.

[0058] The rendering service 302 is used to construct the display interface of the Android application. The display interface can be composed of one or more views. For example, the display interface including the SMS notification icon can include a view for displaying text and a view for displaying pictures.

[0059] The memory service 303 is used to allocate available memory size to the Android application.

[0060] The synthesis service 304 is used to synthesize the display interface of the Android application constructed by the rendering service 302.

[0061] Rendering instruction library 305, that is, open graphics instruction library (Open Graphics Library for Embedded Systems, OpenGL ES). In an embodiment of the present application, the rendering instruction library 305 can be arranged in the operating system, including the interface of the rendering instruction of the GPU of the electronic device, and the process running on the operating system can call the GPU of the electronic device by accessing the interface corresponding to the rendering instruction library 305. In an embodiment of the present application, a rendering instruction library 305 is provided in the operating system of the Android side, and the interface of the rendering instruction of the GPU of the Linux side is defined in the rendering instruction library 305. Since the Android side runs in the container of the Linux side, the Android side is equivalent to the process running on the Linux side. The Android side can directly call the GPU of the Linux side through the rendering instruction library 305 to render the display window of the application.

[0062] The Linux system 400 includes Linux applications, a graphical user interface service 401 (Graphical User Interface Toolkit, Gui Toolkit), and a composition service 402 .

[0063] Linux applications are applications that run on the Linux system and have a display interface.

[0064] Graphical interface service 401 is used to draw the window of the display interface of the application displayed on the Linux system. The window of the Linux system is different from the window of the Android system. When the display interface of the application is displayed on the electronic device of the Linux system, the window of the display interface of the application needs to be drawn by the graphical interface service on the Linux side.

[0065] The synthesis service 402 is used to synthesize the display interface of the Linux application.

[0066] The Linux and Android sides communicate via PIPEs (pipes). PIPE communication is used to transfer data and instructions between processes and the operating system, that is, to transfer data and instructions between the Android and Linux sides. Services used to manage PIPE communication include: session management service 403, data service 404, and instruction service 405.

[0067] The data service 404 is used for the Linux side to receive data from the Android side, for example, to receive window information from the window management service 301 .

[0068] The instruction service 405 is used to compile the instructions on the Android side into an instruction format that can be communicated via PIPE. For example, the synthesis instruction is received from the synthesis service 304.

[0069] The session management service 403 is used to manage the PIPE communication connection between the Linux side and the Android side, as well as the connection between the graphical interface service 401 and the synthesis service 402 on the Linux side and the system kernel layer.

[0070] For instructions sent from the Android side to the Linux side, PIPE communication needs to encode and de-encode the instructions through the instruction service 405 before converting them into instructions that the Linux side can recognize. For example, in an embodiment of the present application, when the Android side sends a synthesis instruction for the application display interface to the Linux side, the Android side needs to encode the synthesis instruction into a data format supported by PIPE communication and then send it to the Linux side via PIPE communication; after receiving the synthesis instruction, the Linux side needs to convert the synthesis instruction into a synthesis instruction supported by itself again through de-encoding.

[0071] The system kernel layer includes a memory management service 406 and a composition management service 407 (Direct Rendering Manager, DRM).

[0072] Among them, the synthesis management service 407 includes three parts: GPU node module 4071, kernel mode setting module 4072 (Kernel Mode Setting, KMS), and graphics execution management module 4073 (Graphic Execution Manager, GEM). The GPU node module 4071 is used to provide an interface for accessing the GPU. For example, the GPU can be accessed by calling the interface of the GPU node module 4071 through the rendering instruction library 305. The kernel mode setting module 4072 is used to update the screen and set display parameters, and the graphics execution management module 4073 is used to allocate and release the display cache. In an embodiment of the present application, the synthesis management service 407 can also obtain the vsync signal of the screen of the electronic device and return the vsync signal to the synthesis service 304 on the Android side.

[0073] Memory management service 406 is a memory management service provided by both the Linux and Android sides. Memory management service 406 can determine a shared memory area on the Linux side. Both the Linux and Android sides can access data in the shared memory area, preventing processes on the Linux side from copying data from one memory area to another. For example, on the Linux side, memory management service 406 can be set at the node / dev / ion. Memory management service 406 here can be the access interface of the node defined on the Linux side, provided to the Android side for access. For example, the node here can refer to a Domain Socket type. This type of node can be used for communication between processes in the Linux operating system, that is, for communication between the Linux side and the Android side.

[0074] The working principle of the memory management service 406 includes: the Android side first performs an open() operation on the device node / dev / ion, and then the Android side can perform an ioctl() memory allocation operation. After completing the memory allocation operation, the Android side can obtain the memory handle of the memory area, that is, the address of the memory area.

[0075] The hardware layer includes: central processing unit (CPU) 101, memory 102, and GPU 103.

[0076] In an embodiment of the present application, the Android side sends the display data of the application's display interface to the memory 102 through the memory management service 406, and calls the GPU 103 to render the display data through the rendering instruction library 305, and then synthesizes the rendered display data into a display interface through the synthesis management service 407.

[0077] In the embodiments of this application, based on Figure 2In the system architecture shown, the synthesis management service 407 on the Linux side can obtain the vsync signal of the electronic device. The synthesis management service 407 serves as a channel between the Linux side and the Android side, and sends the vsync signal of the electronic device to the Android side in real time; in the process of rendering the display interface of the application on the Android side, the Android side can apply for a memory area that can be shared by the Android side and the Linux side on the Linux side through the memory management service 406. After that, the Android side can call the GPU driver on the Linux side through the rendering instruction library 305 within the duration of the current vsync signal to control the GPU on the Linux side to render the data of the display interface in the memory area; after the rendering is completed, the Android side sends a synthesis instruction to the Linux side; within the duration of the next vsync signal, the Linux side synthesizes the rendered display interface data into the display interface of the application, and sends it to the screen display of the electronic device.

[0078] The vsync signal may be acquired by the composition management service 407 of the Linux system and transmitted to the composition service 304 of the Android system to be provided to other system services or applications communicating with the composition service 304 of the Android system.

[0079] The electronic device 100 in the embodiments of the present application can be various terminal devices, for example, including but not limited to laptop computers, desktop computers, tablet computers, mobile phones, servers, wearable devices, head-mounted displays, mobile email devices, portable game consoles, portable music players, reader devices, or other terminal devices capable of accessing the Internet.

[0080] The following uses a car computer as an example of an electronic device 100, and takes the scenario of displaying the display interface of the navigation application on the screen of the car computer on the Linux side after the navigation application installed on the Android side is started as an example to explain the rendering method of the electronic device and its screen according to the present application.

[0081] Figure 3 The structure diagram of the vehicle computer 100 according to an embodiment of the present application is shown. Specifically, Figure 3 As shown, the vehicle computer 100 includes a CPU 101, a memory 102, a GPU 103, a screen 104, an interface management module 105, a power switch 106, a communication processing module 107 and a bus 108, etc.

[0082] CPU 101 can be used to read and execute computer-readable instructions. In a specific implementation, processor 101 may mainly include a controller, an arithmetic unit, and registers. The controller is primarily responsible for decoding instructions and issuing control signals for operations corresponding to the instructions. The arithmetic unit is primarily responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations, and may also perform address operations and conversions. Registers are primarily responsible for storing register operands and intermediate operation results temporarily stored during instruction execution. In a specific implementation, the hardware architecture of processor 101 may be an application-specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.

[0083] The memory 102 is coupled to the CPU 101 and is used to store various software programs and / or multiple sets of instructions. In a specific implementation, the memory 102 may include a high-speed random access memory and a volatile memory.

[0084] GPU 103 is used to render the application's display interface, images, and videos.

[0085] Screen 104 is used to display the application's display interface, images, videos, etc. Screen 104 can be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode (AMOLED) display, a flexible light-emitting diode (FLED) display, a quantum dot light-emitting diode (QLED) display, etc.

[0086] The interface management module 105 is used to manage the display interface of the application running on the vehicle computer 100. For example, the interface management module 105 can build the display interface of the application and add controls on the display interface, such as controls for displaying text and controls for displaying pictures.

[0087] The power switch 106 can be used to control the power supply to various components of the vehicle computer 100 .

[0088] The communication processing module 107 may include a wireless communication processing module (not shown). The wireless communication processing module may also include a cellular mobile communication processing module (not shown). The cellular mobile communication processing module may communicate with other devices (such as a server) via cellular mobile communication technology.

[0089] The bus 108 is a common communication trunk line for transmitting information between the functional components and modules in the vehicle computer 100 .

[0090] It is understandable that Figure 3 The illustrated structure does not constitute a specific limitation on the vehicle computer 100. In other embodiments of the present application, the vehicle computer 100 may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0091] The following explains the process of displaying the display interface of the navigation application 200 on the screen of the vehicle machine 100 on the Linux side after the navigation application 200 installed on the Android side is started. Figure 4 As shown, the above process includes:

[0092] S401: The Android side starts the navigation application 200.

[0093] On the vehicle computer 100 , after the Android side responds to the instruction generated by the user clicking the icon of the navigation application 200 , the application startup service on the Android side obtains and recognizes the instruction to start the navigation application 200 .

[0094] S402 : The Android side renders the display interface 201 of the navigation application 200 .

[0095] Here, the rendering service 302 on the Android side can render the display interface 201 of the navigation application 200 according to its own rendering frequency, for example, 60FPS. That is, within each 16ms time period, the rendering service 302 on the Android side will render the display interface 201 of the navigation application 200 once. Here, the rendering frequency on the Android side can be pre-set by the Android side. For example, the rendering service 302 on the Android side can set the rendering frequency. Therefore, the rendering frequency can also be called a software vsync signal. The specific implementation process of step S402 will be described below.

[0096] S403: The Android side determines whether its rendering frequency is synchronized with the vsync signal of the Linux side.

[0097] The vsync signal here can be the vsync signal corresponding to the rendering frequency of the screen 104 of the vehicle computer 100 obtained by the Linux side. Figure 8 This section describes in detail the process of obtaining the vsync signal on the Linux side.

[0098] After the Android side completes rendering of the display interface of the navigation application 200, if the Android side has received the vsync signal from the Linux side and determines that its own rendering frequency is synchronized with the vsync signal on the Linux side, then S405 is executed, and the Android side sends a synthesis instruction for the display interface of the navigation application 200 to the Linux side. If not, S404 is executed, and the Android side adjusts its own rendering frequency to be synchronized with the vsync signal on the Linux side, and then S405 is executed again, and the Android side sends a synthesis instruction for the display interface of the navigation application 200 to the Linux side.

[0099] Combine Figure 2 As shown, an interface for sending a vsync signal to the Android side can be set in the composition management service 407 on the Linux side, and the composition service 304 on the Android side can also obtain the vsync signal on the Linux side through this interface. The duration of the vsync signal here can be determined by the operating system according to the rendering frequency of the screen 204 of the electronic device 100. For example, the rendering frequency of the screen 204 of the electronic device 100 can be 60FPS, that is, 60 frames are refreshed in 1 second; then the duration of the vsync signal is also 1000 milliseconds / 60=16 milliseconds (i.e., 16ms). The system structure of the composition management service 407 here and the process of the composition management service 407 obtaining the vsync signal will be described below.

[0100] S404: The Android side adjusts its rendering frequency to be synchronized with the vsync signal of the Linux side.

[0101] Here, the rendering frequency on the Android side is synchronized with the vsync signal on the Linux side, which means that the rendering frequency on the Android side is synchronized with the vsync signal on the Linux side in frequency and time. Frequency synchronization can be, for example, if the rendering frequency of the operating system on the Android side is 50FPS and the duration of the vsync signal on the Linux side is 16 milliseconds (i.e., 16ms), then the Android side needs to adjust its own rendering frequency to 1000 milliseconds / 16 milliseconds = 60FPS.

[0102] Time synchronization can be such that, when the rendering frequency on the Android side is the same as the vsync signal on the Linux side, there is no time difference between the rendering frequency on the Android side and the vsync signal. For example, when the rendering frequency and the vsync signal are both 60 FPS, the rendering service 302 on the Android side renders the display interface 201 of the navigation application 200 for 10 ms. At this time, the Android side receives the vsync signal on the Linux side, that is, the rendering frequency on the Android side is 6 ms slower than the vsync signal on the Linux side. In this case, the Android side can advance its own rendering frequency by 6 ms, so that the rendering frequency on the Android side is time synchronized with the vsync signal on the Linux side.

[0103] In another embodiment of the present application, after the Android side completes synchronization of its own rendering frequency with the vsync signal on the Linux side, that is, after calibration of the rendering frequency is completed, the Android side can stop obtaining the vsync signal on the Linux side for a period of time. For example, when the Android side detects that the application is stuck, the Android side can send an instruction to the Linux side to obtain the vsync signal on the Linux side through the vsync signal interface in the Linux side synthesis management service 407, and the Linux side returns the vsync signal to the Android side.

[0104] S405: The Android side sends a synthesis instruction of the display interface of the navigation application 200 to the Linux side.

[0105] Here, after the Android-side navigation application 200 completes rendering of the display data of the display interface 201, the Android-side composition service 304 can send a composition instruction to the Linux-side composition service 402 via PIPE communication. In an embodiment of the present application, the Android-side composition service 304 can send the composition instruction to the Linux-side composition service 402 via the instruction service 405.

[0106] S406 : After synthesizing the display interface of the navigation application 200 , the Linux side displays the display interface 201 of the navigation application 200 on the screen 204 of the electronic device 100 .

[0107] Here, the synthesis service 402 on the Linux side sends the memory handle of the memory area to the synthesis management service 407. The synthesis management service 407 determines the display data of the rendered display interface 201 stored in the memory area based on the memory handle. The synthesis management service 407 synthesizes the window of the display interface 201 of the navigation application 200 and the display data into a display interface, and displays it on the screen 104 of the vehicle computer 100.

[0108] In the above step S402, the process of rendering the display interface 201 of the navigation application 200 on the Android side can be as follows: Figure 5 As shown, the following steps are included:

[0109] S402a: The Android side obtains the window information and display data of the display interface of the navigation application 200.

[0110] For example, Figure 6 (a) and 6 (b) show the display interface of the navigation application 200 displayed on the screen 104 of the vehicle machine 100. Figure 6 As shown in Figures 6(a) and 6(b), the display interface 201 of the navigation application 200 includes a navigation window 2011 and an information window 2012. The window information of the display interface 201 may include the positions of the navigation window 2011 and the information window 2012 within the display interface 201, as well as the positions of the controls contained in the navigation window 2011 and the information window 2012. For example, the positions of the navigation window 2011 and the information window 2012 may be represented as follows: navigation window 2011: centered and upper; navigation window 2011: centered and lower. Navigation window 2011 may include a map control, and the position of the map control may be represented as follows: map control: centered and maximized. Information window 2012 may include an information control 20121 and a toolbar 20122. The position of the information control 20121 may be represented as follows: information control 20121: centered and upper; toolbar 20122: centered and lower.

[0111] The display data of the display interface 201 may be the display content in the display interface 201. For example, the data of the navigation window 2011 may be the navigation map in the navigation window 2011. The data of the information window 2012 may be the navigation information in the information window 2012 and the icon and name of the toolbar.

[0112] Here, the Android side can be Figure 2The Android window management service 301 obtains the window information and display interface data contained in the display interface 201. The window management service 301 can obtain the corresponding positions of the navigation window 2011 and the information window 2012 contained in the display interface 201 from the activity corresponding to the display interface 201, and further obtain the controls contained in the navigation window 2011 and the information window 2012.

[0113] S402b: The Android side applies for a memory area from the Linux side.

[0114] The process of applying for a memory area from the Android side to the Linux side may be that after the navigation application 200 is started, it is necessary to render the display interface 201 to display the display interface 201. Since the rendering of the display interface 201 requires rendering the display data of the display interface 201 in the memory area to be completed, Figure 2 The rendering service 302 on the Android side can determine a shared memory area in the memory 102 on the Linux side through the memory service, and the memory area is used for the Android side and the Linux side to access the stored data in a shared manner. Figure 7 This section describes the process of requesting a memory area from the Android side to the Linux side.

[0115] like Figure 7 As shown, the Android side determines a shared memory area 1021 on the Linux side. The Android side can establish a channel with the Linux side to send a request to determine the memory area. Next, the rendering service 302 on the Android side can drive the memory service 303 to send a memory request to the memory management service 406 on the Linux side. This request can include the size of the requested memory area. After receiving this request, the Linux side can determine a memory area 1021 in its own memory 102 with the same size as the memory area in the memory request and return the memory handle of this memory area 1021 to the rendering service 302 on the Android side. The memory handle here can be the starting address of the memory area 1021. For example, the size of the memory area 1021 requested by the Android side from the Linux side can be 2000, and the address range determined by the Linux side for this memory area 1021 can be [1001, 3000], where 1001 is the starting address of the memory area.

[0116] S402c: The Android side renders the display data of the display interface of the navigation application 200 .

[0117] Here, after the Android side determines the memory area of ​​the Linux side, the navigation application 200 can call Figure 2 The interface of the GPU on the Linux side in the rendering instruction library 305 in the rendering instruction library 305 is used to render the display data of the display interface 201 using the GPU on the Linux side. As described above, the rendering instruction library 305 on the Android side can be an interface for accessing the GPU 103 of the vehicle computer 100. The Android side can establish an access connection with the GPU 103 of the vehicle computer 100 through the rendering instruction library 305, and send rendering instructions to the GPU 103 of the vehicle computer 100 through the rendering instruction library 305. For example, the format of the rendering instruction for rendering the display data of the display interface 201 can be in the form of a rendering display interface ("navigation map", "XX Road", "XX Square", "1000"), and the display data of the display interface 201 is included in the rendering instruction as a parameter. At the same time, the rendering instruction can also include the starting address of the memory area determined in step S402b, for example, "1000" in the above rendering instruction.

[0118] S402d: The Android side sends the window information of the display interface of the navigation application 200 to the Linux side.

[0119] here, Figure 2 The window management service 301 on the Android side can send window information to the Linux side via PIPE communication. For example, the window management service 301 on the Android side can send the positions of the navigation window 2011 and the information window 2012 contained in the display interface 201, as well as the map controls contained in the navigation window 2011 and the information window 2012, and the positions of the information control 20121 and the toolbar 20122 to the Linux side via PIPE communication.

[0120] S402 e : The Linux side generates a window of the display interface of the navigation application 200 according to the window information of the display interface 201 of the navigation application 200 .

[0121] For example, after receiving the window information of the display interface of the navigation application 200, the Linux side Figure 2 The graphical interface service 401 thereof generates a window of the display interface 201 of the navigation application 200 suitable for the screen 104 of the vehicle computer 100 .

[0122] Here, the Linux side can generate a window of the display interface 201 of the navigation application 200 displayed on the screen 204 of the vehicle computer 100 through its own graphical interface service based on the size of the screen 204 of the electronic device 100 and in combination with the position of the navigation window 2011 and the information window 2012 in the window information of the display interface 201, as well as the map controls contained in the navigation window 2011 and the information window 2012 respectively, and the position of the information control 20121 and the toolbar 20122.

[0123] Figure 8 The module structure of the synthesis management service 407 and the process of synthesizing the display interface 201 by the synthesis management service 407 and displaying the display interface 201 on the screen 104 of the vehicle computer 100 are shown. Figure 8 The synthesis management service 407 also includes: a data cache module 4074 (Framebuffer), a view module 4075 (Planes), a synthesis module 4076 (CRTC), an encoder module 4077 (Encoder), a connection module 4078 (Connector) and a signal module 4079 (Framebuffer DevEmulation).

[0124] The connection module 4078 is used to connect to the screen 104 of the vehicle computer 100 and transmit the signal generated by the encoder module 4077 to the screen 104 of the vehicle computer 100. The connection module 4078 may also store parameters of the screen 104 of the vehicle computer 100, such as the refresh rate. The data cache module 4074 is used to cache display data rendered by the GPU 103 of the vehicle computer 100. The view module (Planes) is used to obtain the window of the display interface 201 of the navigation application 200. The synthesis module 4076 is used to read the display data and window from the data cache module 4074 and the view module 4075, respectively, and synthesize the display data and window into a display interface, which is then sent to the encoder module 4077. The synthesis module 4076 may also obtain the vsync signal corresponding to the refresh rate of the screen 104 of the vehicle computer 100 on the Linux side from the connection module 4078 and transmit the vsync signal to the synthesis service 402 on the Linux side. The encoder module 4077 converts the synthesized display interface into a signal that can be displayed on the screen 104 of the vehicle computer 100, such as DVID, VGA, or YPbPr. The signal module obtains the vsync signal of the screen 104 of the vehicle computer 100 on the Linux side from the synthesis module 4076 and provides a method for reporting the vsync signal to the Android side. The signal module 4079 communicates with the synthesis service 304 on the Android side. After the vsync signal is generated on the Linux side, it is reported to the Android side in real time.

[0125] In the embodiment of the present application, during the process of the display interface 201 of the navigation application 200 displayed by the vehicle computer 100, the navigation application 200 continuously renders its own display interface 201, and the vehicle computer 100 continuously refreshes the display interface 201 according to the refresh frequency of the screen 104, wherein each time the display interface 201 of the navigation application 200 is refreshed, the current image frame of the display interface 201 is replaced with the next image frame of the display interface 201, that is, the Android side and the Linux side need to repeat the above steps S402 to S405 to update the image frame of the display interface 201 of the navigation application 200. For example, Figure 6 (a) and Figure 6 (b) Taking the first image frame and the second image frame of the display interface of the navigation application 200 as an example, the navigation application 200 completes rendering in the memory area. Figure 6 After the first image frame of the display interface 201 of the navigation application 200 shown in (a) is displayed, the Android side waits for the first vsync signal sent by the Linux side. After receiving the first vsync signal, the Android side sends a synthesis instruction to the Linux side. After receiving the synthesis instruction, the Linux side reads the first image frame from the memory area and synthesizes the content of the first image frame with the window of the first image frame to form the first image frame. The first image frame is then displayed on the screen 204 of the electronic device 100 within the duration of the first vsync signal.

[0126] Meanwhile, during the duration of the first vsync signal, the navigation application 200 continues to render in the memory area. Figure 6 (b) shows the content of the second image frame of the display interface 201 of the navigation application 200. After receiving the second vsync signal, the Android side sends a synthesis instruction to the Linux side. After receiving the synthesis instruction, the Linux side reads the second image frame from the memory area and synthesizes the content of the second image frame with the window of the second image frame to form the second image frame. The first image frame is then displayed on the screen 104 of the electronic device 100 during the duration of the second vsync signal.

[0127] After introducing the embodiment of the present application, in the dual system of Linux and Android, the solution of displaying the application on the Android side on the screen of the electronic device on the Linux side, Figure 9 A system architecture diagram of a dual system of Linux and Android in a solution in which an application on the Android side is displayed on the screen of an electronic device on the Linux side in another embodiment of the present application is shown.

[0128] like Figure 9As shown, the dual system of Linux and Android includes: user layer, system kernel layer and hardware layer.

[0129] Among them, most components and services of the user layer, system kernel layer and hardware layer are Figure 2 Same system as shown in , except that Figure 9 The rendering instruction library 408 is set in the Linux system 400, and the Linux system 400 also includes: a rendering instruction service 409.

[0130] The rendering instruction service 409 is used for the Linux side to provide an interface for rendering instructions in the rendering instruction library 408 on the Linux side to the Android side, and to receive rendering instructions from the Android side.

[0131] exist Figure 9 In the illustrated embodiment, the Android side sends a memory request instruction to the Linux side via the memory service 303. The memory request instruction may include the amount of memory required to render the application's display interface. The Linux side can determine a memory area in the electronic device's memory based on the memory request instruction.

[0132] For the rendering instructions sent from the Android side to the Linux side, the format of the rendering instructions can be the same as that in S402c, except that the rendering instructions need to be encoded and de-encoded by the instruction service 405 before they can be converted into instructions that can be recognized by the Linux side. For example, in an embodiment of the present application, when the Android side sends the rendering instructions of the application display interface to the Linux side, the Android side needs to encode the rendering instructions into a data format supported by PIPE communication, and then send it to the Linux side via PIPE communication; after the Linux side receives the synthesis instruction, it needs to convert the synthesis instruction into a rendering instruction supported by itself again by de-encoding, and render it in the above-mentioned determined memory area through the rendering instruction library.

[0133] After introducing the embodiments of the present application, a system architecture diagram of the Linux and Android dual system is provided in which an application on the Android side is displayed on the screen of an electronic device on the Linux side under the Linux and Android dual system. Figure 10 The following is a diagram showing the system architecture of an electronic device with an Android system in another embodiment of the present application, in which the display interface of an application is displayed on the screen of the electronic device. Figure 10In the example, the application sends the display data of the display interface to the Android rendering service 302. The rendering service 302 applies for a memory area from the memory 102 of the electronic device through the memory service 303; the Android rendering service 302 can use the GPU 103 to render the display data of the display interface according to the rendering frequency (the rendering frequency here can be called the software vsync signal); after the rendering is completed, the synthesis service 304 can also obtain the vsync signal (the vsync signal here can be called the hardware vsync signal) from the screen 104 of the electronic device. When the rendering frequency and the vsync signal are synchronized, the synthesis service 304 sends a synthesis instruction to the GPU 103; the GPU 103 sends the rendered display data of the display interface to the screen 104 of the electronic device for display.

[0134] Figure 11 The figure shows a system architecture diagram of an electronic device with a Linux system in another embodiment of the present application, in which the display interface of the application is displayed on the screen of the electronic device. Figure 11 In the example, the application sends display data of the display interface to the Linux graphical interface service 401. The graphical interface service 401 calls the GPU node module operation 407 in the composition management service 407 through the Linux rendering instruction library 408. The GPU 103 renders the display data of the display interface in the memory 102. After the rendering is completed, the graphical interface service 401 can also obtain the electronic device's vsync signal from the composition management service 407, such as the vsync signal of the screen 104, and send a composition instruction to the GPU 103 through the composition service 402 according to the vsync signal. The GPU 103 sends the rendered display data of the display interface to the screen 104 of the electronic device for display according to the vsync signal.

[0135] It should be understood that although the terms "first," "second," and the like may be used herein to describe various features, these features should not be limited by these terms. These terms are used merely to distinguish and should not be understood as indicating or implying relative importance. For example, a first feature may be referred to as a second feature, and similarly, a second feature may be referred to as a first feature, without departing from the scope of the exemplary embodiments.

[0136] Furthermore, various operations will be described as multiple, separate operations in a manner that is most helpful for understanding the illustrative embodiments; however, the order of description should not be construed to imply that the operations must be performed in order of description, and many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can terminate when the described operations are completed, but can also have additional operations not included in the figures. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0137] References in the specification to "one embodiment," "an embodiment," "an illustrative embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or property, but that every embodiment may or may not necessarily include the particular feature, structure, or property. Furthermore, these phrases are not necessarily referring to the same embodiment. Furthermore, while particular features may be described in conjunction with a specific embodiment, the knowledge of those skilled in the art may affect how those features may be combined with other embodiments, whether or not those embodiments are explicitly described.

[0138] Unless the context dictates otherwise, the terms "comprising," "having," and "including" are synonymous. The phrase "A / B" means "A or B." The phrase "A and / or B" means "(A), (B), or (A and B)."

[0139] As used herein, the term "module" may refer to, be part of, or include: memory (shared, dedicated, or group) for running one or more software or firmware programs, application-specific integrated circuits (ASICs), electronic circuits and / or processors (shared, dedicated, or group), combinational logic circuits, and / or other suitable components that provide the functionality.

[0140] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order is not required. Rather, in some embodiments, these features may be illustrated in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular drawing does not mean that all embodiments need to include such features. In some embodiments, these features may not be included, or they may be combined with other features.

[0141] The embodiments of the present application are described in detail above with reference to the accompanying drawings. However, the application of the technical solution of the present application is not limited to the various applications mentioned in the embodiments of the present patent. Various structures and variations can be easily implemented with reference to the technical solution of the present application to achieve the various beneficial effects mentioned herein. Various changes made within the knowledge of ordinary technicians in this field without departing from the purpose of the present application should fall within the scope of coverage of the patent application.

Claims

1. A method for rendering a picture, applied to an electronic device comprising a first operating system and a second operating system, wherein the second operating system is a main system of the electronic device, characterized in that: include: The first operating system performs a rendering operation of the picture; The second operating system obtains a synchronization signal of the electronic device and sends it to the first operating system, wherein the synchronization signal is used to synchronize the execution of the rendering operation of the screen and the display of the screen by the electronic device; In response to the synchronization signal, the first operating system notifies the second operating system to trigger the screen of the electronic device to display the image; In response to the synchronization signal, the first operating system notifies the second operating system to trigger the screen of the electronic device to display the picture, including: When the first operating system receives the synchronization signal and the rendering operation of the screen of the first operating system is completed, the first operating system sends a display instruction to the second operating system; In response to the display instruction, the second operating system triggers the screen of the electronic device to display the picture within the duration of the synchronization signal.

2. The method according to claim 1, characterized in that The first operating system determines at least one storage area in the storage component of the electronic device, and the first operating system obtains the address, size and storage format of the storage area, where the storage area is used to store the display data of the screen.

3. The method according to claim 2, characterized in that The first operating system performs a picture rendering operation, including: Configuring a rendering instruction library in the first operating system, wherein the rendering instruction library is used by the first operating system to control a rendering component of the electronic device; The first operating system sends a rendering instruction to a rendering component of the electronic device through the rendering instruction library; The rendering component of the electronic device performs a rendering operation of the screen in the storage area based on the rendering instruction.

4. The method according to claim 3, characterized in that The rendering instruction includes the display data of the picture and the address, size and storage format of the storage area.

5. The method according to claim 4, characterized in that The second operating system acquires the synchronization signal of the electronic device and sends the synchronization signal to the first operating system, including: The synchronization signal is a refresh frequency corresponding to the screen of the electronic device; The second operating system sends the synchronization signal to the first operating system through process communication.

6. The method according to claim 2, characterized in that After the first operating system receives the synchronization signal, the rendering frequency of the rendering operation of the screen of the first operating system is adjusted to be synchronized with the synchronization signal.

7. The method according to claim 6, characterized in that Adjusting the rendering frequency of the rendering operation of the screen of the first operating system to be synchronized with the synchronization signal includes: adjusting the rendering frequency to maintain frequency synchronization and time synchronization with the synchronization signal.

8. The method according to claim 2, characterized in that The display instruction includes the address of the screen in the storage area.

9. The method according to claim 2, characterized in that The synchronization signal includes at least a first synchronization signal and a second synchronization signal, the screen of the first operating system includes at least a first screen and a second screen, and the first synchronization signal is adjacent to the second synchronization signal.

10. The method according to claim 9, characterized in that include: The first operating system completes the rendering operation of the first picture within the duration of the first synchronization signal; During the duration of the second synchronization signal, the screen of the electronic device displays the first picture.

11. The method according to claim 10, characterized in that Within the duration of the second synchronization signal, the first operating system completes the rendering operation of the second picture.

12. The method according to claim 11, characterized in that During the duration of the second synchronization signal, if the first operating system has not completed the rendering operation of the second picture, the screen of the electronic device displays the first picture.

13. The method according to claim 1, wherein The second operating system is configured to periodically send the synchronization signal to the first operating system, or the first operating system is configured to periodically obtain the synchronization signal from the second operating system.

14. The method according to claim 3, characterized in that The rendering component is a GPU of the electronic device, and the storage component is a memory of the electronic device.

15. The method according to claim 1, wherein The first operating system is an Android system, and the second operating system is a Linux system.

16. A readable medium, characterized in that The readable medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the method for rendering a picture according to any one of claims 1 to 15.

17. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device, and The processor is one of the processors of the electronic device, and is used to execute the screen rendering method according to any one of claims 1 to 15.

Images