Layer synthesis method and electronic equipment

Abstract

The invention provides a layer synthesis method and electronic equipment, and is applied to the technical field of terminals. The method comprises the following steps: after a main thread which creates a monitoring signal in advance calls a rendering engine thread, triggering a graphics processing unit (GPU) by using the rendering engine thread to synthesize a graphic layer which is marked as GPU synthesis so as to obtain a GPU synthesis graphic layer. The monitoring signal is used for monitoring whether synthesis of the GPU is completed or not. And outputting the monitoring signal to a hardware synthesizer (HWC). And after the monitoring signal monitors that the synthesis of the GPU is completed, the HWC synthesizes the GPU synthesis layer. Thus, the monitoring signal is used for replacing the GPU Fen to monitor GPU synthesis, and the main thread does not need to wait for obtaining the GPU Fen and then execute the subsequent processing flow, so that the main thread layer synthesis time is saved, the jamming phenomenon is avoided, and the visual expressive force of the graph is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a layer synthesis method and electronic device. Background Art

[0002] The display interface of an electronic device is often composed of multiple layers. For example, the display interface of a mobile phone is composed of a status bar layer at the top, a navigation bar layer at the bottom, and an application interface layer in the middle. Among them, the application interface is the display interface of the application running in the foreground.

[0003] At present, during the dynamic display of the display interface, multiple layers are often synthesized through the main thread SurfaceFlinger to obtain the image frame of the display interface. For example, during the application startup process, SurfaceFlinger obtains multiple rendered layers, synthesizes multiple layers through the hardware composer (HWC) and / or the graphics processing unit (GPU) to obtain the image frame of the display interface.

[0004] However, when electronic devices are in overload scenarios such as startup and exit, blurred real-time windows, etc., display interface freezes often occur, affecting the user's visual experience. Summary of the invention

[0005] The present application provides a layer synthesis method and electronic device, aiming to solve the problem of display interface freeze caused by frequent frame loss when the electronic device is in overloaded scenarios such as startup and exit scenarios, real-time window blur scenarios, too many layers, etc.

[0006] In a first aspect, the present application provides a method for synthesizing a layer, the method comprising:

[0007] After the main thread SurfaceFlinger that pre-creates the monitoring signal calls the rendering engine thread RenderEngine, RenderEngine is used to trigger the graphics processor GPU to synthesize the layer marked as GPU synthesis to obtain the GPU synthesis layer. Among them, the monitoring signal is used to monitor whether the GPU synthesis is completed. The monitoring signal is output to the hardware synthesizer HWC. When the HWC detects that the GPU synthesis is completed through the monitoring signal, it synthesizes the GPU synthesis layer. In this way, the monitoring signal is used instead of the GPU Fence to monitor the completion of the GPU synthesis. SurfaceFlinger does not need to wait for the acquisition of the GPU Fence before executing the subsequent processing flow, which saves the SurfaceFlinger layer synthesis time, and then saves the total synthesis cycle of the layer synthesis work, avoids the jamming phenomenon, and improves the visual expression of the graphics.

[0008] In one implementable manner, when SurfaceFlinger calls RenderEngine, the first index object corresponding to the monitoring signal is passed to RenderEngine. The first index object is used to describe the monitoring content of the monitoring signal, and the monitoring content includes whether the GPU synthesis is completed and whether the GPU synthesis is not completed. According to RenderEngine, the GPU is triggered to synthesize the layer marked as GPU synthesis, and according to the second index object corresponding to the GPU Fence, the first index object is adjusted to determine whether the GPU synthesis is completed according to the first index object. The second index object is used to describe the monitoring content of the GPU Fence. In this way, the monitoring signal can track the second index object of the GPU Fence with the first index object, and more accurately monitor whether the GPU synthesis is completed.

[0009] Optionally, when the first index object corresponding to the pre-created monitoring signal is in an untriggered state, the untriggered state is used to indicate that the GPU synthesis is not completed, and when the second index object is in a triggered state, the first index object is adjusted to a triggered state, and the triggered state is used to indicate that the GPU synthesis is completed. In this way, by using the first index object and adjusting the second index object, the accuracy of monitoring the GPU synthesis by the monitoring signal can be improved.

[0010] In another achievable manner, when the second index object is in a triggered state, a pre-created tracking function is called to adjust the first index object to a triggered state, and the tracking function is used to block until the second index object is in a triggered state and adjust the first index object to a triggered state. Optionally, when the untriggered second index object is first acquired, a tracking function is created based on the second index object and the first index object. Optionally, when the second index object is in an untriggered state, the first index object is not adjusted. Optionally, the trigger state is marked with a pre-set identifier. In this way, the trigger state corresponding to the identifier can be triggered by passing a pre-set identifier, thereby improving operating efficiency.

[0011] Optionally, the first index object is added to a pre-registered calling function, where the calling function is used to call RenderEngine to trigger GPU synthesis; according to the calling function, the first index object is passed to RenderEngine.

[0012] Optionally, the first index object corresponding to the monitoring signal and the preset memory identity Buffer Id of the GPU synthesis output are output to the HWC, and the buffer Id of the GPU synthesis output is used to indicate the identity of the memory that uniquely stores the composite layer synthesized by the GPU. The HWC only needs to obtain the GPU synthesis layer from the corresponding buffer through the buffer Id, and can determine whether the output GPU synthesis layer is the GPU synthesis layer of the current frame according to the monitoring signal FD, thereby improving the synthesis efficiency and ensuring the accuracy of the acquired image frame to be displayed.

[0013] Optionally, the electronic device includes a display framework, which includes: a main thread node and a calling thread node, the main thread node is used to execute the SurfaceFlinger of the electronic device, and the calling thread node is used to execute RenderEngine. After the main thread SurfaceFlinger that pre-creates a monitoring signal calls the rendering engine thread RenderEngine, the main thread node sends the first index object corresponding to the monitoring signal to the calling thread node; the calling thread node triggers the GPU to perform GPU synthesis on the layer marked as GPU synthesis according to RenderEngine, and obtains a GPU synthesis layer.

[0014] Optionally, the display framework further includes a tracking node, and the tracking node adjusts the first index object according to the second index object corresponding to the GPU Fence.

[0015] Optionally, when the first index object is a software synchronization monitoring signal file descriptor Fence FD and the second index object is a GPU Fence FD, the tracking node adjusts the first index object according to the second index object corresponding to the GPU Fence. The tracking node calls the operating system wait function to wait for the GPU Fence FD to be in a triggered state;

[0016] When the GPU Fence FD in the kernel is in the triggered state, the tracking node calls the kernel through ioctl to adjust the software FenceFD to the triggered state.

[0017] Optionally, the first index object is added to the transaction callback function, so that the target application monitors the end time of GPU synthesis through the transaction callback function. In this way, other applications can monitor the end time of GPU synthesis by using the monitoring signal FD.

[0018] Optionally, the monitoring signal is a software Fence.

[0019] In a second aspect, the present application provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor. The memory stores a program, and when the program is executed by the processor, the electronic device executes any one of the methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a layer synthesis method provided in an embodiment of the present application;

[0021] Figure 2 Implement composite signaling graphs for layers of related solutions;

[0022] Figure 3 The schematic diagram of the processing thread obtained by the above-mentioned layer synthesis method;

[0023] Figure 4 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of an exit scenario provided for this application;

[0026] Figure 7 A schematic diagram of a real-time window blur scene provided in an embodiment of the present application;

[0027] Figure 8 A signaling diagram of a layer synthesis method provided in an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of a processing thread obtained by the layer synthesis method obtained in an embodiment of the present application;

[0029] Fig.10 A signaling diagram of another layer synthesis method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The terms "first", "second", "third", etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0031] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0032] In order to enable those skilled in the art to better understand the present application, professional terms are first introduced.

[0033] Layer synthesis technology: When the display interface is dynamically displayed, the electronic device obtains multiple layers, renders and synthesizes the multiple layers to obtain image frames to be displayed according to a predetermined display area, and sends the image frames to be displayed to be displayed on the display interface of the electronic device.

[0034] Each layer has corresponding layer properties. The layer properties include information such as the corresponding layer position, size, and display content. In addition, each layer has a corresponding buffer queue (Bufferqueue). Bufferqueue is used to store the layer parameters of the corresponding layer, which are used to indicate the display content of the corresponding layer. When performing layer synthesis, the main thread SurfaceFlinger is often called to perform layer synthesis. Specifically, when the user performs a layer synthesis operation on the display interface of an electronic device, SurfaceFlinger can create a corresponding layer for the application and return the layer ID of the created layer to the application. Among them, the created layer has corresponding layer properties. SurfaceFlinger can obtain the layer parameters of the layers corresponding to the various running applications, and synthesize the display data of the display layer to be displayed through HWC and / or GPU.

[0035] HWC is a dedicated image processing device used for layer synthesis and display. HWC can be an independent device or integrated into a system-level chip. The layer synthesis method that synthesizes the display data of the layer through HWC is called HWC synthesis.

[0036] GPU is a general-purpose image processing device. In addition to being used for layer synthesis, GPU is also used to complete other graphics processing tasks. The layer synthesis method that uses GPU to synthesize the display data of the layer is called GPU synthesis.

[0037] During actual use, SurfaceFlinger sends the layer parameters of the rendered layer to HWC, and HWC marks the rendered layer according to its own processing capabilities. When the number of rendered layers is greater than the maximum number of layers that the HWC can synthesize, some are marked as GPU synthesis and some are marked as HWC synthesis. The electronic device first calls the GPU to synthesize the layers marked as GPU synthesis to obtain a GPU synthesis layer, and then synthesizes the GPU synthesis layer and the layers marked as HWC synthesis to obtain a HWC synthesis layer. The HWC synthesis layer is also called the image frame to be displayed, which is displayed on the display interface. Example description: See Figure 1 , which is a schematic diagram of a layer synthesis method provided in an embodiment of the present application.

[0038] Among them, SurfaceFlinger first obtains the layer parameters of the status bar from the buffer queue corresponding to the status bar 11, obtains the layer parameters of the application interface from the buffer queue corresponding to the application interface 13, and obtains the layer parameters of the navigation bar from the buffer queue corresponding to the navigation bar 12. Since the number of layers to be displayed exceeds the maximum number of GPU synthesis layers supported by HWC, HWC first marks each layer.

[0039] Exemplarily, the layer corresponding to the status bar 11 and the layer corresponding to the application interface 13 are marked as GPU synthesis, that is, the layer parameters of the status bar 11 and the layer parameters of the application interface 13 are modified. The layer corresponding to the navigation bar 12 is marked as HWC synthesis. HWC returns the marking result to SurfaceFlinger.

[0040] Then SurfaceFlinger sends the layer parameters of the status bar 11 and the layer parameters of the application interface 13 to the GPU, which combines the two layers of the status bar 11 and the middle application interface 13 to obtain a GPU composite layer, and saves the layer parameters of the GPU composite layer in the corresponding output buffer.

[0041] Among them, buffer allocates memory for the layer created by SurfaceFlinger to store the layer parameters corresponding to the layer.

[0042] The layer parameters of the GPU synthesis layer and the layer parameters of the remaining layers of the navigation bar 12 are then synthesized through the HWC to obtain the image frame to be displayed, and the frame image to be displayed is sent for display and displayed on the display interface.

[0043] This application only takes the layer corresponding to the status bar 11 and the layer corresponding to the application interface 13 as an example for explanation, and does not constitute a specific limitation. For example, the layer corresponding to the navigation bar 12 and the layer corresponding to the application interface 13 can also be marked as GPU synthesis, and the layer corresponding to the status bar 11 can be marked as HWC synthesis, or the layer corresponding to the navigation bar 12 and the layer corresponding to the status bar 11 can be marked as GPU synthesis, and the layer corresponding to the application interface 13 can be marked as HWC synthesis, etc.

[0044] In some specific embodiments, see Figure 2 , is the synthetic signaling diagram for the layers of the relevant scheme. Figure 2The application rendering thread node, main thread node, calling thread node and HWC synthesis node involved are all located in the display framework in the application framework layer of the electronic device software layer. The application rendering thread node implements layer rendering, the main synthesis node is used to implement the SurfaceFinger of the electronic device, the calling thread node is used to implement the rendering engine thread RenderEngine of the electronic device, and the HWC synthesis node is used to implement HWC synthesis, obtain the image frame to be displayed and send it for display.

[0045] S21. Apply the rendering thread node to obtain the M rendered layers.

[0046] After receiving the M drawn layers, the application rendering thread node performs rendering processing to obtain the rendered M layers. M ≥ 3, and M is a positive integer.

[0047] It is understandable that when a user performs a graphical user interface (GUI) event such as a touch operation on the display interface, M windows and their properties are obtained, such as window height, width, center coordinates, etc. The UI thread is called to draw M layers. Then the application rendering thread node is called to render the drawn M layers to obtain the rendered M layers.

[0048] The touch operation may be an interface exit operation, or a real-time window blur operation, etc.

[0049] S22. The application rendering thread node sends the rendered layer parameters to the main thread node.

[0050] Layer parameters correspond to layer properties, and layer properties correspond to window properties, including window height, width, center coordinates, etc.

[0051] After the application rendering thread node obtains the M layers after rendering, the layer parameters of the M layers are sent to the main thread node, so that the main thread node calls SurfaceFinger to perform the layer merging operation of the M layers.

[0052] S23. The main thread node sends the rendered layer parameters to the HWC node.

[0053] After receiving the rendered layer parameters, the main thread node sends the layer parameters to the HWC node, so that the HWC determines the synthesis strategy of M layers based on the maximum number of layers it processes.

[0054] The synthesis strategy includes GPU synthesis or HWC synthesis.

[0055] In addition, after the main thread node sends the layer parameters to the HWC node, it falls into sleep waiting. When the HWC node returns the synthesis strategy message, the main thread node continues to work.

[0056] S24. The HWC node returns the annotated layer parameters to the main thread node.

[0057] The HWC node determines N layers to perform GPU synthesis based on the maximum number of layers it processes, and marks these N layers as GPU synthesis. The remaining MN layers perform HWC synthesis, and mark the MN layers as HWC synthesis.

[0058] It is understandable that if the maximum number of layers processed by HWC is 2, and the number of layers after rendering is 4, then 3 layers are marked as GPU synthesis and 1 layer is marked as HWC synthesis.

[0059] The HWC node returns N labeled GPU synthesis layer parameters and MN labeled HWC synthesis layer parameters to the main thread node.

[0060] S25. The main thread node sends the layer display parameters marked as GPU synthesis to the calling thread node.

[0061] The layer display parameters include layer parameters, screen parameters, and the memory identity (bufferIdentity, buffer Id) of the synthesis output. Among them, buffer Id belongs to the buffer queue, which is used to indicate the unique memory identity of the GPU synthesized layer. Buffer Id determines that according to the buffer corresponding to buffer Id, the GPU synthesis layer synthesized by the GPU can be accurately obtained. The screen parameters are used to indicate the size of the electronic device display screen, etc.

[0062] After the main thread node sends the layer display parameters, the main thread node falls into waiting and no longer executes processing. It continues to work after the calling thread node sends the GPU synchronization signal (also called GPU Fence) file descriptor (File Description, FD).

[0063] It is understandable that the sending time of the calling thread node to send the GPU Fence FD will affect the completion time of the main thread node, which will in turn affect the display time of the current frame.

[0064] S26. Call the thread node to trigger the GPU to perform asynchronous synthesis and obtain the GPU Fence FD.

[0065] After receiving the layer display parameters of the layer marked as GPU synthesis, the calling thread node continuously sets drawing instructions to the GPU. The drawing instruction is an OpenGL instruction including the layer display parameters, and the GPU performs GPU synthesis according to the layer display parameters of the drawing instruction.

[0066] When the drawing instructions are set, the Flush function is triggered to notify the CPU to perform asynchronous synthesis. At this time, the GPU returns the GPUFence FD to track GPU synthesis. The initial state of the GPU Fence FD is not triggered.

[0067] Specifically, the calling thread node calls the preset acquisition GPU Fence FD tracking function to track the GPU and obtain the GPU Fence FD synthesized by the GPU. At this time, the GPU Fence FD state is a triggered state.

[0068] S27. The calling thread node returns the GPU Fence FD to the main thread node.

[0069] The calling thread node returns the GPU Fence FD to the main thread node, so that the main thread node can obtain the current GPU synthesis status.

[0070] S28. The main thread node sends the buffer identity and GPU Fence FD of the GPU synthesis output to the HWC node.

[0071] HWC receives the buffer ID and uses GPU to synthesize the layer from the corresponding buffer. It obtains whether GPU synthesis is complete from GPU FenceFD.

[0072] S29, HWC node performs layer synthesis and display.

[0073] See also Figure 3 , which is a schematic diagram of the processing thread obtained by the above layer synthesis method. Among them, when SurfaceFlinger performs GPU synthesis, it first calls RenderEngine to trigger GPU synthesis, that is, sets the layer drawing instructions to the GPU through RenderEnginer. When RenderEnginer sets the layer drawing instructions to the GPU, SurfaceFlinger is not running. When the drawing instructions are set, SurfaceFlinger starts running.

[0074] However, when the setting time of RenderEngine setting the drawing instruction to the GPU is not less than the preset time threshold (for example, 2ms), this will cause SurfaceFlinger to process for a long time, and the total synthesis cycle of the layer synthesis work will exceed the preset time (for example, 16ms), and vertical synchronization is required to wait for the next frame to be drawn and send the next frame to the display interface of the electronic device for display. It is understandable that the total synthesis cycle of the layer synthesis work exceeds the preset time, blocking SurfaceFlinger from completing this frame task, resulting in frame loss in this frame synthesis, missing the display time, and thus frame loss, causing the display interface of the electronic device to freeze.

[0075] In heavy-loaded scenarios such as starting or exiting scenarios, real-time window blur scenarios, or scenarios with weak HWC synthesis capabilities, since the number of layers is greater than the maximum number of layers that can be synthesized by the HWC hardware, SurfaceFliner needs to first use the GPU to synthesize some layers to obtain the GPU synthesis layer, and then use the HWC to combine the GPU synthesis layer and the remaining layers to obtain the image frame to be displayed. This causes the display interface of the electronic device to freeze, thus affecting the user's visual experience.

[0076] In view of the above problems, an embodiment of the present application provides a method for synthesizing a layer. After the main thread SurfaceFlinger that pre-creates a monitoring signal calls RenderEngine, RenderEngine is used to trigger the graphics processor GPU to synthesize the layer marked as GPU synthesis to obtain a GPU synthesis layer. Among them, the monitoring signal is used to monitor whether the GPU synthesis is completed. The monitoring signal is output to the hardware synthesizer HWC. When the HWC detects that the GPU synthesis is completed through the monitoring signal, it synthesizes the GPU synthesis layer. In this way, the monitoring signal is used instead of the GPU Fence to monitor the completion of the GPU synthesis. SurfaceFlinger does not need to wait for the acquisition of the GPU Fence before executing the subsequent processing flow, which saves the SurfaceFlinger layer synthesis time, and then saves the total synthesis cycle of the layer synthesis work, avoids the jamming phenomenon, and improves the visual expression of the graphics.

[0077] First, an exemplary electronic device 100 provided in an embodiment of the present application is introduced.

[0078] In some embodiments, the electronic device 100 may be a wearable electronic device such as a wearable sports watch, a wearable smart watch, a wearable bracelet, a wearable audio headset, etc.; it may also be a mobile phone, a laptop computer, a tablet computer, an augmented reality (AR) device, a virtual reality (VR) device, and a vehicle-mounted device, etc. It should be noted that the embodiment of the present application does not impose any special restrictions on the specific type of the electronic device 100.

[0079] See also Figure 4 , is a structural diagram of an electronic device 100 provided in an embodiment of the present application.

[0080] like Figure 4 As shown, the electronic device 100 may include a processor 110, an internal memory 120, an antenna 1, an antenna 2, a mobile communication module 130, a wireless communication module 140, a display screen 160, a touch sensor 180K, etc.

[0081] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0082] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural network processor (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. For example, in the present application, the processor 110 can use RenderEngine to trigger the graphics processor GPU to synthesize the layer marked as GPU synthesis after the main thread SurfaceFlinger that pre-creates the monitoring signal calls RenderEngine to obtain a GPU synthesis layer. Among them, the monitoring signal is used to monitor whether the GPU synthesis is completed. The monitoring signal is output to the hardware synthesizer HWC. When the HWC monitors the completion of the GPU synthesis in the monitoring signal, the GPU synthesis layer is synthesized. In this way, the monitoring signal is used instead of GPU Fence to monitor the completion of GPU synthesis. SurfaceFlinger does not need to wait for the acquisition of GPU Fence before executing the subsequent processing flow, which saves the SurfaceFlinger layer synthesis time, and then saves the total synthesis cycle of the layer synthesis work, avoids the jamming phenomenon, and improves the visual expression of the graphics. Among them, the controller can be the nerve center and command center of the electronic device. The controller can generate an operation control signal based on the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0083] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0084] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0085] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 130, wireless communication module 140, modem processor and baseband processor.

[0086] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0087] The internal memory 120 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 120. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 120, and / or the instructions stored in the memory provided in the processor.

[0088] A series of graphical user interfaces (GUIs) can be displayed on the display screen 160 of the electronic device, and these GUIs are the main screen of the electronic device. Generally speaking, the size of the display screen 160 of the electronic device is fixed, and only limited controls can be displayed on the display screen 160 of the electronic device. A control is a GUI element, which is a software component included in an application program, and controls all data processed by the application program and interactive operations on the data. Users can interact with the control through direct manipulation to read or edit relevant information of the application program. Generally speaking, a control can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.

[0089] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0090] In addition, an operating system runs on the above components, such as the iOS operating system, the Android open source operating system, the Windows operating system, etc. Application programs can be installed and run on the operating system.

[0091] The operating system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device.

[0092] See also Figure 5 , which is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application. The present application embodiment takes the Android system of layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0093] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer (Applications, APP), framework layer (Framework, FWK), system library layer and kernel layer (Kernel).

[0094] The application layer may include a series of application packages. In some embodiments, the application layer may include applications such as calls, videos, etc.

[0095] The framework layer provides an application programming interface (API) and a programming framework for the application of the application layer. The framework layer may include some predefined functions. In some specific implementations, the framework layer may include a GUI framework and a display framework. Specifically, the framework corresponding to a layer synthesis method provided in the present application is stored in the display framework of the application framework layer. The GUI drawing thread is located in the GUI framework.

[0096] Optionally, the main thread node, the call thread node, the tracking node, the application rendering thread node and the HWC node are located in the display frame. The main thread node is used to execute SurfaceFliner. The call thread node is used to execute RenderEngine. The tracking node is used to execute the tracking thread. The application rendering thread node is used to execute application rendering. The HWC node is used to execute HWC synthesis and display.

[0097] The system library may include multiple functional modules. In some specific implementations, the system library layer includes functional modules such as SurfaceFliner, RenderEngine, tracking, and application rendering threads.

[0098] Among them, SurfaceFlinger is used for layer creation, control and management. RenderEngine is used to trigger GPU synthesis. Tracking is used to implement monitoring signal tracking GPU Fence. Application rendering thread is used to render the drawn layer to obtain the rendered layer.

[0099] The kernel layer is responsible for managing the system's threads, memory, device drivers, files, and network systems, and determines the system's performance and stability. In some specific implementations, the kernel layer hardware driver layer, such as display driver, GPU driver, HWC driver, and CPU driver.

[0100] It should be noted that although the embodiment of the present application is described using the Android system as an example, its basic principles are also applicable to electronic devices 100 based on operating systems such as iOS and Windows.

[0101] The following takes the electronic device 100 as a mobile phone as an example to illustrate the application scenario of the embodiment of the present application.

[0102] See also Figure 6 , is a schematic diagram of an exit scenario, such as Figure 6 As shown, the display scene of the GUI consists of a status bar 61 at the top, a navigation bar 62 at the bottom, and an instant messaging interface 63 corresponding to the instant messaging program in the middle. For details of the instant messaging interface, see Figure 6 When the user starts the exit scene operation, such as clicking the "Back" button, sliding, or voice control to perform the exit operation, the display interface after exit can be entered.

[0103] See also Figure 7 , is a schematic diagram of a real-time window blur scene. Figure 7 As shown, Figure 7 (a) in the figure shows that the GUI of the mobile phone displays the payment interface 71, the instant messaging interface 72 and the browsing interface 73. By presetting, the application interface related to payment security is blurred to obtain Figure 7 The payment interface 71 shown in (b) displays the result in a blurry manner.

[0104] Specifically, you can open "Settings" and enter the settings interface. Select the "Desktop" icon in the settings interface to enter the desktop settings interface. In the desktop settings interface, enter the blurred preview image settings. Click the "Payment Security Application Interface" that needs to be blurred to achieve real-time window blurring. Figure 7 The result shown in (b) in .

[0105] It should be noted that the application scenarios provided in this application are only for illustrative purposes, and the layer synthesis method provided in this application can also be applied to other overloaded scenarios or scenarios with weak HWC capabilities. This application is not specifically limited.

[0106] See also Figure 8 , which is a signaling diagram of a layer synthesis method provided in an embodiment of the present application.

[0107] S81. Apply the rendering thread node to obtain M rendered layers.

[0108] In the embodiment of the present application, the M layers may be M layers corresponding to one application, or may be M layers corresponding to M applications. The embodiment of the present application does not specifically limit the number of applications corresponding to the layers to be displayed.

[0109] For example, the application may be an instant messaging, video or other application.

[0110] Optionally, the user executes an exit scene on the display interface corresponding to the instant communication, or the user executes a real-time window blur operation of a certain application for multiple applications, and the application determines M windows of the interface to be displayed. The application calls the GUI drawing node to draw the M windows and sends them to the application rendering thread node. The application rendering thread node renders the M windows to obtain M layers after rendering.

[0111] Optionally, in the process of developing an application, the developer can configure windows for the interfaces of each application and specify the properties of the window, including but not limited to height, width, center coordinates, scaling properties, and rotation properties. The developer can store several window identifiers included in each interface and the properties of each window in a configuration file. After receiving the layer synthesis operation, the application searches the configuration file for the M windows included in the interface to be displayed.

[0112] Exemplarily, as shown in Table 1, the interface to be displayed is found from the configuration file to include three windows, and the three windows are identified as the status bar Surface, the instant messaging Surface, and the navigation bar Surface. The window height corresponding to the status bar Surface is H1cm, the corresponding window width is W1cm, and the corresponding window center coordinates are (x1, y1), which supports no scaling and no rotation. The window height corresponding to the instant messaging Surface is H2cm, the corresponding window width is W2cm, and the corresponding window center coordinates are (x2, y2), which supports scaling and rotation. The window height corresponding to the navigation bar Surface is H3cm, the corresponding window width is W3cm, and the corresponding window center coordinates are (x3, y3), which supports scaling and rotation.

[0113] Table 1 The interface to be displayed includes 3 windows found from the configuration file

[0114]

[0115] For each frame of the screen to be displayed, the application calls the GUI drawing node to obtain the image content corresponding to the screen from the status bar application, and draws the layer corresponding to the status bar according to the image content and the status bar layer properties. The image content corresponding to the screen can be obtained from the instant messaging application, and the layer corresponding to the instant messaging interface can be drawn according to the image content and the instant messaging layer properties. The image content corresponding to the screen can be obtained from the navigation bar, and the layer corresponding to the navigation bar can be drawn according to the image content and the layer properties of the navigation bar.

[0116] The drawn M layers can be sent to the application rendering thread node at the same time or according to a preset rule, such as the first drawn first sent principle. The application thread rendering node renders the M layers to obtain the rendered M layers.

[0117] S82. The application rendering thread node sends the rendered layer parameters to the main thread node.

[0118] The layer parameters correspond to the layer properties, and the layer properties correspond to the parameters such as the window height, width, and center coordinates described in Table 1.

[0119] Exemplarily, the application rendering thread node sends the layer parameters of the three layers, namely, the layer corresponding to the navigation bar, the layer corresponding to the status bar, and the layer corresponding to the instant messaging interface, to the main thread node.

[0120] Optionally, the layer parameters include window identification and properties.

[0121] Optionally, the sending format of the layer parameters is: window identifier, attributes.

[0122] Exemplarily, the sending format of layer parameters is:

[0123] Status bar Surface, H1cm, W1cm, (x1, y1), no scaling, no rotation;

[0124] Instant messagingSurface, H2cm, W2cm, (x2, y2), scale, rotate;

[0125] Navigation bar Surface, H3m, W3m, (x3, y3), scale, rotate.

[0126] S83. The main thread node sends a first message to the HWC node.

[0127] The first message includes layer parameters of M layers included in the to-be-displayed interface and a desired synthesis strategy.

[0128] Desired compositing strategies include GPU compositing and HWC compositing.

[0129] The format of the first message may be: window identifier, attributes, and expected synthesis mode.

[0130] Exemplarily, it is assumed that the first message is:

[0131] Status bar Surface, H1cm, W1cm, (x1, y1), no scaling, no rotation, HWC synthesis;

[0132] Instant messaging Surface, H2cm, W2cm, (x2, y2), scaling, rotation, HWC synthesis;

[0133] Navigation bar Surface, H3m, W3m, (x3, y3), scaling, rotation, HWC synthesis.

[0134] The first message is used to indicate that the expected synthesis strategy of the layer rendered by the status bar Surface, the layer rendered by the instant messaging Surface, and the layer rendered by the navigation bar Surface is HWC synthesis.

[0135] S84. The HWC node returns the second message to the main thread node.

[0136] HWC determines the synthesis strategy of M layers based on the maximum number of layers it processes.

[0137] The specific synthesis strategy is: GPU synthesis is performed on N layers, and these N layers are marked as GPU synthesis. The remaining MN layers are performed on HWC synthesis, and the MN layers are marked as HWC synthesis.

[0138] The HWC node adds the composition strategy, the identifier and properties of the given window to the second message and returns it to SurfaceFlinger.

[0139] The format of the second message may be: window identifier, attribute, and synthesis strategy.

[0140] Exemplarily, it is assumed that the first message is:

[0141] Status bar Surface, H1cm, W1cm, (x1, y1), no scaling, no rotation, HWC synthesis;

[0142] Instant messaging Surface, H2cm, W2cm, (x2, y2), scaling, rotation, HWC synthesis;

[0143] Navigation bar Surface, H3m, W3m, (x3, y3), scaling, rotation, HWC synthesis.

[0144] The second message sent by HWC to SurfaceFlinger may be:

[0145] Status bar Surface, H1cm, W1cm, (x1, y1), no scaling, no rotation, GPU synthesis;

[0146] Instant messaging Surface, H2cm, W2cm, (x2, y2), scaling, rotation, GPU synthesis;

[0147] Navigation bar Surface, H3m, W3m, (x3, y3), scaling, rotation, HWC synthesis.

[0148] The second message is used to indicate that the recommended synthesis method for the layer to be displayed generated by the status bar Surface rendering and the layer to be displayed generated by the instant messaging Surface rendering is GPU synthesis, and the recommended synthesis method for the layer to be displayed generated by the navigation bar Surface rendering is HWC synthesis.

[0149] S85. The main thread node sends a third message to the calling thread node.

[0150] The third message includes the layer display parameters of the layer marked as GPU synthesis, such as layer parameters, screen parameters and buffer ID of the preset GPU synthesis output. According to the third message, the thread node is called to trigger the GPU to perform layer synthesis.

[0151] The buffer Id output by GPU synthesis is a preset setting, and there is a correspondence between GPU synthesis and the buffer Id output by GPU synthesis. When the buffer Id output by GPU synthesis is certain, GPU synthesis will output the GPU synthesis layer in the buffer corresponding to the buffer Id.

[0152] It is understandable that the buffer queue includes buffers with multiple identity identifiers, such as Buffer A, Buffer B, Buffer C, Buffer D, etc. The preset buffer for GPU synthesis output is Buffer C, and the GPU synthesis layer synthesized by the GPU is output and saved in the buffer corresponding to Buffer C.

[0153] Optionally, the layer parameters include window identification, attributes and synthesis mode, and the screen parameters include screen width, height and screen center coordinates.

[0154] The format of the third message may be: window identifier, attributes, synthesis mode, screen width, screen height, screen center point coordinates, and buffer Id.

[0155] Exemplarily, the third message sent by the main thread node to the RenderEnginer node may be:

[0156] Status bar Surface, H1cm, W1cm, (x1, y1), no scaling, no rotation, GPU synthesis, W0cm, H0cm, (x0, y0), buffer A;

[0157] Instant messaging Surface, H2cm, W2cm, (x2, y2), scaling, rotation, GPU synthesis, W0cm, H0cm, (x0, y0), buffer A;

[0158] S86. Call the thread node to trigger the GPU to perform asynchronous synthesis, and the main thread node executes the subsequent layer processing flow.

[0159] After receiving the third message, the calling thread node first sets the drawing instruction to the GPU. The drawing instruction includes the window identifier, attributes, synthesis mode, screen width, screen height, screen center coordinates and bufferId carried by the third message. When the drawing instruction is set, the Flush function is called to trigger the GPU to perform asynchronous synthesis.

[0160] After the main thread node sends the third message, it executes subsequent processing threads, such as updating the status and the next synthesis time and other subsequent processing contents.

[0161] At this time, the main thread node does not need to wait for the calling thread node to return to the GPU Fence before executing the subsequent process. This reduces the waiting time and shortens the synthesis time of the main thread node. This avoids frame drops caused by long SurfaceFlinger synthesis time, and solves the problem of display interface freezes in scenes such as startup and exit, or real-time window blur.

[0162] S87. The main thread node sends the buffer Id of the GPU synthesis output and the pre-created monitoring signal to the HWC node.

[0163] The main thread node sends the buffer Id of the GPU synthesis output to the HWC node. The HWC node can accurately obtain the GPU synthesis and obtain the GPU synthesis layer according to the buffer corresponding to the bufferId.

[0164] The monitoring signal is a signal pre-created by the main thread node for monitoring whether GPU synthesis is completed. Optionally, the monitoring signal can be a software synchronization monitoring signal (software fence).

[0165] The monitoring signal may be a signal created before step S80, or may be created after the main thread node is started and before the calling thread node is awakened. The application does not specifically limit the pre-creation time of the monitoring signal.

[0166] Optionally, the main thread node may send a first index object corresponding to the monitoring signal to the HWC node. The first index object is used to describe the monitoring content of the monitoring signal. When the monitoring content is different, the first index object corresponds to different values. Among them, the monitoring content includes the completion of GPU synthesis and the incompleteness of GPU synthesis.

[0167] Exemplarily, when the monitored content is that the synthesis of the GPU is completed, the first index object is in a triggered state, and when the monitored content is that the synthesis of the GPU is not completed, the first index object is in an untriggered state.

[0168] Optionally, the triggered state may be identified by a pre-created identifier, such as a signaled identifier. It is understandable that the untriggered state may also be identified by an identifier different from the triggered state, such as an unsignaled identifier.

[0169] Optionally, the first index object may be software Fence FD.

[0170] The HWC node can determine whether the GPU synthesis layer obtained from the buffer corresponding to the buffer Id is in the synthesis completion state according to the first index object.

[0171] For example, if the software Fence FD is in the untriggered state, it means that the GPU synthesis layer obtained by the HWC node from the buffer corresponding to the buffer Id is in an incomplete synthesis state. If the software Fence FD is in the triggered state, it means that the GPU synthesis layer obtained by the HWC node from the buffer corresponding to the buffer Id is in a complete synthesis state.

[0172] Optionally, the first index object may have the same format and content as the second index object corresponding to the GPU Fence. The second index object is used to describe the monitoring content of the GPU Fence.

[0173] For example, when the first index object is a software Fence FD and the second index object is a GPU Fence FD, the format and content of the software Fence FD and the GPU Fence FD are the same.

[0174] Furthermore, in order to improve the ability of the first index object to accurately monitor whether the GPU synthesis is completed, when the second index object is in the triggered state, a pre-created tracking function is called to adjust the first index object to the triggered state. The tracking function is used to block and wait until the second index object is in the triggered state, and adjust the first index object to the triggered state.

[0175] It is worth noting that the tracking function can be located on a RenderEngine node or a SurfaceFlinger node, or can be an asynchronous thread of a RenderEngine or SurfaceFlinger. This embodiment of the present application is not specifically limited.

[0176] S88, HWC node performs layer synthesis and display.

[0177] When the monitoring signal is detected by GPU synthesis, the HWC node executes the layer and sends it to the display.

[0178] Exemplarily, the GPU synthesizes the status bar Surface and the instant messaging Surface to obtain a GPU synthesized layer.

[0179] HWC re-synthesizes the GPU synthesis layer and the layer marked as HWC synthesis to obtain the image frame to be displayed and sends it to the display for display on the display interface.

[0180] Exemplarily, the HWC re-synthesizes the GPU synthesis layer and the navigation bar Surface to obtain an image frame to be displayed.

[0181] See also Fig. 9 , which is a schematic diagram of the processing thread obtained by the layer synthesis method obtained in the embodiment of the present application. At time t1, after SurfaceFlinger sends the third message, RenderEngine executes the processing flow that triggers GPU synthesis, and SurfaceFlinger executes the subsequent processing flow. In this way, when the calling thread node processing process takes too long, the main thread node can be allowed to synchronously execute the next frame, and the two threads are executed in parallel, which improves the synthesis efficiency of the GPU layer, avoids frame loss, and thus avoids the phenomenon of mobile phone freeze, and improves the user's visual experience.

[0182] After the main thread SurfaceFlinger that pre-creates the monitoring signal calls the rendering engine thread RenderEngine, RenderEngine is used to trigger the graphics processor GPU to synthesize the layer marked as GPU synthesis to obtain the GPU synthesis layer. Among them, the monitoring signal is used to monitor whether the GPU synthesis is completed. The monitoring signal is output to the hardware synthesizer HWC, so that the HWC synthesizes the GPU synthesis layer after the monitoring signal detects that the GPU synthesis is completed. In this way, the monitoring signal is used instead of the GPU Fence to monitor the completion of the GPU synthesis. SurfaceFlinger does not need to wait for the acquisition of the GPU Fence before executing the subsequent processing flow, which saves the SurfaceFlinger layer synthesis time, and then saves the total synthesis cycle of the layer synthesis work, avoids the jamming phenomenon, and improves the visual expression of the graphics.

[0183] In order to better illustrate the layer synthesis method provided by the present application, the following takes the Android system as an example, takes the monitoring signal as the software Fence, takes the first index object as the software Fence FD, takes the second index object as the GPU Fencee FD as an example, and combines Fig.10 , another layer synthesis method signaling diagram provided in an embodiment of the present application is described in detail. The method includes:

[0184] S101. Apply a rendering thread node to obtain M rendered layers.

[0185] S102: The application rendering thread node sends the rendered layer parameters to the main thread node.

[0186] S103. The main thread node sends a first message to the HWC node.

[0187] S104. The HWC node returns a second message to the main thread node.

[0188] Steps S101 to S104 are the same as steps S81 to S84 and will not be repeated here.

[0189] S105: The main thread node creates a software Fence FD, and the initialization state is an untriggered state.

[0190] The user creates a software Fence FD through ioctl. Ioctl is a function in the device driver that manages the I / O channels of the device. Managing the I / O channels means controlling some characteristics of the device, such as the transmission baud rate of the serial port. For example, the user creates a software Fence FD through int ioctl (int software Fence FD, int cmd, ...).

[0191] Optionally, determine whether the creation is successful based on the return value of int ioctl. If the return value is 1, it means the creation is successful; if the return value is 0, the creation fails.

[0192] The initial state of the software Fence FD is defined as an untriggered state, and is exemplarily represented by unsignaled.

[0193] S106. The main thread node sends a third message to the calling thread node.

[0194] In the embodiment of the present application, the third message includes not only layer parameters, screen parameters, and the buffer Id of the preset GPU synthesis output, but also software Fence FD.

[0195] Optionally, the software Fence FD is sent to the calling thread node by calling the function DrawLayers. DrawLayers is a function that the main thread node uses to perform GPU compositing on the calling thread node.

[0196] S107, the main thread node executes subsequent threads and calls thread nodes to trigger the GPU to perform asynchronous synthesis.

[0197] After the main thread node sends the software Fence FD to RenderEngine, it does not block and wait for the calling thread node to complete execution, and continues to execute subsequent tasks.

[0198] Optionally, the software Fence FD is sent to each application through Transactioncallback, so that each application can monitor the GPU synthesis end time of SurfaceFlinger.

[0199] S108. The main thread node sends the buffer ID and software Fence FD of the GPU synthesis output to the HWC node.

[0200] Steps S107 and S108 may be performed simultaneously, or S107 may be performed first and then S108. This embodiment of the present application is not specifically limited.

[0201] S109. Call the thread node to obtain the GPU Fence FD.

[0202] Optionally, the thread node pre-registers the trigger function flush function and the GPU Fence FD tracking function eglDupNativeFenceFDANDROID function. The flush function notifies the GPU to start asynchronous layer synthesis. The eglDupNativeFenceFDANDROID function obtains the GPU Fence FD that tracks the GPU synthesis situation.

[0203] Specifically, after receiving the third message, the calling thread node sets the third message including the layer display parameters to the GPU through the drawing instruction. When the setting is completed, the Flush function is called to notify the GPU to start asynchronous synthesis. The calling thread node obtains the GPU Fence FD through the eglDupNativeFenceFDANDROID function.

[0204] S1010, the calling thread node sends the software Fence FD and the GPU Fence FD to the tracking node.

[0205] S1011. The tracking node sets the software Fence FD to a triggered state.

[0206] The tracking node is used to tune the software Fence FD according to the GPU Fence FD.

[0207] Optionally, the tracking node blocks the triggering of the GPU Fence FD according to the pre-created tracking function, thereby triggering the software Fence FD. That is, when the GPU Fence FD is in the triggered state, the software Fence FD is in the triggered state.

[0208] It is worth noting that the tracking function and the RenderEngine thread are asynchronous threads, which are used to block until GPUFenceFD is triggered to trigger the software Fence FD.

[0209] Optionally, the tracking function is an operating system wait function. The tracking node calls the operating system wait function to wait for the GPU Fence FD to be triggered. When the GPU Fence FD in the kernel is in a triggered state, that is, the identifier is signaled, the tracking node calls the kernel through ioctl to set the software Fence FD to signaled, that is, the triggered state.

[0210] S1012, HWC node performs layer synthesis and display.

[0211] S1013. The HWC node clears the software Fence FD, and the main thread node clears the software Fence FD.

[0212] In an embodiment of the present application, after the HWC node returns the second message to the main thread node, the main thread node creates a software FenceFD, the initialization state is an untriggered state, the software Fence FD tracks the GPU Fence FD through a tracking function, and the main thread node sends the buffer ID and software Fence FD of the GPU synthesis output to the HWC node. The HWC can directly obtain the completion status of the GPU synthesis based on the software Fence FD. And the HWC can directly find the corresponding buffer based on the buffer ID and obtain the GPU synthesis layer when the GPU synthesis is completed. In this way, the accuracy and processing efficiency of the layer synthesis are improved.

[0213] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0214] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for synthesizing a layer, characterized in that: Applied to electronic equipment, the method comprises: After the main thread SurfaceFlinger that pre-creates the monitoring signal calls the rendering engine thread RenderEngine, the RenderEngine is used to trigger the graphics processor GPU to synthesize the layer marked as GPU synthesis to obtain the GPU synthesis layer. The monitoring signal is used to monitor whether the synthesis of the GPU is completed; The monitoring signal is output to a hardware synthesizer HWC, so that the HWC synthesizes the GPU synthesis layer after the monitoring signal detects that the synthesis of the GPU is completed.

2. The method according to claim 1, characterized in that: After the main thread SurfaceFlinger that pre-creates the monitoring signal calls the rendering engine thread RenderEngine, the RenderEngine is used to trigger the graphics processor GPU to synthesize the layer marked as GPU synthesis, including: When the SurfaceFlinger calls the RenderEngine, the first index object corresponding to the monitoring signal is passed to the RenderEngine, where the first index object is used to describe the monitoring content of the monitoring signal, and the monitoring content includes the completion of the synthesis of the GPU and the incompleteness of the synthesis of the GPU; According to the RenderEngine, the GPU is triggered to synthesize the layer marked as GPU synthesis, and according to the second index object corresponding to the GPUFence, the first index object is adjusted to determine whether the GPU synthesis is completed according to the first index object, and the second index object is used to describe the monitoring content of the GPU Fence.

3. The method according to claim 2, characterized in that: When the first index object corresponding to the pre-created monitoring signal is in an untriggered state, the untriggered state is used to indicate that the synthesis of the GPU is not completed, and the adjusting the first index object according to the second index object corresponding to GPUFence includes: When the second index object is in a trigger state, the first index object is adjusted to a trigger state, where the trigger state is used to indicate that synthesis of the GPU is completed.

4. The method according to claim 3, characterized in that: When the second index object is in the trigger state, adjusting the first index object to the trigger state includes: When the second index object is in the triggered state, a pre-created tracking function is called to adjust the first index object to the triggered state, and the tracking function is used to block until the second index object is in the triggered state and adjust the first index object to the triggered state.

5. The method according to claim 4, characterized in that: The trace function is created in the following way: When the untriggered second index object is obtained for the first time, the tracking function is created according to the second index object and the first index object.

6. The method according to any one of claims 3 to 5, characterized in that: The method further comprises: When the second index object is in the untriggered state, the first index object is not adjusted.

7. The method according to any one of claims 3 to 6, characterized in that: The trigger state is marked with a preset identifier.

8. The method according to any one of claims 2 to 7, characterized in that: When the SurfaceFlinger calls the RenderEngine, passing the first index object corresponding to the monitoring signal to the RenderEngine includes: Adding the first index object to a pre-registered calling function, the calling function is used to call the RenderEngine to trigger the GPU synthesis; According to the calling function, the first index object is passed to the RenderEngine.

9. The method according to any one of claims 1 to 8, characterized in that: Outputting the monitoring signal to the hardware synthesizer HWC comprises: The first index object corresponding to the monitoring signal and the preset memory identity Buffer Id output by GPU synthesis are output to the HWC, and the buffer Id output by GPU synthesis is used to indicate the identity of the memory that uniquely stores the layer synthesized by the GPU.

10. The method according to any one of claims 1 to 9, characterized in that: The electronic device includes a display framework, and the display framework includes: a main thread node and a calling thread node, the main thread node is used to execute the SurfaceFlinger of the electronic device, and the calling thread node is used to execute the RenderEngine, After the main thread SurfaceFlinger that pre-creates the monitoring signal calls the rendering engine thread RenderEngine, the RenderEngine is used to trigger the graphics processor GPU to synthesize the layer marked as GPU synthesis, including: the main thread node sends the first index object corresponding to the monitoring signal to the calling thread node; The calling thread node triggers the GPU to perform the GPU synthesis on the layer marked as GPU synthesis according to the RenderEngine, so as to obtain the GPU synthesis layer.

11. The method according to claim 10, characterized in that: The display framework further includes a tracking node, and the tracking node adjusts the first index object according to the second index object corresponding to the GPU Fence.

12. The method according to claim 11, characterized in that: When the first index object is a software synchronization monitoring signal file descriptor Fence FD, and the second index object is a GPU Fence FD, the tracking node adjusts the first index object according to the second index object corresponding to the GPU Fence, including: The tracking node calls the operating system wait function to wait for the GPU Fence FD to be in a triggered state; When the GPU Fence FD in the kernel is in a triggered state, the tracking node adjusts the software Fence FD to a triggered state by calling the kernel through ioctl.

13. The method according to any one of claims 1 to 12, characterized in that: The method further comprises: The first index object is added to a transaction callback function, so that a target application monitors the end time of the GPU synthesis through the transaction callback function.

14. The method according to any one of claims 1 to 13, characterized in that: The monitoring signal is a software Fence.

15. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is coupled to the processor; The memory stores a program, and when the program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 14.

Images