A control method of a graphics processor, an electronic device, and a storage medium

Abstract

The application provides a control method of a graphics processor, an electronic device and a storage medium, and relates to the technical field of terminals. The method comprises the following steps: obtaining sampling data of a touch sensor of an electronic device; when a first sliding move event after a pressing down event is recognized according to the sampling data, determining whether a current frame is a first frame of the sliding operation; and when it is determined that the current frame is the first frame of the sliding operation, waking up a GPU. The method wakes up the GPU before application drawing business is issued. When the application drawing task of the first frame is issued, the GPU that is woken up in advance according to the application scheme is still in an active state, can directly perform GPU drawing, does not need to be woken up again, the first frame drawing task can be directly executed without being blocked, the drawing and display process can be quickly completed, the sliding response delay is reduced, the frame loss and the freezing problem are significantly improved, and the system fluency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, and particularly relates to a control method of a graphics processing unit, an electronic device and a storage medium. BACKGROUND

[0002] At present, the screens of electronic devices such as mobile phones and tablet computers generally support touch functions, and a user can perform a touch operation on the screen, and the electronic device updates the display content of the screen in response to the touch operation of the user.

[0003] The touch response process includes a complete process from the user contacting the screen to the screen displaying a feedback response. The timeliness of the touch response process can be evaluated by a sliding response delay, which can be defined as the time interval between the user contacting the screen to start the sliding operation and the display content of the screen first starting to change.

[0004] The electronic device uses a graphics processing unit (GPU) to perform picture rendering. The GPU is usually configured to enter an idle state when the idle time exceeds a preset time threshold. When the GPU enters the idle state, if there is a rendering task, the GPU needs to be woken up again, so that the GPU is switched from the idle state to an active state. The switching process consumes time, causing the rendering task to be in a blocked state of waiting for scheduling execution and being unable to be interrupted during the GPU state switching process, causing the rendering task of the first frame of the sliding operation to be unable to be completed in time, causing the first frame of the picture to be stuck, and increasing the sliding response delay. SUMMARY

[0005] To solve the above problems, the present application provides a control method of a graphics processing unit, an electronic device and a storage medium, which can alleviate the first frame of the sliding operation being stuck and significantly reduce the sliding response delay.

[0006] In a first aspect, the present application provides a control method of a graphics processing unit, which is applied to an electronic device including a touch sensor, and the method includes: acquiring sampling data of the touch sensor of the electronic device; when a first sliding move event after a press down event is recognized according to the sampling data, determining whether a current frame is a first frame of a current sliding operation; and when it is determined that the current frame is the first frame of the current sliding operation, waking up a GPU.

[0007] With the method, when a first move event after the down event is identified and it is determined that the current frame is the first frame of the sliding, power-on logic of the GPU is executed to wake up the GPU. At this time, the wake-up of the GPU occurs before the application drawing task is issued. After the GPU is woken up, the GPU can maintain the Active state for a preset threshold. In actual application, the time from the identification of the first move event to the issuance of the application drawing task of the first frame of the sliding is less than the preset threshold, so when the application drawing task of the first frame of the sliding is issued, the GPU woken up in advance according to the method of the application is still in the Active state and can directly perform GPU drawing without being woken up again. At this time, the first frame drawing task can be directly executed without being blocked, the drawing and display process can be quickly completed, and the sliding response delay is reduced, the frame loss and lag problem is significantly improved, and the system fluency is improved.

[0008] In a possible implementation, the determination of whether the current frame is the first frame of the sliding operation includes:

[0009] When it is determined that the current frame is the first frame after the down event, it is determined that the current frame is the first frame of the sliding operation.

[0010] In a possible implementation, the determination of whether the current frame is the first frame of the sliding operation includes:

[0011] When the SurfaceFlinger does not receive a vertical synchronization VSync signal after the down event, it is determined that the current frame is the first frame of the sliding operation, and the VSync signal indicates that the SurfaceFlinger starts to read the buffer for composition.

[0012] When the SurfaceFlinger does not receive a vertical synchronization VSync signal after the down event, it is determined that the current frame is the first frame of the sliding operation, and the VSync signal indicates that the SurfaceFlinger starts to read the buffer for composition.

[0013] In a possible implementation, when it is determined that the current frame is the first frame of the sliding operation, a graphics processing unit GPU is woken up, and the waking up of the GPU includes:

[0014] When the SurfaceFlinger determines that the current frame is the first frame of the sliding operation, the SurfaceFlinger sends a wake-up signal to a kernel graphics support KGSL layer.

[0015] The KGSL layer wakes up the GPU after receiving the wake-up signal.

[0016] In a possible implementation, the KGSL layer wakes up the GPU after receiving the wake-up signal, and the waking up of the GPU includes:

[0017] After the KGSL layer receives the wake-up signal, the GPU wake-up node is updated to call the GPU activation function to activate the GPU.

[0018] In a possible implementation, the GPU wake-up node includes a flag bit, and updating the GPU wake-up node to call the GPU activation function to activate the GPU specifically includes:

[0019] The flag bit is updated from the second value to the first value to call the GPU activation function to activate the GPU, the flag bit being the second value representing that the GPU activation function is not called, and the flag bit being the first value representing that the GPU activation function is called.

[0020] In a possible implementation, when the first move event after the press down event is identified according to the sampling data, it is determined whether the current frame is the first frame of the current sliding operation, specifically including:

[0021] The touch sensor sends the sampling data to a touch sensor driver;

[0022] The touch sensor driver identifies the down event and the move event according to the sampling data, and reports the down event and the move event to an Input subsystem;

[0023] The Input subsystem sends a call request to a SurfaceFlinger when the move event is first identified after the down event is received;

[0024] The SurfaceFlinger identifies the call request of the Input subsystem to determine whether the current frame is the first frame of the current sliding operation.

[0025] In a possible implementation, when it is determined that the current frame is not the first frame of the current sliding operation, the method further includes:

[0026] When an application drawing task of the current frame is sent to the KGSL layer, the GPU is woken up.

[0027] In a second aspect, the present application further provides an electronic device, which includes a processor and a memory, the memory being used to store a program, the program being executed by the processor to implement the control method of the graphic processor provided in the first aspect and any implementation manner of the first aspect.

[0028] By using the electronic device, the GPU is woken up in advance, so that the first frame drawing task can be directly executed without being blocked, the drawing and display process can be quickly completed, and the sliding response time delay is reduced, the frame loss and the lag problem are significantly improved, and the system fluency is improved.

[0029] In a third aspect, the present application also provides a storage medium having a computer program stored thereon, which, when executed by a processor of an electronic device, implements the control method of the graphic processor provided in the above first aspect and any one of the implementation manners of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A schematic diagram of a touch response process is provided for the present application;

[0031] Figure 2 A schematic diagram of a display process is provided for the present application embodiment;

[0032] Figure 3 A scene schematic is provided for the present application embodiment Figure 1 ;

[0033] Figure 4 A principle schematic is provided for the present application embodiment Figure 1 ;

[0034] Figure 5 A flowchart of a control method of a graphic processor is provided for the present application embodiment;

[0035] Figure 6 A flowchart of another control method of a graphic processor is provided for the present application embodiment;

[0036] Figure 7 A principle schematic is provided for the present application embodiment Figure 2 ;

[0037] Figure 8 A schematic diagram of an electronic device is provided for the present application embodiment;

[0038] Figure 9 A software structure schematic diagram of an electronic device is provided for the present application embodiment. DETAILED DESCRIPTION

[0039] In order to make the person skilled in the art more clearly understand the scheme of the present application, first, the touch response process related to the technical scheme of the present application will be specifically described.

[0040] Referring to Figure 1 , the figure is a schematic diagram of a touch response process provided for the present application.

[0041] The electronic device includes a touch sensor and a touch screen (TP). For a one-time screen touch event, the touch sensor first scans and samples the touch screen to identify the touch event and collect initial data, and the touch sensor can also detect the occurrence time of the touch event.

[0042] A corresponding hardware interrupt is sent to the kernel layer TP driver. The TP driver processes the initial data using touch algorithms to produce raw input events (including touch coordinates, timestamps of touch operations, etc.). The raw input events are stored in the kernel layer.

[0043] The raw input events are reported to the Input subsystem in the application framework layer, which identifies the control to which the input event corresponds. Specifically, the Input subsystem includes an InputFlinger module, which collects, processes, and distributes various input events (key events, touch events, etc.) in the application framework layer. The Input subsystem can distribute input events to application processes through the view system to trigger the application rendering process.

[0044] The view system (View) includes visual controls, such as controls that display text and controls that display images. The view system can be used to build an application. A display interface can be composed of one or more views. View is an abstraction of a user interface layer control, and is the foundation of all controls in the Android system, which can sense events such as pressing and sliding.

[0045] After the application rendering is complete, the display process begins. To ensure smooth display without frame loss, the display process of an electronic device running the Android system includes GPU rendering, SF synthesis, and driving display.

[0046] The vertical synchronization (VSync) signal of the electronic device synchronizes the image frame rate of the application with the refresh rate, which helps to establish stability. The VSync signal includes a VSync-app signal and a VSync-sf signal. The application starts rendering based on the Vsync-app signal and sends the completed buffer to the SurfaceFlinger for synthesis. The role of the SurfaceFlinger is to accept multiple sources of graphical display data, synthesize them, and then send them to the display device.

[0047] The SurfaceFlinger starts to take buffers for synthesis based on the VSync-sf. When the SurfaceFlinger receives the VSync-sf signal, the SurfaceFlinger traverses its layer list to find new buffers. If the SurfaceFlinger finds new buffers, the SurfaceFlinger acquires the buffers; otherwise, the SurfaceFlinger continues to use the last acquired buffer.

[0048] The synthesized layer is selected to be refreshed to hardware composer (HWC) synthesis or GPU synthesis according to the specification parameters. The finally synthesized layer is sent to display by a display driver, and is refreshed to a screen when a screen hardware tearing effect (TE) signal arrives, thereby completing the entire display process.

[0049] Further, a fence mechanism is added in the Android system. The following is specifically described in combination with the accompanying drawings.

[0050] Referring to Figure 2 , this figure is a schematic diagram of a display process provided by an embodiment of the present application.

[0051] The fence mechanism is a resource synchronization mechanism. After the fence mechanism is added in the buffer flow process, whether the fence is available is detected in the next operation. If the fence is available, the buffer is taken for consumption. If the fence is not available, the buffer is consumed after the fence is available. Therefore, when the driving is sent to the display, the fence needs to be released after the synthesis process is completed before the driving can continue to work, otherwise the driving will be in a waiting state, resulting in frame loss.

[0052] Currently, a GPU is generally configured to enter an idle (Idle) state when an idle time exceeds a preset time threshold. The Idle state refers to a state in which the GPU is idle, and at this time, it does not perform any work or task. A common specification of the preset time threshold is generally 80 milliseconds. When the GPU enters the Idle state, if there is a drawing task at this time, the adreno driver is reawakened through a kernel graphics support layer (KGSL) layer, that is, the GPU is switched from the Idle state to an Active state, and the GPU is powered on to work again.

[0053] However, due to the limitation of the hardware device, the process of powering on the GPU again will consume time, at this time, the related drawing instructions stored in the KGSL layer cannot be executed, and need to wait until the GPU is powered on to continue to execute. Therefore, these drawing tasks are in a blocked state of waiting for scheduling execution and cannot be interrupted during the state switching process of the GPU. The following is described in combination with a specific example.

[0054] Referring to Figure 3 , this figure is a schematic diagram of a scenario provided by an embodiment of the present application Figure 1 .

[0055] Since users generally perform sliding actions on the screen frequently when playing games, and the perception of the sliding response delay is more obvious, an electronic device running a game application is taken as an example for description.

[0056] As Figure 3 shown in the scenario, for a complete sliding action, including the process of pressing the screen, starting the sliding action and stopping the sliding. Specifically, at T1 time, the user presses the screen, at this time, the sliding has not yet been performed, the touch_down event of the user is sampled, that is, the pressing event, which is referred to as down event in the following description. At T2 time, the user starts to slide, and the touch_move event of the user is sampled, that is, the sliding event, which is referred to as move event in the following description. From T2 time to T3 time is the sliding process, during which the touch_move event of the user can be sampled multiple times. At T3 time, the sliding stops, the user stops pressing the screen, at this time, the touch_up event of the user can be sampled, that is, the lifting event, which is referred to as up event in the following description.

[0057] In actual control process, due to the uncertainty of the user's hand interaction action, the time interval from T1 time to T2 time is uncertain, that is, there is an uncertain time delay from the touch_down event sampled by the touch to the first reported touch_move event, and the application first starts to consume the touch_move event for drawing is also affected by the application lock point threshold and the sliding speed, so the time interval from the time when the drawing task of the first frame of the sliding is issued to the GPU to the time when the last GPU processing drawing task is uncontrollable, and once this time interval exceeds 80 ms, the GPU will inevitably switch to the Idle state, and when the first frame drawing task is executed, the action of switching the GPU from the Idle state to the Active state will inevitably occur, at this time, the device power-on time of the GPU will directly affect the execution of the first frame drawing task of the sliding, resulting in that the system cannot complete the drawing and display process within the expected time, and finally causing poor sliding response time delay data.

[0058] For electronic devices supporting GPU hardware rendering, when the application program Open GL interface is used to draw the user interface (user interface, UI), the UI of the Android application is rendered by the Hardware accelerated rendering engine for user interface (HWUI) technology. The main thread of the upper application creates a view, and calls the OpenGL interface of the HWUI through the RenderThread thread of the application.

[0059] Wherein, the RenderThread thread is a processing thread controlled by the system, which can keep the animation smooth when the UI thread is blocked. Then the interface of the Device Development Kit (DDK) is called, and the GPU DDK calls the GPU Driver to finally translate the instructions into various register operations that can be recognized by the GPU hardware to drive the GPU hardware to complete the task. Therefore, the blocking time of the RenderThread thread can represent the size of the sliding response delay.

[0060] Referring to Figure 4 The figure is a principle schematic provided by the embodiment of the application Figure 1 .

[0061] The input reader (InputReader) of the InputFlinger module acquires the input event, and when it is determined that the input event is the first move event after the down event, the input dispatcher (InputDispatcher) distributes the input event to the application process through the view system to trigger the application drawing process.

[0062] Then, if the GPU needs to be powered on again after the application drawing task is stored to the KGSL layer, the related drawing instructions stored in the KGSL layer are in an uninterruptible sleep state of waiting for scheduling execution and cannot be interrupted during the GPU state switching process. The RenderThread thread is blocked, and the RenderThread thread can normally run to realize GPU drawing after the GPU is powered on. After the GPU drawing is completed, SF synthesis is performed.

[0063] Through testing, the blocking time of the RenderThread thread can reach tens of milliseconds or even twenty-some milliseconds, which increases the sliding response delay. The delay can cause the first frame of picture to appear a lagging abnormality, which affects the hand following property of the system.

[0064] In order to solve the above technical problems, the application provides a control method of a graphics processor, an electronic device and a storage medium. When the first sliding move event after the pressing down event is recognized by the touch sensor, and it is determined that the current frame is the first frame of the sliding operation, the GPU is woken up, that is, the wake-up of the GPU occurs before the application drawing service is issued. When the application drawing task of the first frame of the subsequent sliding is issued, the GPU has been woken up, and the GPU drawing can be directly performed. At this time, the first frame drawing task can be directly executed without being blocked, the drawing and display process can be quickly completed, the sliding response delay is reduced, the frame loss and lag problem is significantly improved, and the system fluency is improved.

[0065] In order to make the person skilled in the art more clearly understand the scheme of the present application, the technical scheme in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0066] Hereinafter, the terms "first" and "second" are only used for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0067] The embodiments of the present application provide a control method of a graphics processor, which will be specifically described below in conjunction with the drawings.

[0068] Referring to Figure 5 The figure is a flowchart of the control method of the graphics processor provided by the embodiments of the present application.

[0069] The method comprises the following steps:

[0070] S11: Obtain sampling data of a touch sensor of an electronic device.

[0071] When the user's hand contacts the display screen of the electronic device, the touch sensor scans and samples the touch screen to identify a touch event and collect initial data, and a corresponding hardware interrupt is given to the touch sensor driver in the kernel layer.

[0072] The touch sensor driver can determine that a down event occurs at present by using a touch algorithm. The down event represents that the user currently presses the touch screen, but has not yet performed sliding.

[0073] The touch sensor driver can also determine that a move event occurs at present by using the touch algorithm. The move event represents that the position of the contact point between the user's finger and the screen moves to reach a sliding threshold.

[0074] S12: When the first move event after the down event is identified according to the sampling data, determine whether the current frame is the first frame of the sliding operation.

[0075] If yes, execute S13, otherwise, execute S14.

[0076] A swipe operation can include one down event, multiple move events and one up event. Due to the uncertainty of user hand interaction actions, the time interval between the down event and the first move event after the down event is uncertain. If the time interval is short, there can be no frame rendering, drawing, driving display process between the down event and the first move event, and thus the first frame of the swipe appears after the first move event is reported. If the time interval is long, there is a frame rendering, drawing, driving display process between the down event and the first move event, that is, the first frame of the swipe appears before the first move event is reported.

[0077] S13: Wake up the GPU.

[0078] When the current frame is determined to be the first frame of the current swipe operation, that is, the first frame of the swipe appears after the first move event is reported, the power-on logic of the GPU is executed to wake up the GPU. At this time, the rendering, drawing, driving display process of the first frame of the swipe has not started, that is, the wake-up of the GPU occurs before the application drawing task is issued to the KGSL layer, and when the application drawing task of the first frame of the swipe is issued to the KGSL layer, the GPU has been woken up, so that the GPU drawing can be directly performed without task blocking.

[0079] S14: Wake up the GPU after the application drawing task is issued to the KGSL layer.

[0080] When it is determined that the current frame is not the first frame of the swipe, the GPU is woken up by the KGSL layer according to the conventional logic, that is, the GPU is woken up after the application drawing task is issued to the KGSL layer.

[0081] In summary, by using the method provided in the embodiments of the present application, when the first move event after the down event is recognized and it is determined that the current frame is the first frame of the swipe, the power-on logic of the GPU is executed to wake up the GPU. At this time, the wake-up of the GPU occurs before the application drawing task is issued to the KGSL layer, that is, the wake-up of the GPU is completed before the relevant drawing instructions are stored in the KGSL layer. When the GPU is woken up, the GPU can maintain the Active state for a preset threshold. In actual applications, the time from the recognition of the first move event to the issuance of the application drawing task of the first frame of the swipe to the KGSL layer is less than the preset threshold, so that when the application drawing task of the first frame of the swipe is issued to the KGSL layer, the GPU woken up in advance by using the method of the present application is still in the Active state, and the GPU drawing can be directly performed without being woken up again. At this time, the first frame drawing task can be directly executed without being blocked, the drawing and driving display process can be quickly completed, the swipe response time delay is reduced, the frame loss and lag problems are significantly improved, and the system fluency is improved.

[0082] The specific implementation will be described below.

[0083] Referring to Figure 6 and Figure 7 . Wherein, Figure 6 a flow chart of another control method of a graphic processor provided by the embodiment of the application; Figure 7 a principle schematic Figure 2 .

[0084] S20: The touch sensor scans and samples at a fixed period.

[0085] When the user's hand contacts the display screen of the electronic device, the touch sensor scans and samples the touch screen to identify a touch event and collect initial data, and a corresponding hardware interrupt is sent to the touch sensor driver in the kernel layer.

[0086] The embodiment of the application does not specifically limit the scanning and sampling period of the touch sensor.

[0087] S21: The touch sensor sends the current sampling data to the touch sensor driver.

[0088] S22: The touch sensor driver identifies a down event according to the current sampling data.

[0089] The touch sensor driver determines that a down event occurs by using a touch algorithm. The down event represents that the user currently presses the touch screen, but has not yet performed sliding.

[0090] S23: The touch sensor driver reports the down event to the Input subsystem.

[0091] S24: The touch sensor driver identifies a move event according to the current sampling data.

[0092] The touch sensor continues to scan and sample at a fixed period, and continuously sends the current sampling data to the touch sensor driver, and the touch sensor driver continues to identify the pressing event according to the current sampling data. When the user presses the screen but has not yet performed sliding, no move event is identified at this time.

[0093] The user's finger starts to slide on the screen of the electronic device, at which time the touch sensor continuously reports the sampling data to the touch sensor driver, and the touch sensor driver identifies a move event after determining that the position of the contact point moves to reach a sliding threshold according to the sampling data.

[0094] Once the move event is identified, it is characterized that the user starts to slide at this time, and the touch sensor driver starts to release the displacement report point, that is, the touch sensor driver starts to report the move event to the Input subsystem, and carries the coordinate information corresponding to the move event and the timestamp and other information in the move event reported to the Input subsystem. A sliding operation can include a down event, multiple move events and an up event.

[0095] S25: The touch sensor driver reports the move event to the Input subsystem.

[0096] Specifically, the move event is reported to the InputFlinger module of the Input subsystem. The InputFlinger module collects, processes, and distributes various input events at the application framework layer.

[0097] S26: When the Input subsystem first identifies the move event after the down event, a vertical synchronization enabling signal is called to enable the vertical synchronization signal.

[0098] The scheme of the present application is to reduce the sliding response delay when the first frame of sliding is drawn. The GPU resources required for drawing need to be preloaded when the first frame of sliding is identified, that is, the GPU is controlled to switch to the Active state when the first frame of sliding is identified.

[0099] The first identified move event specifically refers to the first move event after the down event of a sliding operation, which means that the sliding starts.

[0100] In one possible implementation, the Input subsystem can send a call request to the SurfaceFlinger to call wakeupVSync to enable the VSync model.

[0101] S27: The SurfaceFlinger identifies the call request of the Input subsystem.

[0102] When the Input subsystem calls the vertical synchronization enabling signal to enable the vertical synchronization signal (VSync), the SurfaceFlinger side in the scheme of the present application identifies the call request of the Input to determine whether it is the first frame of sliding.

[0103] The first frame of sliding is the first frame corresponding to the present sliding operation. A complete sliding operation generally corresponds to a down event, multiple move events and an up event. Due to the uncertainty of user hand interaction action, the time for detecting the first move event is uncertain.

[0104] by Figure 7 For example, the time interval between the down event and the first move event after the down event is represented by t0. The length of t0 is unpredictable. If t0 is short, there may be no frame rendering, drawing, and drive delivery process between the down event and the first move event. That is, the first frame of the slide appears after the first move event is reported. In this case, the solution in this application wakes up the GPU in advance to reduce the sliding response latency.

[0105] If the time interval between the down event and the first move event is too long, there is a frame rendering, drawing, and drive delivery process between the down event and the first move event. That is, the first frame of the slide appears before the first move event is reported. In this case, the KGSL layer wakes up the GPU according to the usual logic, that is, the GPU is woken up after the application drawing task is sent to the KGSL layer.

[0106] S28: SurfaceFlinger determines that the current frame is the first frame of the slide.

[0107] In one possible implementation, when SurfaceFlinger receives a call request from the Input subsystem, it determines that the current frame is the first frame after the down event.

[0108] In another possible implementation, SurfaceFlinger determines that the current frame is the first frame of a slide when it determines that it has not yet started fetching the buffer for first frame composition based on VSync-sf. In practical applications, SurfaceFlinger determines that the current frame is the first frame of a slide when it has not received a VSync signal after the down event of the current slide operation. The VSync signal instructs SurfaceFlinger to start reading the buffer for composition.

[0109] Understandably, when it is determined that the current frame is not the first frame of the slide, the KGSL layer wakes up the GPU according to the usual logic, that is, the GPU is woken up after the application drawing task is sent to the KGSL layer.

[0110] S29: SurfaceFlinger sends a wake-up signal to the KGSL layer.

[0111] When SurfaceFlinger determines that the current frame is the first frame of a slideshow, to prevent related drawing instructions in the KGSL layer from being blocked while waiting for the GPU to power on, SurfaceFlinger adds a step of sending a wake-up signal to the KGSL layer. This wake-up signal is used to control the KGSL layer to execute the GPU power-on logic.

[0112] S30: After receiving the wake-up signal, the KGSL layer executes the GPU power-on logic to wake up the GPU.

[0113] The KGSL layer receives the wake-up signal and calls the driver function to activate the GPU.

[0114] In a possible implementation, the driver function is an adreno function, the adreno function wakes up the adreno driver, and the adreno driver can implement the function of activating the GPU. Adreno TM the driver of the GPU.

[0115] In a possible implementation, a GPU wake-up node is added to the KGSL layer, which can be represented as touch_wake. The touch_wake node can include a flag, for example, when the flag is a first value 1, it indicates that the adreno function needs to be called to activate the GPU, and when the flag is a second value 0, it indicates that the adreno function does not need to be called to activate the GPU at this time. The flag is written as 1 when the KGSL layer receives the wake-up signal, and 0 otherwise. It can be understood that the flag can also use other parameter values to indicate that the adreno function needs to be called to activate the GPU, which will not be described herein again. After the GPU is activated, the normal process of waking up the VSync signal can be continued.

[0116] If the GPU is in the Active state at this time, the GPU refreshes the preset time threshold, that is, the timing of the preset time threshold is restarted, and the activation state of the preset time threshold can be continued. If the GPU is in the Idle state, the GPU starts switching from the Idle state to the Active state and continues to maintain the activation state of the preset time threshold.

[0117] The technical effects of the scheme of the present application will be described below in combination with Figure 1 , Figure 4 and Figure 7 .

[0118] The GPU is configured to enter the Idle state when the idle time exceeds the preset time threshold, and the preset time threshold is not limited in the embodiments of the present application. In a typical configuration mode, the preset time threshold can be set to 80 ms.

[0119] For Figure 1 and Figure 4In the prior related solution in the related art, the Input subsystem does not send a call request to the SurfaceFlinger. After the application drawing task is stored to the KGSL layer, the KGSL layer re-wakes up the GPU. After the GPU wakes up, the GPU draws the UI by calling the interface of the OpenGL of the HWUI through the RenderThread thread. After the GPU draws, the SurfaceFlinger performs SF synthesis.

[0120] In the technical solution of the present application, as shown in Figure 7 the Input subsystem sends a call request to the SurfaceFlinger when the move event is first identified after the down event. The SurfaceFlinger not only performs SF synthesis, but also has the function of identifying the call request. When it is identified that the current is the first frame of sliding, the SurfaceFlinger sends a wake-up signal to the KGSL layer to make the KGSL layer wake up the GPU in advance. The wake-up of the GPU by the KGSL layer occurs before the application drawing task is stored to the KGSL layer, that is, the wake-up of the GPU is completed before the relevant drawing instructions are stored to the KGSL layer. When the GPU is woken up, the GPU can maintain the Active state for 80 milliseconds.

[0121] In actual application, the time for the application drawing task of the first move event to the first frame of sliding to be stored to the KGSL layer is less than 80 milliseconds. Therefore, when the application drawing task of the first frame of sliding is stored to the KGSL layer, the GPU woken up in advance by the technical solution of the present application is still in the Active state and can directly draw the GPU. The following is a specific example.

[0122] Through simulation test, Figure 4 the blocking time in the related art is about 26.7 milliseconds, that is, the time for waking up the GPU is 26.7 milliseconds. Taking the maximum value of 80 milliseconds for the time for the application drawing task of the first move event to the first frame of sliding to be stored to the KGSL layer as an example. The time consumption of S26-S29 in the above steps in the system is t1. After the KGSL layer receives the wake-up signal, the time for executing the power-on logic of the GPU to wake up the GPU is 26.7 milliseconds obtained by simulation. Therefore, the remaining time between the time when the GPU switches to the Active state and the time when the application drawing task of the first frame of sliding is stored to the KGSL layer is 80 milliseconds-26.7 milliseconds-t1=53.3 milliseconds-t1. Obviously, the time is less than the preset time threshold. That is, when the application drawing task of the first frame of sliding is stored to the KGSL layer, the GPU is in the Active state and does not need to be woken up again. At this time, the first frame drawing task can be directly executed without being blocked, the drawing and display process can be quickly completed, and the sliding response time delay is reduced, the frame loss and the freezing problem are significantly improved, and the system fluency is improved.

[0123] The above steps of the embodiments of the present application are only for convenient description and do not constitute a limitation on the technical solutions of the present application. For example, the behavior of the touch sensor in S20 above is continuously performed multiple times throughout the process, and in the above description, only one step is replaced. For another example, due to the shorter sampling period and higher sampling frequency of the touch sensor in actual application, in the scenario of fast sliding of the user, the electronic device can start the drawing process of the first frame of picture only after recognizing multiple move events. Therefore, in another possible implementation manner, in S26, the Input subsystem can also call the vertical synchronization enabling signal to enable the vertical synchronization signal when the first preset number of move events are continuously recognized after the down event. The first preset number can be set to a small positive integer, for example, set to 2 or 3, which can also achieve the early activation of the GPU and can also reduce the sliding response delay to a certain extent.

[0124] Based on the control method of the graphics processor provided in the above embodiments, the embodiments of the present application further provide an electronic device, which will be specifically described below with reference to the accompanying drawings.

[0125] Referring to Figure 8 , the figure is a schematic diagram of an electronic device provided by the embodiments of the present application.

[0126] The embodiments of the present application do not specifically limit the type of electronic device, and the electronic device can be a mobile phone, a tablet computer, a wearable electronic device (for example, a smart watch), a notebook computer, a smart screen, etc., and the following description takes the electronic device as a mobile phone as an example.

[0127] The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195, etc.

[0128] Among them, the sensor module 180 can include a touch sensor 180A, and can also include other types of sensors, which will not be described herein again.

[0129] It can be understood that the structure of the embodiments of the present application does not constitute a specific limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than those shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0130] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a GPU, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0131] The processor 110 can also be provided with a memory for storing instructions and data.

[0132] The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0133] The electronic device 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0134] The internal memory 121 can be used to store computer executable program codes, including instructions. The internal memory 121 can also be referred to simply as memory, and the stored program codes are executed by the processor to realize the control method of the graphics processor in the above embodiments of the present application.

[0135] The touch sensor 180A can be disposed on the display screen 194, and the touch sensor 180A and the display screen 194 form a touch screen, also referred to as a "touch panel". The touch sensor 180A is configured to detect a touch operation applied thereto or in the vicinity thereof. The touch sensor 180A can transmit the detected touch operation to the application processor to determine a touch event type. Visual output related to the touch operation can be provided through the display screen 194. In some embodiments, the touch sensor 180A can also be disposed on the surface of the electronic device 100, at a position different from that of the display screen 194.

[0136] The software structure of the electronic device 100 is exemplarily described in the embodiments of the present application by taking an Android system with a layered architecture as an example.

[0137] Referring to Figure 9 FIG. 4 is a block diagram of a software structure of an electronic device according to an embodiment of the present application.

[0138] The Android system includes, from top to bottom, an application layer, an application framework layer, a system runtime library layer, a hardware abstraction layer (HAL), and a kernel layer (Linux Kernel). For the convenience of description, Figure 9 Some devices of the hardware layer are also shown in FIG. 4.

[0139] The application layer can include an application 1, an application 2, and a series of applications.

[0140] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0141] The application framework layer can specifically include an Input subsystem, a window manager system (WMS), a content provider, a View system, a phone manager, a resource manager, a notification manager, and the like.

[0142] Window Manager Service (WMS): mainly responsible for managing window views on the Android device, and controlling the interface and correct display and input of the application.

[0143] View: includes visual controls, such as controls that display text, controls that display images, and so on. Views can be used to build an application. A display interface can be composed of one or more views. View is an abstraction of a control that is a part of the interface layer, and is the base class of all controls in Android, which can sense events such as pressing, sliding, and so on.

[0144] The system runtime library layer mainly includes the Android runtime library and the native C / C++ library.

[0145] The native C / C++ library can include a plurality of functional modules. For example, a surface 1 manager (surface manager), media libraries, a three-dimensional graphics processing library (for example, OpenGL ES), a 2D graphics engine (for example, SGL), and the like.

[0146] The Android runtime library includes a core library and an Android runtime (ART). The ART is responsible for scheduling and management of the Android system, and is a running environment on the Android operating system. In the ART environment, the bytecode is pre-compiled into machine code when the application is installed for the first time, so that it becomes a real native application.

[0147] HWUI (hardware accelerated rendering UI) is a module added by Android for 2D hardware drawing. Before HWUI, Android mainly used skia for software drawing. Due to the problems of drawing performance and the like, almost all the drawing of Android now uses hwui hardware accelerated drawing. HWUI mainly uses OpenGL for GPU hardware drawing to improve the drawing performance of the entire system.

[0148] libgui is a GUI library, which is used to create a graphical user interface.

[0149] The function of SurfaceFlinger is to accept multiple sources of graphical display data, synthesize them, and then send them to the display device.

[0150] The hardware abstraction layer (Hardware Abstraction Layer, HAL) is a routine package of the software layer, which is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware, and to simulate the details of a specific system platform so that the program can directly access the resources of the hardware.

[0151] HAL interface definition language (HIDL) is an interface definition language that specifies interfaces between a HAL and its users. HIDL allows types and method calls to be specified (which are then assembled into interfaces and software packages).

[0152] The composer is a hardware compositor that takes the composited layer parameters sent from the SurfaceFlinger to perform composition.

[0153] The kernel layer (Linux Kernel) is a layer between hardware and software.

[0154] The display subsystem frame buffer (DSS Frame Buffer) is used to implement display driving of a display screen and output of display port (DP) data.

[0155] The electronic device adopts the technical solution of the present application, and the touch sensor performs scanning sampling at a fixed period and sends current sampling data to the touch sensor driver.

[0156] The touch sensor driver identifies a down event and a move event using a touch control algorithm according to the current sampling data, and reports the down event and the move event to the Input subsystem.

[0157] When the Input subsystem first identifies the move event after the down event, the Input subsystem calls a vertical synchronization enabling signal to enable a vertical synchronization signal. At this time, the Input subsystem can send a call request to the SurfaceFlinger.

[0158] The SurfaceFlinger identifies the call request of the Input subsystem, and when the SurfaceFlinger determines that it is the first frame of sliding, the SurfaceFlinger sends a wake-up signal to the KGSL layer. After receiving the wake-up signal, the KGSL layer executes GPU power-on logic to wake up the GPU.

[0159] By using the electronic device provided in the present application, the first frame drawing task can be directly executed without being blocked, the drawing and display process can be quickly completed, and the sliding response delay is reduced, thereby significantly improving the frame loss and lag problem and improving the system fluency.

[0160] Further, the present application also provides a storage medium having a program stored thereon, and the program is executed by a processor of an electronic device to implement the control method of the graphics processor provided in the above embodiments.

[0161] Storage media includes permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of storage media include, but are not limited to, parameter random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technologies.

[0162] It should be understood that in this application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association between the associated objects, which means that there can be three kinds of relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0163] The above and the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A control method of a graphics processor, characterized by, The method is applied to an electronic device, and comprises: Obtaining sampling data of a touch sensor of the electronic device; When a first slide move event after a press down event is identified according to the sampling data, determining whether a current frame is a first frame of a current slide operation; When it is determined that the current frame is the first frame of the current slide operation, waking up a graphic processor GPU.

2. The method of claim 1, wherein, The determination of whether the current frame is the first frame of the current slide operation specifically comprises: When it is determined that the current frame is the first frame after the down event, it is determined that the current frame is the first frame of the current slide operation.

3. The method of claim 1, wherein, The determination of whether the current frame is the first frame of the current slide operation specifically comprises: When the SurfaceFlinger determines that no vertical synchronization VSync signal is received after the down event, it is determined that the current frame is the first frame of the current slide operation, and the VSync signal indicates that the SurfaceFlinger starts to read a buffer for composition.

4. The method of claim 1, wherein, The waking up of the GPU when it is determined that the current frame is the first frame of the current slide operation specifically comprises: When the SurfaceFlinger determines that the current frame is the first frame of the current slide operation, it sends a wake-up signal to a kernel graphics support KGSL layer; The KGSL layer wakes up the GPU after receiving the wake-up signal.

5. The method of claim 4, wherein, The waking up of the GPU by the KGSL layer after receiving the wake-up signal specifically comprises: After receiving the wake-up signal, the KGSL layer updates a GPU wake-up node to call a GPU activation function to activate the GPU.

6. The method of claim 5, wherein, The GPU wake-up node comprises a flag bit, and the updating of the GPU wake-up node to call the GPU activation function to activate the GPU specifically comprises: The flag bit is updated from a second value to a first value to call the GPU activation function to activate the GPU, and the flag bit being the second value represents that the GPU activation function is not called, and the flag bit being the first value represents that the GPU activation function is called.

7. The method of claim 1, wherein, The determination of whether the current frame is the first frame of the current slide operation when the first slide move event after the press down event is identified according to the sampling data specifically comprises: The touch sensor sends the sampling data to a touch sensor driver; The touch sensor driver identifies the down event and the move event according to the sampling data, and reports the down event and the move event to an Input subsystem; The Input subsystem sends a calling request to the SurfaceFlinger when the move event is identified for the first time after the down event is received; The SurfaceFlinger identifies the calling request of the Input subsystem to determine whether the current frame is the first frame of the current slide operation.

8. The method of claim 1, wherein, When it is determined that the current frame is not the first frame of the current slide operation, the method further comprises: The GPU is woken up when an application drawing task of the current frame is sent to the KGSL layer.

9. An electronic device, comprising: The electronic device includes a processor and a memory for storing a program, which, when executed by the processor, implements the control method of the graphics processor of any one of claims 1-8.

10. A storage medium, characterized by The storage medium has a program stored thereon, which, when executed by an electronic device, implements the control method of the graphics processor of any one of claims 1-8.

Images