Data processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a data processing method and device, equipment, a storage medium and a program product. In the embodiment of the invention, the cancelling task for cancelling the timed task is added in the message queue, the execution state of the frame drawing task is timed through the timed task, and if the frame drawing task is in the set execution state (such as execution is completed or execution is started) before the timed task is ended, the cancelling task cancels the timed task. If the UI thread cannot be in the set execution state before the timed task is ended, sudden load may occur, so that drawing resources of the UI thread are insufficient, the frame drawing task execution time is relatively long, the timed task cannot be canceled when the task is canceled, and the UI thread needs more operation resources, at the moment, the operation resources of the UI thread can be increased in time, and the user experience is improved. Therefore, the UI thread can continue to execute the remaining frame drawing tasks.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to a data processing method, apparatus, device, storage medium and program product. Background Art

[0002] Users can interact with electronic devices through the display screen. For example, they can perform interactive operations (such as clicking or sliding operations) on the display screen to issue interactive instructions to the electronic device. During the interaction process, the drawing operation of the display screen can be performed through the user interface (UI) thread. The drawing operation of the screen consumes certain computing resources. If there is a sudden load such as loading a large picture or querying a database during the execution of the UI thread, the UI thread may drop frames due to insufficient resources. Summary of the invention

[0003] Multiple aspects of the present application provide a data processing method, apparatus, device, storage medium and program product to reduce frame loss caused by insufficient UI thread resources.

[0004] In a first aspect, an embodiment of the present application provides a data processing method, which is applicable to an electronic device, comprising: responding to user operations on an application interface, determining an image frame to be drawn; when a frame drawing trigger signal arrives, adding multiple frame drawing tasks corresponding to the image frame to a message queue of a user interface thread, the message queue corresponding to at least one timed task, each timed task being used to time a target frame drawing task to reach a set execution state; adding at least one cancel task corresponding to at least one timed task to the message queue, the cancel task being used to cancel the corresponding timed task in advance when the corresponding target frame drawing task is in the set execution state; controlling the user interface thread to execute multiple frame drawing tasks and at least one cancel task in the message queue, and if any timed task ends at the scheduled time, increasing the running resources of the user interface thread so that the user interface thread can continue to execute the remaining frame drawing tasks.

[0005] In the data processing method provided in the first aspect, a cancellation task for canceling the scheduled task is added to the message queue, and the execution status of the frame drawing task is timed by the scheduled task. If the frame drawing task is in the set execution state (such as execution completion or start of execution) before the end of the scheduled task, the cancellation task cancels the scheduled task. If it cannot be in the set execution state before the end of the scheduled task, it means that a sudden load may have occurred, resulting in insufficient drawing resources for the UI thread, a long execution time for the frame drawing task, and the cancellation task cannot cancel the scheduled task. The UI thread needs more running resources. At this time, the running resources of the UI thread are increased in time to allow the UI thread to continue to execute the remaining frame drawing tasks.

[0006] In one possible implementation, at least one cancellation task corresponding to at least one scheduled task is added to a message queue, including: for any scheduled task, adding the cancellation task corresponding to any scheduled task as the first target subtask or the last target subtask in the target frame drawing task corresponding to any scheduled task to the message queue; adding cancellation tasks to the message queue for different scheduled tasks, and before the scheduled task ends, if the cancellation task corresponding to the scheduled task is executed, the scheduled task is canceled; at the end of the scheduled task, if the cancellation task corresponding to the scheduled task in the message queue is not executed, the running resources of the UI thread are increased to allow the UI thread to continue to execute the remaining frame drawing tasks in the message queue.

[0007] In a possible implementation, the at least one scheduled task includes a first scheduled task, a second scheduled task and / or a third scheduled task; wherein:

[0008] The first timing task is used to time all the frame drawing tasks, and the timing interval is a first time interval starting from the arrival of the frame drawing trigger signal, and the first time interval is greater than a first preset time length from the arrival of the frame drawing trigger signal to the completion of at least two frame drawing tasks;

[0009] The second timed task is used to time the frame drawing task before the first frame drawing task, and the timing interval is a second time interval starting from the arrival of the frame drawing trigger signal, and the second time interval is greater than a second preset time length from the arrival of the frame drawing trigger signal to the start of execution of the first frame drawing task;

[0010] The third timed task is used to time the second drawing frame task, and the timing interval is a third time interval from the start of the second drawing frame task, and the third time interval is greater than a third preset time length from the start of the second drawing frame task to the completion of the second drawing frame task.

[0011] Optionally, the time point at which the third time interval ends and the time point at which the predicted next frame drawing trigger signal arrives are arrived earlier are taken as the first time point; when the first time point is arrived, if the third timed task is not canceled in advance, the running resources of the UI thread are increased so that the UI thread can continue to execute the remaining frame drawing tasks.

[0012] Optionally, the following provides a method for adding the cancellation task corresponding to the scheduled task to the message queue for various scheduled tasks, as follows:

[0013] For the first scheduled task, add the first cancel task corresponding to the first scheduled task as the last target subtask in the target frame drawing task to the message queue, where the target frame drawing task refers to the last frame drawing task in all frame drawing tasks corresponding to the first scheduled task;

[0014] For the second scheduled task, add the second cancel task corresponding to the second scheduled task as the first target subtask in the target frame drawing task to the message queue, where the target frame drawing task refers to the first frame drawing task corresponding to the second scheduled task;

[0015] For the third scheduled task, the third cancellation task corresponding to the third scheduled task is added to the message queue as the last target subtask in the target frame drawing task, and the target frame drawing task is the second frame drawing task corresponding to the third scheduled task.

[0016] Optionally, during the execution of multiple frame drawing tasks, the execution status of each task in the message queue is monitored; if a target subtask in a target frame drawing task corresponding to any scheduled task is executed, any scheduled task is cancelled.

[0017] In a possible implementation, at least one timing task further includes: a fourth timing task, and the message queue includes: a signal application task located after the plurality of drawing frame tasks; the signal application task is used to apply for the next drawing frame trigger signal; wherein the timing interval of the fourth timing task is a fourth time interval starting from the arrival of the drawing frame trigger signal, and the fourth time interval is greater than a fourth preset duration of preset adjacent drawing frame trigger signals; the method includes:

[0018] When multiple drawing frame tasks in the message queue are completed, the signal application task is executed to apply for the next drawing frame trigger signal; when the next drawing frame trigger signal is received, the fourth timing task is canceled in advance;

[0019] If the fourth scheduled task is completed at the scheduled time, the operating resources of the user interface thread are increased so that the user interface thread can continue to execute the remaining frame drawing tasks.

[0020] In a possible implementation, increasing the operating resources of the UI thread includes:

[0021] Increase the target computing resource object corresponding to the UI thread from the current first frequency to the second frequency;

[0022] and / or

[0023] The occupancy rate of the target computing resource object by the UI thread is increased from a first occupancy rate to a second occupancy rate, so as to increase the running resources of the UI thread.

[0024] Optionally, after the message queue is cleared, the method can also restore the increased operating resources to the initial operating resources, and execute the frame drawing task of the next image frame through the initial operating resources. Specifically, the frequency of the target computing resource object is restored from the second frequency to the first frequency, and the occupancy rate of the UI thread to the target computing resource object is restored from the second occupancy rate to the first occupancy rate.

[0025] In a second aspect, an embodiment of the present application further provides a data processing device, which is applied to an electronic device, and the device includes from top to bottom: an application layer, an application framework layer, a system library and a kernel layer; the application layer includes a target application, the application framework layer includes a message queue, a frame drawing module and a scheduling module; the system library includes: an interface module of the scheduling module; the kernel layer includes: a resource adjustment module;

[0026] The frame drawing module is used to add multiple frame drawing tasks corresponding to the image frame to the message queue of the user interface thread when the frame drawing trigger signal arrives, the message queue corresponds to at least one timed task, and each timed task is used to time the target frame drawing task to reach a set execution state; and call the user interface thread to execute multiple frame drawing tasks and at least one cancel task in the message queue; the image frame is the image frame determined by the target application in response to the user operation to be drawn;

[0027] a scheduling module, configured to add at least one cancellation task corresponding to at least one scheduled task in a message queue, wherein the cancellation task is used to cancel the corresponding scheduled task in advance when the corresponding target frame drawing task is in a set execution state; and during the execution of multiple frame drawing tasks, if any scheduled task ends on time, notifying the resource adjustment module through the interface module to increase the running resources of the user interface thread;

[0028] The resource adjustment module is used to increase the operating resources of the user interface thread so that the UI thread can continue to execute the remaining frame drawing tasks.

[0029] Among them, if the application framework layer and the kernel layer communicate frequently, it will cause a large overhead. In the embodiment of the present application, compared with the implementation method of adding at least one scheduled task and its corresponding at least one cancel task to the kernel layer, at least one scheduled task and its corresponding at least one cancel task are added to the application framework layer. When it is determined that the operating resources of the UI thread can be increased, the application framework layer interacts with the resource adjustment module of the kernel layer through the interface module in the system library, which can reduce the communication overhead between the application framework layer and the kernel layer.

[0030] In a possible implementation, the resource adjustment module includes: a first adjustment module and / or a second adjustment module;

[0031] A first adjustment module is used to increase the target computing resource object corresponding to the UI thread from the current first frequency to the second frequency, so as to increase the running resources of the UI thread;

[0032] The second adjustment module is used to increase the occupancy rate of the UI thread on the target processor from the first occupancy rate to the second occupancy rate, so as to increase the running resources of the UI thread.

[0033] In a third aspect, an embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the data processing method provided in the embodiment of the present application.

[0034] In a fourth aspect, the embodiments of the present application also provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the data processing method provided in the embodiments of the present application.

[0035] In a fifth aspect, the embodiments of the present application also provide a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the data processing method provided in the embodiments of the present application.

[0036] It should be understood that the second to fifth aspects of the embodiments of the present application are consistent with the technical solutions of the first aspect of the embodiments of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0038] Figure 1a A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present application;

[0039] Figure 1b A flowchart of a data processing method provided for an exemplary embodiment of the present application;

[0040] Figure 2a A schematic diagram of increasing operating resources when a first scheduled task is not cancelled provided by an exemplary embodiment of the present application;

[0041] Figure 2b A schematic diagram of increasing the running resources when the second scheduled task is not cancelled provided by an exemplary embodiment of the present application;

[0042] Figure 2c A schematic diagram of increasing operating resources when a third scheduled task is not cancelled provided by an exemplary embodiment of the present application;

[0043] Figure 2d A schematic diagram of increasing operating resources when a fourth scheduled task is not cancelled provided by an exemplary embodiment of the present application;

[0044] Figure 3A software architecture diagram corresponding to an electronic device provided for an exemplary embodiment of the present application;

[0045] Figure 4 A timing interaction diagram is provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0047] In view of the frame loss phenomenon caused by insufficient resources of the UI thread, increasing the operating resources is a direct feature to solve the frame loss problem. Under what conditions is increasing the operating resources the key. In an embodiment of the present application, a cancellation task for canceling the scheduled task is added to the message queue, and the execution status of the frame drawing task is timed by the scheduled task, that is, the progress of the frame drawing is perceived. If the frame drawing task is in the set execution state (such as execution completion or start of execution) before the end of the scheduled task, the scheduled task is canceled by the cancellation task. If it cannot be in the set execution state before the end of the scheduled task, it means that a sudden load may occur, resulting in insufficient drawing resources for the UI thread, and the execution time of the frame drawing task is long. The cancellation task cannot cancel the scheduled task, and the UI thread needs more operating resources. Therefore, the operating resources of the UI thread are increased so that the UI thread can continue to execute the remaining frame drawing tasks.

[0048] Furthermore, during the frame drawing process, by inserting a stake in the frame drawing process of the upper layer of the electronic device (such as the application framework layer) to monitor the frame drawing progress, the frame drawing progress delay caused by burst loads is promptly identified, and the running resources of the UI thread are increased through the module in the kernel layer, so as to achieve the purpose of accelerating frame drawing and reducing frame loss.

[0049] Furthermore, when the application framework layer is inserted and a delay in the frame drawing progress is monitored, the kernel layer is notified to increase the running resources of the UI thread. Compared with the implementation method of inserting a stake in the kernel layer to monitor the frame drawing progress, the method provided by the present application can reduce the communication overhead between the application framework layer and the kernel layer.

[0050] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0051] Figure 1b The following is a flow chart of a data processing method provided by an exemplary embodiment of the present application. Figure 1b As shown, the method includes:

[0052] R201, responding to user operations on the application interface, determining an image frame to be drawn;

[0053] R202. When a frame drawing trigger signal arrives, multiple frame drawing tasks corresponding to the image frame are added to the message queue of the UI thread. The message queue corresponds to at least one timed task, and each timed task is used to time the target frame drawing task to reach a set execution state;

[0054] R203, adding at least one cancel task corresponding to at least one scheduled task in the message queue, where the cancel task is used to cancel the corresponding scheduled task in advance when the corresponding target frame drawing task is in a set execution state;

[0055] R204. Control the UI thread to execute multiple frame drawing tasks and at least one cancellation task in the message queue. If any scheduled task ends on time, increase the running resources of the UI thread so that the UI thread can continue to execute the remaining frame drawing tasks.

[0056] In this embodiment, the application interface of the application can be displayed on the electronic device. The application can be any type of application, for example, a social application, a game application, a video application, and a shopping application. The application interface can be a home page or other pages, without limitation. When the application interface is displayed on the electronic device, the user can perform user operations on the screen of the electronic device. For example, the user operation may include but is not limited to: a sliding operation or a clicking operation, etc. The sliding operation may include but is not limited to: a zoom-in operation, a zoom-out operation, or a drag operation, etc., and the clicking operation may include but is not limited to: a single-click operation or a double-click operation, etc.

[0057] In this embodiment, the electronic device can respond to user operations on the application interface and determine the image frame to be drawn. The image frame to be drawn is an image frame that can be displayed on the UI and is determined in response to the user operation. For example, in a social application, the user can swipe up to view "more posts" or swipe down to "refresh content". The image frame to be drawn is the image frame corresponding to the "more posts" determined in response to the user swiping up, or it can be the image frame corresponding to the "refresh content" determined in response to the user swiping up. One user operation can correspond to one image frame to be drawn, or it can correspond to multiple image frames to be drawn, for example, 2, 3 or 5.

[0058] In this embodiment, the frame drawing trigger signal is a signal that can trigger an electronic device to draw an image frame. The frame drawing trigger signal can be an (On Vertical synchronization, OnVsync) signal (literally translated as a vertical synchronization signal). The frame drawing trigger signal is generally triggered after the synchronization signal arrives. Specifically, after receiving the synchronization signal, the OnVsync method is called to generate a frame drawing trigger signal. After calling the OnVsync method, the frame drawing (doFrame) operation can be performed. doFrame includes various frame drawing tasks, such as input, animation, and drawing.

[0059] Among them, the synchronization signal is a signal that synchronizes the screen refresh rate and frame rate to avoid screen tearing. For example, the synchronization signal can be a vertical synchronization (Vertical Synchronization, Vsync) signal. Generally, an electronic device can send a synchronization signal (Vsync signal) after each frame length (such as 16.6ms). However, this signal is not received by all applications, only subscribed applications can receive it. Each time you subscribe, you can receive a synchronization signal. If you want to receive the next synchronization signal, you can re-subscribe. The advantage of this is: if the UI thread does not change, you do not subscribe to the synchronization signal, and you will not receive the synchronization signal, reducing unnecessary drawing operations.

[0060] Among them, the screen refresh rate (refresh) refers to the number of times the screen refreshes within 1 second, and the unit is hertz (hz). The screen refresh frequency depends on the fixed parameters of the hardware and generally does not change. For example, the screen refresh rate can be 60hz, 90hz or 120hz, etc. Correspondingly, if the screen refresh rate is 60hz, the frame length is 1 / 60=16.7ms, if the screen refresh rate is 90hz, the frame length is 1 / 90=11.1ms, and if the screen refresh rate is 120hz, the frame length is 1 / 120=8.33ms. The frame rate (Frame Rate) refers to the number of frames drawn by the application within 1s, and the unit is frames per second (fps). The frame rate changes dynamically. For example, when the screen is still, no drawing operation is performed, and the data in the buffer is still displayed after the screen is refreshed.

[0061] Among them, after receiving the first frame drawing trigger signal, multiple frame drawing tasks of the first image frame are executed, and then when waiting for the second frame drawing trigger signal to arrive, multiple frame drawing tasks of the second image frame are executed. If multiple frame drawing tasks are completed before the second frame drawing trigger signal arrives, then when the second frame drawing trigger signal arrives, the UI will be refreshed to the content of the first image frame. Otherwise, the UI will not refresh, and will still display the content of the previous image frame, indicating that frame loss has occurred. If a sudden load is found in the UI thread before the second frame drawing trigger signal arrives, the operating resources for the UI thread are increased in time, so that the UI thread completes multiple frame drawing tasks of the second image frame before the second frame drawing trigger signal arrives, then frame loss will no longer occur.

[0062] In this embodiment, the progress of image frame drawing is perceived based on the message queue, which no longer relies on the native drawing logic of the electronic device, and does not need to perceive the logic of the application itself and the load brought by new drawing frameworks such as Flutter. It is no longer limited to the situation where sudden loads can be perceived, and covers more application scenarios. It can more accurately increase the operating resources (such as frequency increase) for the UI thread to ensure the supply of drawing frame resources. Among them, Flutter is a toolkit for building UI.

[0063] In this embodiment, the UI thread corresponds to a message queue, which is used to store various tasks that the UI thread needs to perform. For example, the message queue may include a frame drawing task. A frame drawing task is an operation involved in the drawing process of an image frame, and a frame drawing task may be various callbacks. A frame drawing task may include but is not limited to: input callback (CALLBACK_INPUT), animation callback (CALLBACK_ANIMATION), insert animation callback (CALLBACK_INSETS_ANIMATION), window refresh callback (CALLBACK_TRAVERSAL), commit callback (CALLBACK_COMMIT), drawing callback (CALLBACK_DRAWING), preloading (Recycler View prefetch, RVprefetch), scrolling (RV scroll) and binding view (RV on bind view), etc. Among them, each image frame corresponds to multiple frame drawing tasks, and multiple frame drawing tasks may include multiple of the above-mentioned frame drawing tasks. When a frame drawing trigger signal arrives, multiple frame drawing tasks corresponding to the image frame may be added to the message queue of the UI thread.

[0064] In this embodiment, the message queue corresponds to at least one timed task, and each timed task is used to time the target frame drawing task to reach a set execution state. Among them, the execution state can be started, completed, or being executed. Among them, each timed task can time at least one frame drawing task, and the target frame drawing task can be the last frame drawing task in at least one frame drawing task. It should be noted that when each timed task times a frame drawing task, the one frame drawing task is the target frame drawing task.

[0065] In this embodiment, in addition to adding multiple frame drawing tasks to the message queue, at least one cancellation task corresponding to the scheduled task can also be added to the message queue. Each cancellation task is added to the message queue by inserting a stub, and can be inserted into the target frame drawing task corresponding to each scheduled task. Each scheduled task corresponds to a cancellation task. Each cancellation task is used to cancel the corresponding scheduled task in advance when the corresponding target frame drawing task is in the set execution state. Among them, the set execution state can be that the target frame drawing task is in the start execution state or the execution completion state. Among them, if the target frame drawing task is in the set execution state before the end of the scheduled task, it is considered that the initial running resources of the UI thread are sufficient for the frame drawing, and no frame loss will occur due to the occurrence of a sudden load. There is no need to increase the running resources for the UI thread. The scheduled task is canceled by the cancellation task, and the canceled scheduled task will no longer be timed; if the target frame drawing task is not in the set execution state at the end of the scheduled task, it is considered that a sudden load may occur, resulting in the initial running resources of the UI thread being insufficient to complete the above-mentioned multiple frame drawing tasks, then the running resources can be increased for the UI thread so that the UI thread can continue to execute the remaining frame drawing tasks.

[0066] In this embodiment, the UI thread is controlled to execute each task in the message queue in sequence, wherein the message queue includes: multiple frame drawing tasks and cancellation tasks. In the process of executing each task in the message queue, it is monitored whether the timing of at least one timer has ended. If the timing of any timed task has ended, it means that the target frame drawing task corresponding to any timed task or the task before it takes a long time, and a sudden load may occur, resulting in insufficient running resources for the UI thread to execute the frame drawing tasks. In this case, the running resources of the UI thread are increased so that the UI thread can continue to execute the remaining frame drawing tasks.

[0067] Optionally, if any scheduled task is cancelled by the corresponding cancellation task before completion, it can be considered that no sudden load occurs, and the UI thread can execute the frame drawing task on time based on the initial running resources without increasing the running resources of the UI thread.

[0068] The operating resources may be computing resources or memory resources, etc. The computing resource objects providing computing resources may include but are not limited to: central processing unit (CPU), graphics processing unit (GPU), data processing unit (DPU), tensor processing unit (TPU), cloud infrastructure processor (CIPU) and application specific integrated circuit (ASIC), etc.

[0069] In an embodiment of the present application, a cancellation task for canceling the scheduled task is added to the message queue, and the execution status of the frame drawing task is timed by the scheduled task. If the frame drawing task is in the set execution state (such as completion or start of execution) before the end of the scheduled task, the cancellation task cancels the scheduled task. If it cannot be in the set execution state before the end of the scheduled task, it means that a sudden load may have occurred, resulting in insufficient drawing resources for the UI thread, a long execution time for the frame drawing task, and the cancellation task cannot cancel the scheduled task. The UI thread needs more running resources. At this time, the running resources of the UI thread can be increased in time so that the UI thread can continue to execute the remaining frame drawing tasks.

[0070] In an optional embodiment, the implementation method of adding at least one cancellation task corresponding to at least one scheduled task in the message queue is not limited. For any scheduled task, the cancellation task corresponding to the scheduled task is added to the message queue as the first target subtask in the target frame drawing task of the scheduled task, or the cancellation task corresponding to the scheduled task is added to the message queue as the last target subtask in the target frame drawing task of the scheduled task, or the cancellation task corresponding to the scheduled task can be added to the message queue as the last task in the message queue.

[0071] For example, the message queue includes: frame drawing task B1, frame drawing task B2 and frame drawing task B3. The timing task A1 is used to time the frame drawing task B2. The frame drawing task B2 can be used as the target frame drawing task, and the cancellation task C1 corresponding to the timing task A1 can be added to the message queue as the first target subtask of the frame drawing task B2, or the cancellation task C1 can be added to the message queue as the last target subtask of the frame drawing task B2.

[0072] For another example, the message queue includes: frame drawing task D1, frame drawing task D2, frame drawing task D3 and frame drawing task D4. The timing task A2 is used to time the frame drawing task D1, frame drawing task D2 and frame drawing task D3. The frame drawing task D3 can be used as the target frame drawing task, and the cancellation task C2 corresponding to the timing task A2 can be added to the message queue as the last target subtask of the frame drawing task D3.

[0073] In an optional embodiment, the scheduled task may include at least one of a first scheduled task, a second scheduled task, and a third scheduled task. The first scheduled task, the second scheduled task, and the third scheduled task are exemplarily described below.

[0074] The first timed task refers to timing all the drawing frame tasks of the image frame, and the timing interval is the first time interval from the arrival of the drawing frame trigger signal, and the first time interval is greater than the first preset time length for the UI thread to complete the execution of the at least two drawing frame tasks. The first preset time length may be an empirical value, that is, the first preset time length may be the time length from the arrival of the drawing frame trigger signal to the completion of the execution of the at least two drawing frame tasks by the UI thread through the initial running resources when there is no burst load. For example, if the current frame length is 16.67ms and the first preset time length is 4ms, then the first time interval may be greater than 4ms and less than 16.67ms, for example, the first time interval may be 8ms.

[0075] The second timed task is used to time the drawing frame task before the first drawing frame task, and the timing interval is the second time interval starting from the arrival of the drawing frame trigger signal, the second time interval is greater than the second preset time length from the arrival of the drawing frame trigger signal to the UI thread starting to execute the first drawing frame task, the second preset time length can be an empirical value, and the second preset time length can be the preset time length from the arrival of the drawing frame trigger signal to the UI thread starting to execute the first drawing frame task through the initial running resource when there is no burst load. The first drawing frame task can be a non-first drawing frame task among multiple drawing frame tasks. For example, the first drawing task is implemented as a drawing task, the second preset time length is 1 / 3 of the frame length, such as (1 / 3)*16.67=5.56ms, and the second time interval can be greater than 5.56ms and less than 16.67ms.

[0076] The third timed task is used to time the second drawing frame task, and the timed interval is the third time interval from the start of the second drawing frame task. The second drawing frame task can be any one of a plurality of drawing frame tasks. The time at which the second drawing frame task starts to execute is not certain, and is affected by the execution of other tasks before the second drawing frame task. For example, other tasks can be other drawing frame tasks before the second drawing frame task, or tasks that add multiple image frames to the message queue, or tasks that add at least one cancellation task to the message queue, and there is no limitation on this. If the execution speed of other tasks is fast, the time point at which the second drawing frame task starts to execute is relatively early, and if the execution speed of other tasks is slow, the time point at which the second drawing frame task starts to execute is relatively late. Therefore, the third time interval can be set to be greater than the third preset time length from the start of the second drawing frame task to the completion of the execution of the second drawing frame task by the UI thread. The third preset time length can be an empirical value, and the third preset time length can be a preset time length from the start of the second drawing frame task to the completion of the execution when there is no burst load. For example, the second frame drawing task is implemented as an animation task, and the third preset duration is 1 / 2 of the frame length, such as (1 / 2)*16.67=8.34 ms. The third time interval may be greater than 8.34 ms and less than 16.67 ms.

[0077] Optionally, the time when the second drawing frame task starts to execute is uncertain. If the time when the second drawing frame task starts is late, the time point from the start of the second drawing frame task to the end of the third time interval may be later than the predicted time point when the next drawing frame trigger signal arrives, wherein the implementation method of the predicted time point when the next drawing frame trigger signal arrives is not limited. For example, the time point with an interval of one frame length after the drawing frame trigger signal arrives can be used as the predicted time point when the next drawing frame trigger signal arrives. It can be understood that: when the predicted time arrives, the second drawing frame task has not been completed, resulting in the UI thread not completing the execution of all the drawing frame tasks in the message queue, resulting in frame loss. Based on this, the time point that arrives earlier between the time point when the third time interval ends and the time point when the next drawing frame trigger signal arrives is determined as the first time point; when the first time point arrives, if the third timed task is not canceled in advance, the running resources of the UI thread are increased so that the UI thread can continue to execute the remaining drawing frame tasks.

[0078] Optionally, for the convenience of distinction and description, the cancellation task corresponding to the first scheduled task is called the first cancellation task, the cancellation task corresponding to the second scheduled task is called the second cancellation task, and the cancellation task corresponding to the third scheduled task is called the third cancellation task. According to different scheduled tasks, the target frame drawing task is different, and the implementation method of adding the cancellation task corresponding to the scheduled task to the message queue is also different. An exemplary description is given below.

[0079] For the first scheduled task, the target frame drawing task refers to the last frame drawing task among all frame drawing tasks corresponding to the first scheduled task; the first cancellation task is added to the message queue as the last target subtask in the target frame drawing task.

[0080] For the second scheduled task, the target frame drawing task refers to the first frame drawing task corresponding to the second scheduled task; the second cancellation task is added to the message queue as the first target subtask in the first frame drawing task.

[0081] For the third scheduled task, the target frame drawing task refers to the second frame drawing task corresponding to the third scheduled task. The third cancellation task corresponding to the third scheduled task is added to the message queue as the last target subtask in the second frame drawing task.

[0082] Further optionally, during the execution of multiple frame drawing tasks, the execution status of each task in the message queue can also be monitored; if the target subtask in the target frame drawing task corresponding to any scheduled task is executed, it means that the corresponding cancellation task can be executed, then any scheduled task is canceled, and the target subtask can be the first target subtask or the last target subtask. If any scheduled task ends and the target subtask corresponding to any scheduled task is not executed, the operating resources of the UI thread are increased so that the UI thread can continue to execute the remaining frame drawing tasks.

[0083] An exemplary description is given below with reference to the accompanying drawings.

[0084] For example, the first scheduled task is used to time all the frame drawing tasks of the image frame. The first timing interval is 1 / 2 of the frame length starting from the arrival of the frame drawing trigger signal. If the frame length is 16ms, the first timing interval is 8ms. The first cancellation task can be used as the last target subtask of the last frame drawing task. If the target subtask is executed before the end of the first scheduled task, the first scheduled task is canceled in advance. If the target subtask is not executed when the first scheduled task ends, the operating resources of the UI thread are increased to allow the UI thread to continue to execute the remaining frame drawing tasks. Figure 2a In the example, before the first scheduled task ends, the first cancellation task is not executed. When the first scheduled task ends, the UI thread's running resources are increased from the start of the frame drawing trigger signal to the time point of 8ms interval. The white box represents the frame drawing task, and the black box represents the cancellation task.

[0085] For another example, the second scheduled task is used to time the first frame drawing task. The first frame drawing task is implemented as a frame drawing operation. The second timing interval is 1 / 3 of the frame length starting from the arrival of the frame drawing trigger signal. If the frame length is 16ms, the second timing interval is 5ms. The second cancellation task can be used as the first target subtask of the first frame drawing task. The first target subtask can be implemented as a start drawing (beginDraw) task. If the target subtask is executed before the end of the second scheduled task, and the task has already started to execute the first frame drawing task, the second scheduled task is canceled. If the target subtask has not been executed when the second scheduled task ends, it is considered that the first frame drawing task has not yet started to be executed, and the running resources of the UI thread are increased to allow the UI thread to continue to execute the remaining frame drawing tasks. Figure 2b In the example, before the second scheduled task ends, the first frame drawing task has not started to execute. When the second scheduled task ends, that is, when the time point from the start of the frame drawing trigger signal to the time point of 5ms interval arrives, the operating resources of the UI thread are increased. The white box represents the frame drawing task, and the black box represents the canceled task.

[0086] For another example, the third scheduled task is used to time the second frame drawing task, and the second frame drawing task is implemented as an animation operation. The third scheduled interval is 1 / 2 frame length from the start of the second frame drawing operation. If the frame length is 12ms, 1 / 2 frame length is 6ms. The third cancellation task can be the last target subtask of the second frame drawing task, and the last target subtask can be the end animation (endAnim) task. If the target subtask is executed before the end of the third scheduled task, indicating that the second frame drawing task has been completed, the third scheduled task is canceled. If the target subtask is not executed when the third scheduled task ends, indicating that the second frame drawing task has not been completed, the running resources of the UI thread are increased to allow the UI thread to continue to execute the remaining frame drawing tasks. Figure 2c As shown, before the third scheduled task ends, the second frame drawing task has not been completed. When the third scheduled task ends, that is, from the time when the second frame drawing task starts to execute the function to the time point of 6ms, the operating resources of the UI thread are increased. Among them, the white box represents the frame drawing task, and the black box represents the cancellation task.

[0087] In an optional embodiment, at least one timing task may include a fourth timing task in addition to the first timing task, the second timing task, and the third timing task. The fourth timing task is used to perform delay detection on the drawing frame trigger signal. The timing interval of the fourth timing task is the fourth time interval starting from the arrival of the drawing frame trigger signal. The fourth time interval is greater than the fourth preset duration of adjacent drawing frame trigger signals. The fourth preset duration may be an empirical value. For example, the duration between adjacent drawing frame trigger signals may be one frame. For example, the fourth preset duration is 16.7ms, and the fourth time interval is greater than 16.7ms. For example, the fourth time interval may be 16.7ms plus 1ms and rounded up, that is, Indicates rounding up.

[0088] Among them, in addition to adding multiple frame drawing tasks and at least one cancellation task to the message queue, a signal application task can also be added to the message queue. For example, add the signal application task after the multiple frame drawing tasks. The signal application task is used to apply for the next frame drawing trigger signal. Based on this, when multiple frame drawing tasks in the message queue are executed, the signal application task is executed to apply for the next frame drawing trigger signal. After applying for the next frame drawing trigger signal, you can apply for (subscribe to) the synchronization signal. After the synchronization signal is reached, execute the OnVsync method to generate the next frame drawing trigger signal. Among them, if the next frame drawing trigger signal is received before the end of the fourth timed task, it means that the multiple frame drawing tasks of the image frame have been executed, then the fourth timed task is canceled, and there is no need to increase the running resources of the UI thread. If the fourth timed task ends on time, it means that the multiple frame drawing tasks of the image frame have not been completed, then the running resources of the UI thread are increased so that the UI thread can continue to execute the remaining frame drawing tasks. Figure 2d As shown, the fourth timing task is used for delay detection of the drawing frame trigger signal, and the fourth time interval is 18ms. If the next drawing frame trigger signal is not received at the end of the fourth timing task, the running resources of the UI thread are increased.

[0089] In an optional embodiment, the implementation method of increasing the operating resources of the UI thread is not limited. An exemplary description is given below.

[0090] For example, the target computing resource object (such as CPU) corresponding to the UI thread is increased from the current first frequency to the second frequency to increase the operating resources of the UI thread. For example, the CPU frequency is increased from 2 GHz to 3 GHz, or to the highest frequency of the CPU.

[0091] For another example, the occupancy rate of the UI thread to the target computing resource object is increased from the first occupancy rate to the second occupancy rate to increase the operating resources of the UI thread. The occupancy rate of the UI thread to the target computing resource object can be set by the utilization clamping parameter (UCLAMP), wherein the UCLAMP parameter includes: an upper limit value (UCLAMP_MAX) and a lower limit value (UCLAMP_MIN), and the occupancy rate of the UI thread to the target computing resource object can be between the lower limit value and the upper limit value. UCLAMP_MIN means to increase the occupancy rate of the UI thread, and UCLAMP_MAX means to limit the occupancy rate of the UI thread, then the lower limit value can be increased, for example, the lower limit value can be adjusted to 80%, 90% or 100%, etc.

[0092] For another example, the target computing resource object corresponding to the UI thread is increased from the current first frequency to the second frequency; and the occupancy rate of the target computing resource object by the UI thread is increased from the first occupancy rate to the second occupancy rate to increase the running resources of the UI thread.

[0093] Optionally, after the message queue is cleared, the frequency of the target computing resource object can be restored from the second frequency to the first frequency, and the occupancy rate of the target computing resource object by the UI thread can be restored from the second occupancy rate to the first occupancy rate to reset the running resources of the UI thread, so that the UI thread performs drawing operations on the next image frame through the initial running resources.

[0094] Regarding the embodiments of the present application Figure 1b The detailed implementation and beneficial effects of each step in the method have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0095] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps S201 to S203 can be device A; for another example, the execution subject of steps S201 and S202 can be device A, and the execution subject of step S203 can be device B; and so on.

[0096] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as S201, S202, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0097] In addition to providing a data processing method embodiment, the embodiment of the present application also provides a data processing device. The process of the data processing device provided by the embodiment of the present application is described below.

[0098] Figure 3 A schematic diagram of the structure of a data processing device is provided for an embodiment of the present application. The device includes, from top to bottom: an application layer, an application framework layer, a system library, and a kernel layer. Each layer has a clear role and division of labor. The layers communicate with each other through software interfaces.

[0099] The application layer includes a target application, and the target application may be any application in the application layer.

[0100] The application framework layer may also include: a message queue, a frame drawing module and a scheduling module. The message queue may be a message queue of a UI thread, the frame drawing module may be implemented as a choreographer module, and the scheduling module may be a frame-aware scheduling module.

[0101] The system library includes: an interface module for the scheduling module;

[0102] The kernel layer includes: resource adjustment module.

[0103] The frame drawing module is used to add multiple frame drawing tasks corresponding to the image frame to the message queue of the UI thread when the frame drawing trigger signal arrives, the message queue corresponds to at least one timed task, and each timed task is used to time the target frame drawing task to reach a set execution state; and call the UI thread to execute multiple frame drawing tasks and at least one cancel task in the message queue; the image frame is the image frame to be drawn determined by the target application in response to the user operation;

[0104] The scheduling module is used to add at least one cancellation task corresponding to at least one scheduled task in the message queue, and the cancellation task is used to cancel the corresponding scheduled task in advance when the corresponding target frame drawing task is in the set execution state; and in the process of executing multiple frame drawing tasks, if any scheduled task ends on time, the resource adjustment module is notified through the interface module to increase the running resources of the UI thread;

[0105] The resource adjustment module is used to increase the operating resources of the UI thread so that the UI thread can continue to execute the remaining frame drawing tasks.

[0106] In this embodiment, if the application framework layer and the kernel layer communicate frequently, it will cause a large overhead. In the embodiment of the present application, compared with the implementation method of adding at least one scheduled task and its corresponding at least one cancel task to the kernel layer, at least one scheduled task and its corresponding at least one cancel task are added to the application framework layer. When it is determined that the operating resources of the UI thread can be increased, the application framework layer interacts with the resource adjustment module of the kernel layer through the interface module in the system library, which can reduce the communication overhead between the application framework layer and the kernel layer.

[0107] Optionally, the resource adjustment module includes: a first adjustment module and a second adjustment module.

[0108] A first adjustment module is used to increase the target computing resource object corresponding to the UI thread from the current first frequency to the second frequency, so as to increase the running resources of the UI thread;

[0109] The second adjustment module is used to increase the occupancy rate of the UI thread to the target computing resource object from the first occupancy rate to the second occupancy rate to increase the running resources of the UI thread. The first adjustment module and the second adjustment module may exist simultaneously or separately.

[0110] Among them, the detailed implementation and effect description of the above-mentioned data processing device can be found in the above-mentioned embodiments, which will not be repeated here.

[0111] In some embodiments, when the data processing device is applied to an Android system, in addition to the above-mentioned application layer, application framework layer, system library, and kernel layer, the device also includes an Android runtime in the layer where the system library is located, such as Figure 3 As shown in the figure, the application framework layer, system library, and kernel layer contain Figure 3 In addition to the modules shown, some other modules may be included, but are not included in Figure 3 Shown in.

[0112] The detailed introduction of each layer is as follows:

[0113] The application layer includes a series of applications, which may include but are not limited to: camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0114] The application framework layer provides API and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. The application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0115] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0116] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0117] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0118] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0119] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0120] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0121] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0122] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0123] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0124] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. Figure 3 Only part is shown.

[0125] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0126] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats.

[0127] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0128] A 2D graphics engine is a drawing engine for 2D drawings.

[0129] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0130] In addition to the above internal architecture diagram, this embodiment also provides a timing interaction diagram. Figure 3 The structure shown, such as Figure 4 As shown, the timing interaction diagram includes the following steps:

[0131] S1. The user initiates a sliding operation on the application interface.

[0132] S2. The target application responds to the user operation on the application interface and determines the image frame to be drawn.

[0133] S3. The frame drawing module determines multiple frame drawing tasks of the image frame.

[0134] S4. When a frame drawing trigger signal arrives, the frame drawing module adds the multiple frame drawing tasks to the message queue of the UI thread.

[0135] S5, the scheduling module maintains at least one timed task corresponding to the message queue; each timed task is used to time the target frame drawing task to reach a set execution state;

[0136] S6. The scheduling module adds at least one cancel task to the message queue, where the cancel task is used to cancel the corresponding scheduled task in advance when the corresponding target frame drawing task is in a set execution state;

[0137] S7, the frame drawing module calls the UI thread to execute the task in the message queue;

[0138] S8, the scheduling module monitors the execution status of each task in the message queue during the execution of multiple frame drawing tasks;

[0139] S9. If the scheduling module executes any cancel task, the scheduled task corresponding to the cancel task will be canceled.

[0140] S10. If any scheduled task ends, the scheduling module notifies the interface module in the system library to increase the running resources of the UI thread; wherein the interface module is not in Figure 4 Shown in.

[0141] S11, the first adjustment module increases the target computing resource object corresponding to the UI thread from the current first frequency to the second frequency;

[0142] S12. The second adjustment module increases the occupancy rate of the target computing resource object from the first occupancy rate to the second occupancy rate to increase the running resources of the UI thread so that the UI thread can continue to execute the remaining frame drawing tasks.

[0143] S13. After the message queue is cleared, the scheduling module notifies to reset the frequency boost state and restore the initial running resources of the UI thread;

[0144] S14, the first adjustment module restores the frequency of the target computing resource object from the second frequency to the first frequency;

[0145] S15. The second adjustment module restores the occupancy rate of the target computing resource object by the UI thread from the second occupancy rate to the first occupancy rate.

[0146] The present application also provides an electronic device corresponding to Figure 1a The electronic device 100 shown has Figure 1aVarious modules and components of the electronic device 100 shown. Among them, the processor in the electronic device can implement the following operations when executing the computer program / instruction stored in the memory: respond to the user operation on the application interface and determine the image frame to be drawn; when the frame drawing trigger signal arrives, add multiple frame drawing tasks corresponding to the image frame to the message queue of the user interface thread, the message queue corresponds to at least one timed task, and each timed task is used to time the target frame drawing task to reach the set execution state; add at least one cancellation task corresponding to at least one timed task to the message queue, and the cancellation task is used to cancel the corresponding timed task in advance when the corresponding target frame drawing task is in the set execution state; control the user interface thread to execute multiple frame drawing tasks and at least one cancellation task in the message queue, and if any timed task ends at the scheduled time, increase the running resources of the user interface thread so that the user interface thread can continue to execute the remaining frame drawing tasks.

[0147] For the specific implementation of the processor to perform the above operations, please refer to the above Figure 1b The description in the illustrated method embodiment will not be repeated here.

[0148] The following is a description of the embodiments of the present application. Figure 1a The electronic equipment shown is introduced.

[0149] The electronic device may be a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) or a dedicated camera (such as a SLR camera, a card camera), etc. The present application does not impose any restrictions on the specific type of the electronic device.

[0150] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0151] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0152] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0153] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0154] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0155] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0156] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0157] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0158] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0159] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0160] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0161] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0162] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0163] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0164] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0165] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0166] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0167] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0168] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0169] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0170] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0171] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0172] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0173] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0174] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0175] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0176] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0177] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0178] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0179] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0180] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0181] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0182] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0183] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0184] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0185] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0186] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0187] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0188] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194.

[0189] There are many types, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. Capacitive pressure sensors can be at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0190] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0191] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0192] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0193] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0194] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0195] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0196] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0197] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0198] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0199] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0200] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0201] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0202] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0203] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0204] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0205] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0206] Accordingly, the embodiments of the present application also provide a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is caused to implement each step in the above method embodiment that can be executed by an electronic device.

[0207] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0208] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0209] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0210] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0211] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0212] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0213] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0214] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0215] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: Suitable for electronic devices including: Responding to user operations on the application interface, determining the image frame to be drawn; When a frame drawing trigger signal arrives, multiple frame drawing tasks corresponding to the image frame are added to a message queue of a user interface thread, wherein the message queue corresponds to at least one timed task, and each timed task is used to time a target frame drawing task to reach a set execution state; Adding at least one cancel task corresponding to the at least one scheduled task in the message queue, the cancel task being used to cancel the corresponding scheduled task in advance when the corresponding target frame drawing task is in a set execution state; The user interface thread is controlled to execute multiple frame drawing tasks and at least one cancellation task in the message queue. If any scheduled task ends on time, the running resources of the user interface thread are increased so that the user interface thread can continue to execute the remaining frame drawing tasks.

2. The method according to claim 1, characterized in that Adding at least one cancellation task corresponding to the at least one scheduled task in the message queue includes: For any scheduled task, the cancel task corresponding to any scheduled task is added to the message queue as the first target subtask or the last target subtask in the target frame drawing task corresponding to any scheduled task.

3. The method according to claim 1, characterized in that The at least one scheduled task includes a first scheduled task, a second scheduled task and / or a third scheduled task; wherein: The first timing task is used to time all the frame drawing tasks, and the timing interval is a first time interval starting from the arrival of the frame drawing trigger signal, and the first time interval is greater than a first preset time length from the arrival of the frame drawing trigger signal to the completion of the execution of the at least two frame drawing tasks; The second timed task is used to time the frame drawing task before the first frame drawing task, and the timing interval is a second time interval starting from the arrival of the frame drawing trigger signal, and the second time interval is greater than a second preset time length from the arrival of the frame drawing trigger signal to the start of execution of the first frame drawing task; The third timed task is used to time the second drawing frame task, and the timing interval is a third time interval from the start of the second drawing frame task, and the third time interval is greater than a third preset time length from the start of the second drawing frame task to the completion of the second drawing frame task.

4. The method according to claim 3, characterized in that Also includes: The first time point is the time point when the third time interval ends and the time point when the predicted next frame trigger signal arrives. When the first time point arrives, if the third scheduled task is not canceled in advance, the running resources of the user interface thread are increased so that the user interface thread can continue to execute the remaining frame drawing tasks.

5. The method according to claim 3, characterized in that: Also includes: For the first scheduled task, adding a first cancellation task corresponding to the first scheduled task as the last target subtask in a target frame drawing task to the message queue, where the target frame drawing task refers to the last frame drawing task in all frame drawing tasks corresponding to the first scheduled task; For the second scheduled task, adding the second cancellation task corresponding to the second scheduled task as the first target subtask in the target frame drawing task to the message queue, where the target frame drawing task refers to the first frame drawing task corresponding to the second scheduled task; For the third scheduled task, a third cancellation task corresponding to the third scheduled task is added to the message queue as the last target subtask in the target frame drawing task, and the target frame drawing task is the second frame drawing task corresponding to the third scheduled task.

6. The method according to claim 5, characterized in that Also includes: During the execution of the plurality of frame drawing tasks, monitoring the execution status of each task in the message queue; If a target subtask in a target frame drawing task corresponding to any scheduled task is executed, the scheduled task is canceled.

7. The method according to claim 3, characterized in that The at least one timing task further includes: a fourth timing task, the message queue includes: a signal application task located after the plurality of drawing frame tasks; the signal application task is used to apply for a next drawing frame trigger signal; wherein the timing interval of the fourth timing task is a fourth time interval starting from the arrival of the drawing frame trigger signal, and the fourth time interval is greater than a fourth preset duration of preset adjacent drawing frame trigger signals; the method includes: When multiple drawing frame tasks in the message queue are completed, the signal application task is executed to apply for the next drawing frame trigger signal; when the next drawing frame trigger signal is received, the fourth timing task is canceled in advance; If the fourth scheduled task is completed on time, the operating resources of the user interface thread are increased so that the user interface thread can continue to execute the remaining frame drawing tasks.

8. The method according to claim 1, characterized in that Increasing the operating resources of the user interface thread includes: Increasing the target computing resource object corresponding to the user interface thread from the current first frequency to the second frequency; and / or The occupancy rate of the target computing resource object by the user interface thread is increased from a first occupancy rate to a second occupancy rate, so as to increase the running resources of the user interface thread.

9. The method according to claim 8, characterized in that After the message queue is cleared, the method further includes: The frequency of the target computing resource object is restored from the second frequency to the first frequency, and the occupancy rate of the target computing resource object by the user interface thread is restored from the second occupancy rate to the first occupancy rate.

10. A data processing device, characterized in that: The device is applied to an electronic device, and the device comprises from top to bottom: an application layer, an application framework layer, a system library and a kernel layer; the application layer comprises a target application, the application framework layer comprises a message queue, a frame drawing module and a scheduling module; the system library comprises: an interface module of the scheduling module; the kernel layer comprises: a resource adjustment module; The frame drawing module is used to add multiple frame drawing tasks corresponding to the image frame to the message queue of the user interface thread when a frame drawing trigger signal arrives, the message queue corresponds to at least one timed task, and each timed task is used to time the target frame drawing task to reach a set execution state; and call the user interface thread to execute the multiple frame drawing tasks and at least one cancel task in the message queue; the image frame is the image frame determined by the target application to be drawn in response to the user operation; The scheduling module is used to add at least one cancellation task corresponding to the at least one scheduled task in the message queue, wherein the cancellation task is used to cancel the corresponding scheduled task in advance when the corresponding target frame drawing task is in a set execution state; and in the process of executing the multiple frame drawing tasks, if any scheduled task ends on time, the resource adjustment module is notified through the interface module to increase the running resources of the user interface thread; The resource adjustment module is used to increase the operating resources of the user interface thread so that the user interface thread can continue to execute the remaining frame drawing tasks.

11. The device according to claim 10, characterized in that The resource adjustment module includes: a first adjustment module and / or a second adjustment module; The first adjustment module is used to increase the target computing resource object corresponding to the user interface thread from the current first frequency to the second frequency, so as to increase the running resources of the user interface thread; The second adjustment module is used to increase the occupancy rate of the user interface thread on the target processor from a first occupancy rate to a second occupancy rate, so as to increase the running resources of the user interface thread.

12. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method according to any one of claims 1 to 9.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 9.

14. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 9.

Images