Interface image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an interface image processing method and device, electronic equipment and a storage medium, relates to the technical field of electronic equipment, and can improve the picture processability. The interface image processing method comprises the steps of executing an input event task and a drawing event task of a t-th frame in the interface image processing process of the t-th frame; in the interface image processing process of the t-th frame, after the input event task and the drawing event task of the t-th frame are executed, the animation event task of the (t + 1)-th frame is executed.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to an interface image processing method, device, electronic equipment and storage medium. Background Art

[0002] When users operate the interface of electronic devices such as mobile phones and tablets, such as clicking or sliding pages, the processor will render the interface to update the screen. If the processor load is high or the processor resources are occupied by other processes, the image processing process of the interface may be delayed, resulting in screen display freezes and unsmooth sliding operations. Summary of the invention

[0003] The technical solution of the present application provides an interface image processing method, device, electronic device and storage medium, which can improve the flow of the screen.

[0004] In a first aspect, an interface image processing method is provided, comprising: during the interface image processing process of the tth frame, executing the input event task and the drawing event task of the tth frame; during the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, executing the animation event task of the t+1th frame.

[0005] In a possible implementation, during the interface image processing of the tth frame, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: after receiving the vertical synchronization Vsync signal of the tth frame, determining whether the animation event task of the tth frame has been executed, if so, executing the input event task and the drawing event task of the tth frame; if the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and the drawing event task of the tth frame, execute the animation event task of the t+1th frame.

[0006] In a possible implementation, the interface image processing method further includes: after receiving the vertical synchronization Vsync signal of the tth frame, if it is determined that the animation event task of the tth frame is not executed, executing the input event task, animation event task and drawing event task of the tth frame.

[0007] In a possible implementation, the interface image processing method further includes: after executing the input event task and the drawing event task of the tth frame, if the time before receiving the Vsync signal of the t+1th frame is not enough to execute the animation event task of the t+1th frame, then executing the rendering task of the tth frame.

[0008] In one possible implementation, if the time before the Vsync signal of the t+1th frame is received is sufficient to execute the animation event task of the t+1th frame, then before the Vsync signal of the tth frame is received, after the input event task and the drawing event task of the tth frame are executed, the process of executing the animation event task of the t+1th frame includes: if the time before the Vsync signal of the t+1th frame is received is sufficient to process the animation event task of the t+1th frame, then before the Vsync signal of the tth frame is received, after the input event task and the drawing event task of the tth frame are executed, the animation event task of the t+1th frame is executed by delaying the interval time.

[0009] In one possible implementation, after receiving the vertical synchronization Vsync signal of the t-th frame, determining whether the animation event task of the t-th frame has been executed, if so, executing the input event task and the drawing event task of the t-th frame includes: after the display event receiver receives the vertical synchronization Vsync signal of the t-th frame, determining whether the animation event task of the t-th frame has been executed, if so, removing the animation event task of the t-th frame from the task list, and sending an interface image processing request for the t-th frame to the choreographer; the choreographer executes the input event task and the drawing event task of the t-th frame in the task list in response to the interface image processing request for the t-th frame.

[0010] In a possible implementation, before the display event receiver receives the vertical synchronization Vsync signal of the tth frame, it also includes: the view root layout sends a callback request to the choreographer in response to the input event; the choreographer adds callback information in response to the callback request and sends a Vsync request to the display event receiver.

[0011] In one possible implementation, input event tasks, animation event tasks, drawing event tasks, and rendering tasks are all executed based on the first type core of the processor, and the processor includes a first type core, a second type core, and a third type core. The frequency of the first type core is lower than the frequency of the second type core, and the frequency of the first type core is greater than the frequency of the third type core.

[0012] In a second aspect, an interface image processing device is provided, including: a first module, used to execute the input event task and the drawing event task of the tth frame during the interface image processing process of the tth frame; a second module, used to execute the animation event task of the t+1th frame after executing the input event task and the drawing event task of the tth frame during the interface image processing process of the tth frame.

[0013] In a third aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store at least one instruction, and when the instruction is loaded and executed by the processor, the electronic device executes the above-mentioned interface image processing method.

[0014] In a fourth aspect, a computer-readable storage medium is provided, comprising a program or an instruction. When the program or the instruction runs on a computer, the above-mentioned interface image processing method is executed.

[0015] The interface image processing method, device, electronic device and storage medium in the embodiments of the present application advance the animation event task of the t+1th frame to the interface image processing process of the tth frame, thereby reducing the probability of screen freeze caused by the interface image of the t+1th frame not being processed in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1a It is a timing diagram of an interface image processing process in the related art;

[0017] Figure 1b It is a timing diagram of another interface image processing process in the related art;

[0018] Figure 2 This is a structural block diagram of an electronic device in an embodiment of the present application;

[0019] Figure 3 This is a software structure diagram in an embodiment of the present application;

[0020] Figure 4 This is a timing diagram of an interface image processing process in an embodiment of the present application;

[0021] Figure 5 A flowchart of an interface image processing process in an embodiment of the present application;

[0022] Figure 6 A schematic diagram of the signaling relationship between multiple modules in a software structure in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0024] Before describing the embodiments of the present application, the related technologies and their problems are described first. Figure 1a and Figure 1b As shown, after receiving the vertical synchronization (Vsync) signal, the electronic device sequentially executes the input event task, the animation event task and the drawing event task to realize interface rendering, and then sends the rendered interface to the display to realize the display of each frame. Figure 1a and Figure 1bThe circular F mark in the middle indicates the time it takes to display the rendered interface. If the drawing event task of a certain frame is delayed, or even delayed to the next frame, it will cause a conflict between the rendering of two adjacent frames, resulting in screen display freezes, unsmooth sliding operations, etc. Figure 1b As shown, if the drawing event task 3 of the 3rd frame is delayed to after the display time of the 3rd frame, it will cause abnormal display of the 3rd frame. In order to solve this problem, the embodiment of the present application is described below.

[0025] like Figure 2 As shown, an embodiment of the present application provides an electronic device 100, which may include a processor 110, an internal memory 121, a sensor module 180, a display screen 194, etc.

[0026] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0027] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0028] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0029] The communication function of the electronic device 100 can be implemented in a wired or wireless manner.

[0030] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0031] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0032] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0033] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0034] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0035] Figure 3 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0036] The application layer can include a series of application packages. Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. The application framework layer may include managers and view system, etc.

[0037] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. The display interface can be composed of one or more views. The manager includes activities, which are one of the basic components of the Android operating system and can be used to provide users with interactive interfaces or views in applications. When an application calls another application, the calling application calls the activity in the other application instead of the entire application. In this way, the activity acts as an entry point for the application to interact with the user.

[0038] (Android Runtime) includes core libraries and virtual machines.

[0039] (Android Runtime) is responsible for the scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0040] The system library can include multiple functional modules. For example: surface manager, media library, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc. The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0041] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver.

[0042] Some concepts and application scenarios involved in the embodiments of the present application are explained below.

[0043] The view system includes the view root layout (ViewRootlmpl), which is used to manage the root view (View) of the window, control the measurement, layout and drawing (draw) of the view, and the distribution interaction of all window events is executed or transmitted through the view root layout.

[0044] The view system also includes a choreographer, which is a class in the operating system that coordinates the timing of input event tasks, animation event tasks, and draw event tasks. The choreographer executes the input event task through the input callback method (CALLBACK_INPUT), and the input event task is the data that the application encapsulates and distributes the input event to the choreographer. The input event can be the interaction between the user and the application monitored by the application. After determining the input event, the application performs preliminary processing on the input event, generates an input event task for the input event, and passes the input event task to the choreographer. The choreographer executes the animation event task through the animation callback method (CALLBACK_ANIMATION), and the animation event task is the data required for the application to calculate the animation. The choreographer executes the measure event task, layout event task, and draw event task through the traversal callback method (CALLBACK_TRAVERSAL). The view root layout and the choreographer are both modules in the view system.

[0045] The activity includes a display event receiver (DisplayEventReceiver), which is a module in the activity that monitors the vertical synchronization (Vsync) signal and controls the choreographer.

[0046] Taking the electronic device as a mobile phone as an example, the scenario involved in the embodiment of the present application includes the process in which the electronic device updates and displays the image based on the user interaction when the user touches the application interface or the system main interface (desktop), such as clicking or sliding. When the user clicks or slides on the interface, an input event is triggered, and the user interface (UI) thread executes the input event task, animation event task and drawing event task through the choreographer, and sends the data after the drawing is completed to the rendering thread. The rendering thread performs the actual rendering based on the data after the UI (User Interface) thread completes the drawing, and the rendered data will be displayed.

[0047] like Figure 4 As shown, an embodiment of the present application provides an interface image processing method, the method comprising: in the interface image processing process of the tth frame, executing the input event task and the drawing event task of the tth frame; in the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, executing the animation event task of the t+1th frame.

[0048] Specifically, Figure 4The Arabic numerals after each time task in the numerals represent the frame number corresponding to the time task. For example, in the first frame, after the input event task and the drawing event task of the first frame are executed, before the second frame, if the animation event task of the second frame has been received, since the animation event task does not depend on the next frame and the user operation, and the processing process of the interface image of the first frame has been completed, the animation event task of the second frame can be executed before the second frame. In this way, when the second frame is reached, since the animation event task of the second frame has been executed in the first frame, after the input event task and the drawing event task of the second frame are executed in the second frame, the rendering and display process of the interface image of the second frame can be continued. Compared with the related art, the rendering and display process of the interface image of the second frame can be executed faster. That is to say, assuming that a complex scene is encountered in the second frame, the load is large, and the resources are crowded with each other, in the related art, since the animation event task of the second frame needs to be executed in the second frame, it takes more time, which may result in not having enough time to perform the rendering and display process of the interface image of the second frame, thereby causing the screen to freeze. In the interface image processing method of the embodiment of the present application, since the animation event task of the second frame has been executed in advance in the first frame, the time occupied by the interface image processing process of the second frame is shorter in the second frame. Therefore, the probability that the interface image processing process of the second frame will not be processed in time is smaller, that is, the probability of screen freeze is lower.

[0049] The interface image processing method can be applied to the electronic device 100 described above.

[0050] The electronic devices involved in this application may be smart TVs, mobile phones, tablet computers, personal computers (PCs), personal digital assistants (PDAs), smart watches, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, vehicle-mounted devices, drone devices, smart cars, smart speakers, robots, smart glasses, and any other products.

[0051] The interface image processing method in the embodiment of the present application advances the animation event task of the t+1th frame to the interface image processing process of the tth frame, thereby reducing the probability of the interface image of the t+1th frame not being processed in time and causing the screen to freeze. In other words, in the embodiment of the present application, the idle time of the processor after the corresponding task is processed in one frame is used to advance the processing of the animation event task that does not depend on the next frame and user operation. In this way, when a complex scene is encountered in the next frame, the probability of problems such as frame loss and freeze caused by excessive processor load and crowded resources not being able to process tasks in time can be reduced.

[0052] In a possible implementation, Figure 5 As shown, in the interface image processing process of the t-th frame, after executing the input event task and the drawing event task of the t-th frame, the process of executing the animation event task of the t+1-th frame includes: after step 101, receiving the vertical synchronization Vsync signal of the t-th frame, executing step 102, determining whether the animation event task of the t-th frame has been executed, if so, executing step 103, removing the animation event task of the t-th frame from the task list, and executing step 104, executing the input event task and the drawing event task of the t-th frame; if not, that is, if it is determined that the animation event task of the t-th frame has not been executed, executing step 105, executing the input event task, the animation event task and the drawing event task of the t-th frame; in step After the process of step 104 and step 105, step 106 is executed to determine whether the time before receiving the Vsync signal of the t+1th frame is sufficient to process the animation event task of the t+1th frame. If so, that is, if the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, step 107 is executed to execute the animation event task of the t+1th frame and step 108 is executed to execute the rendering task of the tth frame. If not, that is, if the time before receiving the Vsync signal of the t+1th frame is insufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, step 108 is executed. In step 106, judging whether the time is sufficient is not judging whether the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, but judging whether the idle time of the processor before receiving the Vsync signal of the t+1th frame other than executing necessary tasks (such as rendering tasks) is sufficient to execute the animation event task of the t+1th frame. If it is determined in step 106 that there is enough time, the animation event task of the t+1th frame is executed in advance in the tth frame, and then the rendering task of the tth frame is executed. After rendering, it is sent for display, that is, step 109 is executed to display the interface image of the tth frame, and then the t+1th frame is entered. Since the animation event task of the t+1th frame has been executed in advance in the tth frame, this task does not need to be executed in the t+1th frame, and the t+1th frame has more events to execute other tasks. If it is determined in step 106 that there is not enough time, the animation event task of the t+1th frame is not executed in the tth frame, but other necessary tasks are directly executed, including steps 108 and 109. In this way, it can be ensured that there is more time to execute the task of displaying the interface image of the tth frame.

[0053] In a possible implementation, if the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: if the time before receiving the Vsync signal of the t+1th frame is sufficient to process the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and the drawing event task of the tth frame, delay the interval time, execute the animation event task of the t+1th frame, and then execute the rendering task of the tth frame and display the interface image of the tth frame, and the interval time is, for example, a few ms. The idle time of the processor after executing the input event task and drawing event task of the tth frame and before the t+1th frame is determined. For example, if the processor idle time is a milliseconds, the time required to process the animation event task of the t+1th frame is b milliseconds, and the preparation time required to process the animation event task of the t+1th frame is c milliseconds, as long as a≥b+c, then any time between c and abc can be used as the interval time, and the preparation time required to process the animation event task of the t+1th frame can be, for example, the time to release the process lock. By setting the interval time, exceptions caused by the continuous execution of the tasks of the tth frame and the t+1th frame can be prevented.

[0054] In a possible implementation, Figure 6As shown, the following describes the embodiment of the present application based on the signaling relationship between multiple modules in the software structure. First, the view root layout sends a callback request to the choreographer in response to an input event. The input event is, for example, a user's sliding or clicking on the interface. The callback request is, for example, a postCallback(CALLBACK_XXX) method; the choreographer adds callback information in response to the callback request and sends a vertical synchronization Vsync request to the display event receiver. Adding callback information can be achieved by, for example, the addCallbacklocked method. The Vsync request is, for example, the scheduleVsync method. Vsync is provided by the operating system and is used to coordinate and synchronize the choreographer, i.e., the signal of the operation of other modules. The display event receiver sends a vertical synchronization (Vsync) request in response to the choreographer's Vsync request, such as the nativeSchedeleVsync method; then the display event receiver receives the vertical synchronization signal, such as the dispatchVsync method; after the display event receiver receives the vertical synchronization Vsync signal of the t-th frame, it receives data through the onVsync method and determines whether the animation event task of the t-th frame has been executed. If so, the animation event task of the t-th frame is removed from the task list through the remove(postAnim) method, and the interface image processing request of the t-th frame is sent to the choreographer through the doFrame method; the choreographer executes the input event task and drawing event task of the t-th frame in the task list in response to the interface image processing request of the t-th frame, for example, the input event task is executed through the doCallback(CALLBACK_INPUT) method, and the doCallback(CALLBACK_ANIMATION) method can also be executed, but since the animation event task of the t-th frame has been removed from the task list, the animation event task of the t-th frame will not be actually executed, and then the drawing event task is executed through the doCallback(CALLBACK_TRAVERSAL) method. If the choreographer has enough time to process the animation event task of frame t+1 before receiving the Vsync signal of frame t+1, the animation event task of frame t+1 is registered by postAnim method after a delay of Tms, and the animation event task of frame t+1 is executed by doCallback(CALLBACK_ANIMATION) method. Then the rendering task of frame t is executed and the interface image of frame t is displayed. Then, in frame t+1, the view root layout is restarted from the step of sending a callback request to the choreographer in response to the input event to realize the interface image processing process of frame t+1. The specific principle is the same as that of the interface image processing process of frame t, which will not be repeated here.

[0055] Since the choreographer executes the doCallback(CALLBACK_INPUT) method, the doCallback(CALLBACK_ANIMATION) method and the doCallback(CALLBACK_TRAVERSAL) method in response to the interface image processing request of the current frame in each frame process, the above describes the situation where the animation event task of the current frame is removed from the task list. If the animation event task of the current frame is not removed from the task list, that is, the animation event task of the current frame is not executed in advance, then the execution of the doCallback(CALLBACK_ANIMATION) method will trigger the execution of the animation event task of the current frame. In addition, if the choreographer does not have enough time to process the animation event task of the next frame before receiving the Vsync signal of the next frame, the animation event task of the next frame will not be executed in advance, but the rendering task of the current frame and the necessary tasks of the current frame such as sending to display will be executed normally to avoid the situation where resources are crowded and there is no time to process in complex scenes.

[0056] In a possible implementation, input event tasks, animation event tasks, drawing event tasks, and rendering tasks are all executed based on the first type of core of the processor, and the processor includes a first type of core, a second type of core, and a third type of core, and the frequency of the first type of core is less than the frequency of the second type of core, and the frequency of the first type of core is greater than the frequency of the third type of core. The first type of core can be called a medium core, the second type of core can be called a large core, and the third type of core can be called a small core. The higher the frequency of the core, the greater the power consumption, and the lower the frequency, the lower the power consumption. In the prior art, when a choreographer processes an interface image, he may switch between a small core and a medium core when performing different tasks, which results in more cross-cores, a large scheduling load, and a low energy efficiency ratio; in the embodiment of the present application, the processing process of the interface image is bound to the medium core, that is, the above steps 104, 105, 107, and 108 are all executed by the medium core, thereby improving the response priority, with a small load and a high energy efficiency ratio.

[0057] An embodiment of the present application also provides an interface image processing device, including: a first module, used to execute the input event task and the drawing event task of the tth frame during the interface image processing process of the tth frame; a second module, used to execute the animation event task of the t+1th frame after executing the input event task and the drawing event task of the tth frame during the interface image processing process of the tth frame.

[0058] The interface image processing device can apply the interface image processing method in any of the above embodiments. The specific process and principle are the same as those in the above embodiments and will not be repeated here.

[0059] It should be understood that the above division of the interface image processing device is only a division of logical functions. In actual implementation, all or part of the modules can be integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, any one of the first module and the second module can be a separately established processing element, or it can be integrated in the interface image processing device, such as being integrated in a chip of the interface image processing device. In addition, it can also be stored in the memory of the interface image processing device in the form of a program, and called and executed by a processing element of the interface image processing device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0060] For example, the first module and the second module can be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a certain module above is implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0061] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the interface image processing method in any of the above embodiments.

[0062] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk), etc.

[0063] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or the second signal line may represent the existence of A alone, the existence of A and the second signal line at the same time, and the existence of the second signal line alone. Among them, A, the second signal line can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, a, at least one of the second signal line and the third signal line can represent: a, the second signal line, the third signal line, a-the second signal line, a-the third signal line, the second signal line-the third signal line, or a-the second signal line-the third signal line, wherein a, the second signal line, the third signal line can be single or multiple.

[0064] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An interface image processing method, characterized in that: include: During the interface image processing of the tth frame, the input event task and the drawing event task of the tth frame are executed; In the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, the animation event task of the t+1th frame is executed.

2. The interface image processing method according to claim 1, characterized in that: In the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: After receiving the vertical synchronization Vsync signal of the tth frame, determine whether the animation event task of the tth frame has been executed, and if so, execute the input event task and drawing event task of the tth frame; If the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and drawing event task of the tth frame, the animation event task of the t+1th frame is executed.

3. The interface image processing method according to claim 2, characterized in that: Also includes: After receiving the vertical synchronization Vsync signal of the tth frame, if it is determined that the animation event task of the tth frame has not been executed, the input event task, the animation event task and the drawing event task of the tth frame are executed.

4. The interface image processing method according to claim 2, characterized in that: Also includes: After executing the input event task and the drawing event task of the tth frame, if the time before receiving the Vsync signal of the t+1th frame is not enough to execute the animation event task of the t+1th frame, the rendering task of the tth frame is executed.

5. The interface image processing method according to claim 2, characterized in that: If the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: If the time before receiving the Vsync signal of the t+1th frame is sufficient to process the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, the interval time is delayed after executing the input event task and drawing event task of the tth frame, and the animation event task of the t+1th frame is executed.

6. The interface image processing method according to claim 2, characterized in that: After receiving the vertical synchronization Vsync signal of the t-th frame, determining whether the animation event task of the t-th frame has been executed, if so, the process of executing the input event task and the drawing event task of the t-th frame includes: After the display event receiver receives the vertical synchronization Vsync signal of the tth frame, it determines whether the animation event task of the tth frame has been executed. If so, the animation event task of the tth frame is removed from the task list, and an interface image processing request of the tth frame is sent to the choreographer; The choreographer executes the input event task and the drawing event task of the tth frame in the task list in response to the interface image processing request of the tth frame.

7. The interface image processing method according to claim 6, characterized in that: Before the display event receiver receives the vertical synchronization Vsync signal of the tth frame, it also includes: The view root layout sends a callback request to the choreographer in response to an input event; The choreographer adds callback information in response to the callback request and sends a Vsync request to the display event receiver.

8. The interface image processing method according to any one of claims 1 to 7, characterized in that: The input event task, the animation event task, the drawing event task and the rendering task are all executed based on the first type core of the processor, and the processor includes the first type core, the second type core and the third type core. The frequency of the first type core is lower than the frequency of the second type core, and the frequency of the first type core is higher than the frequency of the third type core.

9. An interface image processing device, characterized in that: include: The first module is used to execute the input event task and the drawing event task of the tth frame during the interface image processing of the tth frame; The second module is used to execute the animation event task of the t+1th frame after executing the input event task and the drawing event task of the tth frame during the interface image processing of the tth frame.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the electronic device executes the interface image processing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: It comprises a program or an instruction. When the program or the instruction is run on a computer, the interface image processing method according to any one of claims 1 to 8 is executed.

Images