Interface image processing method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the technical field of electronic devices, and provide an interface image processing method and apparatus, an electronic device, and a storage medium, capable of improving picture smoothness. The interface image processing method comprises: in the process of processing an interface image frame of frame t, executing an input event task and a drawing event task of frame t; and in the process of processing the interface image of frame t, after the input event task and the drawing event task of frame t are executed, executing an animation event task of frame t+1.

Description

Interface image processing method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 24, 2023, with application number 202310922653.4 and application name “Interface image processing method, device, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of electronic devices, and in particular to an interface image processing method, device, electronic device and storage medium. Background Art

[0003] When users perform user interface operations on electronic devices like mobile phones and tablets, such as clicking or swiping pages, the processor renders the interface to update the screen. If the processor is overloaded or its resources are occupied by other processes, this can cause delays in the interface's image processing, resulting in display freezes and choppy scrolling.

[0004] Summary of the Invention

[0005] The technical solution of the present application provides an interface image processing method, device, electronic device and storage medium, which can improve the flow of the screen.

[0006] In a first aspect, an interface image processing method is provided, comprising: in the interface image processing process of the tth frame, executing the input event task and the drawing event task of the tth frame; in the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, executing the animation event task of the t+1th frame.

[0007] In one possible embodiment, during the interface image processing of the tth frame, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: after receiving the vertical synchronization Vsync signal of the tth frame, determining whether the animation event task of the tth frame has been executed, and if so, executing the input event task and the drawing event task of the tth frame; if the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and the drawing event task of the tth frame, execute the animation event task of the t+1th frame.

[0008] In a possible implementation, the interface image processing method further includes: after receiving the vertical synchronization Vsync signal of the tth frame, if it is determined that the animation event task of the tth frame has not been executed, executing the input event task, animation event task and drawing event task of the tth frame.

[0009] In a possible embodiment, the interface image processing method also includes: after executing the input event task and the drawing event task of the tth frame, if the time before receiving the Vsync signal of the t+1th frame is not enough to execute the animation event task of the t+1th frame, then executing the rendering task of the tth frame.

[0010] In one possible implementation, if the time before the Vsync signal of the t+1th frame is received is sufficient to execute the animation event task of the t+1th frame, then before the Vsync signal of the tth frame is received, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: if the time before the Vsync signal of the t+1th frame is received is sufficient to process the animation event task of the t+1th frame, then before the Vsync signal of the tth frame is received, after executing the input event task and the drawing event task of the tth frame, the animation event task of the t+1th frame is executed after delaying the interval time.

[0011] In one possible embodiment, after receiving the vertical synchronization Vsync signal of the t-th frame, determining whether the animation event task of the t-th frame has been executed, and if so, executing the input event task and the drawing event task of the t-th frame includes: after the display event receiver receives the vertical synchronization Vsync signal of the t-th frame, determining whether the animation event task of the t-th frame has been executed, and if so, removing the animation event task of the t-th frame from the task list, and sending the interface image processing request of the t-th frame to the choreographer; the choreographer executes the input event task and the drawing event task of the t-th frame in the task list in response to the interface image processing request of the t-th frame.

[0012] In a possible implementation, before the display event receiver receives the vertical synchronization Vsync signal of the tth frame, it also includes: the view root layout sends a callback request to the choreographer in response to the input event; the choreographer adds callback information in response to the callback request and sends a Vsync request to the display event receiver.

[0013] In one possible implementation, input event tasks, animation event tasks, drawing event tasks, and rendering tasks are all executed based on the first-class core of the processor. The processor includes a first-class core, a second-class core, and a third-class core. The frequency of the first-class core is less than the frequency of the second-class core, and the frequency of the first-class core is greater than the frequency of the third-class core.

[0014] In the second aspect, an interface image processing device is provided, including: a first module, used to execute the input event task and drawing event task of the tth frame during the interface image processing process of the tth frame; a second module, used to execute the animation event task of the t+1th frame after executing the input event task and drawing event task of the tth frame during the interface image processing process of the tth frame.

[0015] In a third aspect, an electronic device is provided, comprising: a processor and a memory, wherein the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the electronic device executes the above-mentioned interface image processing method.

[0016] In a fourth aspect, a computer-readable storage medium is provided, comprising a program or an instruction. When the program or the instruction runs on a computer, the above-mentioned interface image processing method is executed.

[0017] The interface image processing method, device, electronic device and storage medium in the embodiments of the present application advance the animation event task of the t+1th frame to the interface image processing process of the tth frame, thereby reducing the probability of screen freeze caused by the interface image of the t+1th frame not being processed in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG1a is a timing diagram of an interface image processing process in the related art;

[0019] FIG1b is a timing diagram of another interface image processing process in the related art;

[0020] FIG2 is a structural block diagram of an electronic device according to an embodiment of the present application;

[0021] FIG3 is a block diagram of a software structure in an embodiment of the present application;

[0022] FIG4 is a timing diagram of an interface image processing process in an embodiment of the present application;

[0023] FIG5 is a flow chart of an interface image processing process in an embodiment of the present application;

[0024] FIG6 is a schematic diagram of the signaling relationship between multiple modules in a software structure in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0026] Before describing the embodiments of the present application, the related technologies and their problems are first described. As shown in Figures 1a and 1b, after receiving the vertical synchronization (Vsync) signal, the electronic device executes the input (Input) event task, the animation (Animation) event task and the drawing (draw) event task in sequence to realize interface rendering, and then sends the rendered interface to the display to realize the display of each frame. The circular F mark in Figures 1a and 1b indicates the time when the rendered interface is sent to the display. If the drawing event task of a certain frame is delayed, or even delayed to the next frame, then a conflict will occur between the rendering of the two adjacent frames, resulting in phenomena such as the screen display being stuck and the sliding operation being not smooth. For example, as shown in Figure 1b, assuming that the drawing event task 3 of the 3rd frame is delayed to after the display time of the 3rd frame, it will cause the display of the 3rd frame to be abnormal. In order to solve this problem, the embodiments of the present application are described below.

[0027] As shown in FIG2 , an embodiment of the present application provides an electronic device 100 , which may include a processor 110 , an internal memory 121 , a sensor module 180 , a display screen 194 , and the like.

[0028] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0029] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0030] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0031] The communication function of the electronic device 100 can be implemented in a wired or wireless manner.

[0032] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0033] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0034] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0035] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0036] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0037] Figure 3 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0038] The application layer includes a series of application packages. As shown in Figure 3, these packages may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short messaging. The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes predefined functions and may include managers and a view system.

[0039] The view system includes visual controls, such as those for displaying text or images. The view system is used to build applications. A display interface can consist of one or more views. The manager includes activities, a fundamental component of the Android operating system that provides users with interactive interfaces or views within applications. When one application calls another, the calling application invokes an activity within the other application, rather than the entire application. In this way, activities serve as entry points for interaction between applications and users.

[0040] (Android Runtime) includes core libraries and virtual machines.

[0041] The Android Runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one for Java-based functions and the other for the Android core library. The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files in the application layer and application framework layer as binary files. The virtual machine is responsible for performing functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0042] The system library can include multiple functional modules. For example: surface manager, media library, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0043] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver.

[0044] The following describes some concepts and application scenarios involved in the embodiments of this application.

[0045] The view system includes the view root layout (ViewRootlmpl), which is used to manage the root view (View) of the window, control the measurement, layout and drawing (draw) of the view, and the distribution interaction of all window events is executed or transmitted through the view root layout.

[0046] The view system also includes the Choreographer, a class in the operating system that coordinates the timing of input event tasks, animation event tasks, and draw event tasks. The Choreographer executes input event tasks through the input callback method (CALLBACK_INPUT). The input event task encapsulates input events from the application and distributes them to the Choreographer. An input event can be an interaction between the application and the user that the application monitors. After identifying an input event, the application performs preliminary processing, generates an input event task for the input event, and passes it to the Choreographer. The Choreographer executes animation event tasks through the animation callback method (CALLBACK_ANIMATION). The animation event task contains the data required by the application to calculate animations. The Choreographer executes measure event tasks, layout event tasks, and draw event tasks through the traversal callback method (CALLBACK_TRAVERSAL). The view root layout and the Choreographer are both modules in the view system.

[0047] The activity includes a display event receiver (DisplayEventReceiver), which is a module in the activity that monitors the vertical synchronization (Vsync) signal and controls the choreographer.

[0048] Taking a mobile phone as an example, the scenarios involved in the embodiments of the present application include the process in which a user touches an application interface or a system main interface (desktop), such as clicking or sliding, and the electronic device updates and displays an image based on the user interaction. When a user clicks or slides on the interface, an input event is triggered. The user interface (UI) thread executes input event tasks, animation event tasks, and drawing event tasks through the choreographer, and sends the data after the drawing is completed to the rendering thread. The rendering thread actually renders the data based on the data after the UI (User Interface) thread completes the drawing, and the rendered data is displayed.

[0049] As shown in Figure 4, an embodiment of the present application provides an interface image processing method, which includes: during the interface image processing process of the tth frame, executing the input event task and the drawing event task of the tth frame; during the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, executing the animation event task of the t+1th frame.

[0050] Specifically, the Arabic numerals following each time task in FIG4 indicate the frame number corresponding to the time task. For example, in the first frame, after executing the input event task and drawing event task of the first frame, before the second frame, if the animation event task of the second frame has been received, since the animation event task does not depend on the next frame and user operation, and the processing of the interface image of the first frame has been completed, the animation event task of the second frame can be executed before the second frame. In this way, when the second frame is reached, since the animation event task of the second frame has been executed in the first frame, after executing the input event task and drawing event task of the second frame, the rendering and display process of the interface image of the second frame can be continued. Compared with the related art, the rendering and display process of the interface image of the second frame can be executed faster. That is to say, assuming that a complex scene is encountered in the second frame, the load is large, and the resources are crowded with each other, in the related art, since the animation event task of the second frame needs to be executed in the second frame, it takes more time, which may result in not having enough time to render and display the interface image of the second frame, thereby causing the screen to freeze. However, in the interface image processing method of the embodiment of the present application, since the animation event task of the second frame has been executed in advance in the first frame, the time occupied by the interface image processing process of the second frame in the second frame is shorter, so the probability of not having enough time to process the interface image of the second frame is smaller, that is, the probability of screen freeze is lower.

[0051] The interface image processing method can be applied to the electronic device 100 described above.

[0052] The electronic devices involved in this application may be any product such as smart TVs, mobile phones, tablet computers, personal computers (PCs), personal digital assistants (PDAs), smart watches, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, vehicle-mounted devices, drone devices, smart cars, smart speakers, robots, smart glasses, etc.

[0053] The interface image processing method in the embodiment of the present application advances the execution of the animation event task of the t+1th frame to the interface image processing process of the tth frame, thereby reducing the probability of the interface image of the t+1th frame not being processed in time and causing the screen to freeze. In other words, in the embodiment of the present application, the idle time of the processor after the corresponding task of one frame is processed is used to advance the processing of animation event tasks that do not depend on the next frame and user operations. In this way, when a complex scene is encountered in the next frame, the probability of problems such as frame loss and freezes caused by excessive processor load and resource congestion and inability to process tasks can be reduced.

[0054] In a possible implementation, as shown in FIG5 , during the interface image processing of the t-th frame, after executing the input event task and the drawing event task of the t-th frame, the process of executing the animation event task of the t+1-th frame includes: after step 101, receiving the vertical synchronization Vsync signal of the t-th frame, executing step 102, determining whether the animation event task of the t-th frame has been executed, if so, executing step 103, removing the animation event task of the t-th frame from the task list, and executing step 104, executing the input event task and the drawing event task of the t-th frame; if not, that is, if it is determined that the animation event task of the t-th frame has not been executed, executing step 105, executing the input event task, the animation event task and the drawing event task of the t-th frame. Control event task; After the process of step 104 and step 105, execute step 106 to determine whether the time before receiving the Vsync signal of the t+1th frame is sufficient to process the animation event task of the t+1th frame; if so, that is, if the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, execute step 107 to execute the animation event task of the t+1th frame and step 108 to execute the rendering task of the tth frame; if not, that is, if the time before receiving the Vsync signal of the t+1th frame is insufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, execute step 108. In step 106, determining whether the time is sufficient is not to determine whether the time before receiving the Vsync signal of the t+1 frame is sufficient to execute the animation event task of the t+1 frame. Instead, it is to determine whether the idle time of the processor before receiving the Vsync signal of the t+1 frame, other than executing necessary tasks (such as rendering tasks), is sufficient to execute the animation event task of the t+1 frame. If it is determined in step 106 that the time is sufficient, the animation event task of the t+1 frame is executed in advance in the t frame, and then the rendering task of the t frame is executed. After rendering, it is sent for display, that is, step 109 is executed to display the interface image of the t frame, and then the t+1 frame is entered. Since the animation event task of the t+1 frame has been executed in advance in the t frame, this task does not need to be executed in the t+1 frame, and the t+1 frame has more events to execute other tasks. If it is determined in step 106 that the time is insufficient, the animation event task of the t+1th frame is not executed in the tth frame, but other necessary tasks are directly executed, including steps 108 and 109. In this way, it can be ensured that the task of displaying the interface image of the tth frame has more time to execute.

[0055] In one possible implementation, if the time before receiving the Vsync signal of the t+1 frame is sufficient to execute the animation event task of the t+1 frame, then before receiving the Vsync signal of the t frame, after executing the input event task and the drawing event task of the t frame, the process of executing the animation event task of the t+1 frame includes: if the time before receiving the Vsync signal of the t+1 frame is sufficient to process the animation event task of the t+1 frame, then before receiving the Vsync signal of the t frame, after executing the input event task and the drawing event task of the t frame, delay the interval time, execute the animation event task of the t+1 frame, and then execute the rendering task of the t frame and display the interface image of the t frame, and the interval time is, for example, a few ms. The processor's idle time between executing the input event task and the drawing event task for frame t and before frame t+1 is fixed. For example, if the processor idle time is a milliseconds, the time required to process the animation event task for frame t+1 is b milliseconds, and the preparation time required to process the animation event task for frame t+1 is c milliseconds, then any time between c and abc can be used as the interval time, as long as a ≥ b + c. The preparation time required to process the animation event task for frame t+1 can be, for example, the time it takes to release the process lock. By setting the interval time, exceptions caused by the consecutive execution of the tasks for frame t and frame t+1 can be prevented.

[0056] In a possible implementation, as shown in Figure 6, the following describes an embodiment of the present application based on the signaling relationship between multiple modules in the software structure. First, the view root layout sends a callback request to the choreographer in response to an input event. The input event is, for example, an operation such as a user sliding or clicking on the interface. The callback request is, for example, a postCallback(CALLBACK_XXX) method. The choreographer adds callback information in response to the callback request and sends a vertical synchronization Vsync request to the display event receiver. Adding callback information can be achieved, for example, by using the addCallbacklocked method. The Vsync request is, for example, the scheduleVsync method. Vsync is provided by the operating system and is used to coordinate and synchronize the signals of the choreographer and other module operations. The display event receiver sends a vertical synchronization (Vsync) request in response to the choreographer's Vsync request, such as the nativeSchedeleVsync method; the display event receiver then receives the vertical synchronization signal, such as the dispatchVsync method; after the display event receiver receives the vertical synchronization Vsync signal of the t-th frame, it receives data through the onVsync method and determines whether the animation event task of the t-th frame has been executed. If so, the animation event task of the t-th frame is removed from the task list through the remove(postAnim) method, and the interface image processing request of the t-th frame is sent to the choreographer through the doFrame method; the choreographer executes the input event task and drawing event task of the t-th frame in the task list in response to the interface image processing request of the t-th frame, such as the input event task is executed through the doCallback(CALLBACK_INPUT) method, and the doCallback(CALLBACK_ANIMATION) method can also be executed, but since the animation event task of the t-th frame has been removed from the task list, the animation event task of the t-th frame will not be actually executed. The drawing event task is then executed through the doCallback(CALLBACK_TRAVERSAL) method. If the choreographer has enough time to process the animation event task for frame t+1 before receiving the Vsync signal for frame t+1, the choreographer registers the animation event task for frame t+1 via the postAnim method after a delay of Tms and executes the animation event task for frame t+1 via the doCallback(CALLBACK_ANIMATION) method. The rendering task for frame t is then executed and the interface image for frame t is displayed. Then, in frame t+1, the view root layout responds to the input event and sends a callback request to the choreographer, restarting the process to process the interface image for frame t+1. The specific principles are the same as those for frame t and will not be repeated here.

[0057] During each frame, the choreographer executes the doCallback(CALLBACK_INPUT) method, the doCallback(CALLBACK_ANIMATION) method, and the doCallback(CALLBACK_TRAVERSAL) method in response to the interface image processing request of the current frame. The above describes the situation where the current frame animation event task is removed from the task list. If the current frame animation event task is not removed from the task list, that is, the current frame animation event task is not executed in advance, then when the doCallback(CALLBACK_ANIMATION) method is executed, the execution of the current frame animation event task will be triggered. In addition, if the choreographer does not have enough time to process the next frame animation event task before receiving the next frame's Vsync signal, the next frame animation event task will not be executed in advance, but the current frame rendering task and display and other necessary tasks of the current frame will be executed normally to avoid resource congestion and insufficient time to process in complex scenes.

[0058] In one possible implementation, input event tasks, animation event tasks, drawing event tasks, and rendering tasks are all executed based on the first type of core of the processor. The processor includes a first type of core, a second type of core, and a third type of core. The frequency of the first type of core is lower than the frequency of the second type of core, and the frequency of the first type of core is higher than the frequency of the third type of core. The first type of core can be called a medium core, the second type of core can be called a large core, and the third type of core can be called a small core. The higher the frequency of the core, the greater the power consumption, and the lower the frequency, the lower the power consumption. In the prior art, when processing interface images, choreographers may switch between small cores and medium cores when performing different tasks. This results in more cross-cores, a large scheduling load, and low energy efficiency. In the embodiment of the present application, the interface image processing process is bound to the medium core, that is, the above steps 104, 105, 107, and 108 are all executed by the medium core, thereby improving response priority, reducing load, and improving energy efficiency.

[0059] An embodiment of the present application also provides an interface image processing device, including: a first module, used to execute the input event task and drawing event task of the tth frame during the interface image processing process of the tth frame; a second module, used to execute the animation event task of the t+1th frame after executing the input event task and drawing event task of the tth frame during the interface image processing process of the tth frame.

[0060] The interface image processing device can apply the interface image processing method in any of the above embodiments. The specific process and principle are the same as those in the above embodiments and will not be repeated here.

[0061] It should be understood that the above division of the interface image processing device is merely a division of logical functions. In actual implementation, all or part of the modules can be integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in the form of software called through a processing element; or entirely in the form of hardware; or some modules can be implemented in the form of software called through a processing element, and some modules can be implemented in the form of hardware. For example, either the first module or the second module can be a separate processing element, or integrated into the interface image processing device, such as being implemented in a chip of the interface image processing device. In addition, they can be stored in the memory of the interface image processing device in the form of a program, and called and executed by a processing element of the interface image processing device. The implementation of other modules is similar. In addition, these modules can be integrated together in whole or in part, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by an integrated logic circuit in the hardware of the processor element or by instructions in the form of software.

[0062] For example, the first module and the second module may be one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), one or more digital singnal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when one of the above modules is implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0063] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the interface image processing method in any of the above embodiments.

[0064] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0065] In the embodiment of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or the second signal line can represent the existence of A alone, the existence of A and the second signal line at the same time, and the existence of the second signal line alone. Among them, A, the second signal line can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, a, at least one of the second signal line and the third signal line can represent: a, the second signal line, the third signal line, a-the second signal line, a-the third signal line, the second signal line-the third signal line, or a-the second signal line-the third signal line, wherein a, the second signal line, the third signal line can be single or multiple.

[0066] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. An interface image processing method, characterized in that: include: During the interface image processing of the tth frame, the input event task and the drawing event task of the tth frame are executed; In the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, the animation event task of the t+1th frame is executed.

2. The interface image processing method according to claim 1, characterized in that: In the interface image processing process of the tth frame, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: After receiving the vertical synchronization Vsync signal of the tth frame, determine whether the animation event task of the tth frame has been executed, and if so, execute the input event task and drawing event task of the tth frame; If the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and drawing event task of the tth frame, the animation event task of the t+1th frame is executed.

3. The interface image processing method according to claim 2, characterized in that: Also includes: After receiving the vertical synchronization Vsync signal of the tth frame, if it is determined that the animation event task of the tth frame has not been executed, the input event task, the animation event task and the drawing event task of the tth frame are executed.

4. The interface image processing method according to claim 2, characterized in that: Also includes: After executing the input event task and the drawing event task of the tth frame, if the time before receiving the Vsync signal of the t+1th frame is not enough to execute the animation event task of the t+1th frame, the rendering task of the tth frame is executed.

5. The interface image processing method according to claim 2, characterized in that: If the time before receiving the Vsync signal of the t+1th frame is sufficient to execute the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, after executing the input event task and the drawing event task of the tth frame, the process of executing the animation event task of the t+1th frame includes: If the time before receiving the Vsync signal of the t+1th frame is sufficient to process the animation event task of the t+1th frame, then before receiving the Vsync signal of the tth frame, the interval time is delayed after executing the input event task and drawing event task of the tth frame, and the animation event task of the t+1th frame is executed.

6. The interface image processing method according to claim 2, characterized in that: After receiving the vertical synchronization Vsync signal of the t-th frame, determining whether the animation event task of the t-th frame has been executed, if so, the process of executing the input event task and the drawing event task of the t-th frame includes: After the display event receiver receives the vertical synchronization Vsync signal of the tth frame, it determines whether the animation event task of the tth frame has been executed. If so, the animation event task of the tth frame is removed from the task list, and an interface image processing request of the tth frame is sent to the choreographer; The choreographer executes the input event task and the drawing event task of the tth frame in the task list in response to the interface image processing request of the tth frame.

7. The interface image processing method according to claim 6, characterized in that: Before the display event receiver receives the vertical synchronization Vsync signal of the tth frame, it also includes: The view root layout sends a callback request to the choreographer in response to an input event; The choreographer adds callback information in response to the callback request and sends a Vsync request to the display event receiver.

8. The interface image processing method according to any one of claims 1 to 7, characterized in that: The input event task, the animation event task, the drawing event task and the rendering task are all executed based on the first type core of the processor, and the processor includes the first type core, the second type core and the third type core. The frequency of the first type core is lower than the frequency of the second type core, and the frequency of the first type core is higher than the frequency of the third type core.

9. An interface image processing device, characterized in that: include: The first module is used to execute the input event task and the drawing event task of the tth frame during the interface image processing of the tth frame; The second module is used to execute the animation event task of the t+1th frame after executing the input event task and the drawing event task of the tth frame during the interface image processing of the tth frame.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the electronic device executes the interface image processing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: It comprises a program or an instruction. When the program or the instruction is run on a computer, the interface image processing method according to any one of claims 1 to 8 is executed.

Images