Animation effect display method and electronic device
By sinking the animation effect computing power to the Surface Flinger process, the problem of poor motion effect display fluency when electronic devices run multiple applications is solved, achieving smoother motion effect display and improving user experience.
Patent Information
- Application Number
- PCT/CN2024/096102
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-30
- Filing Date
- 2024-05-29
- Publication Date
- 2025-01-02
AI Technical Summary
When existing electronic devices run multiple applications, they are prone to problems such as stuttering, frame dropping, and jittering, resulting in poor fluency in the display of animation and affecting the user experience.
Sink the animation effect calculation capability to the Surface Flinger process, reduce the number of cross-process communication times, avoid System Server or application processes from executing the animation effect calculation process, and complete animation effect calculation and image synthesis through the Surface Flinger process.
It improves the smoothness of the animation effect display of electronic devices, avoids the problems of stuttering, frame loss, and jittering of the animation effect, and improves the user experience.
Smart Images

Figure CN2024096102_02012025_PF_FP_ABST
Abstract
Description
A dynamic effect display method and electronic device
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on June 30, 2023, with application number 202310798033.4 and application name “A Motion Effect Display Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field
[0003] The present application relates to the technical field of electronic devices, and in particular to a motion effect display method and electronic device. Background Art
[0004] To enrich the screen and attract users' attention, electronic devices can display animations during operation. For example, electronic devices can display window animations when users open or close windows, minimize or resize windows, or operate floating windows. For another example, when an electronic device jumps from one page to another, or from one application's user interface (UI) to another application's user interface, it can display transition animations.
[0005] To enrich and facilitate users' lives, software developers have developed a variety of applications (APPs). Furthermore, as applications become increasingly powerful, their size also increases. Consequently, when electronic devices run multiple applications, overload scenarios often occur, which can further lead to animation lag, frame drops, and jitter.
[0006] Summary of the Invention
[0007] The present application provides a motion effect display method and an electronic device, which are used to ensure the smoothness of motion effect display of the electronic device, avoid problems such as motion effect freeze, frame loss, and jitter, and improve user experience.
[0008] In a first aspect, an embodiment of the present application provides a method for displaying an animated effect. The method can be applied to an electronic device and includes the following steps:
[0009] A first process receives motion effect information from a second process, where the motion effect information is used to indicate at least one motion effect operation in the motion effect; then, the first process displays the motion effect based on the motion effect information, specifically including: the first process calculates display data of the at least one motion effect operation at different times based on the motion effect information; and calculates image frame data corresponding to different times based on the display data of the at least one motion effect operation at different times; the first process synthesizes and displays image frames corresponding to different times based on the image frame data corresponding to different times to display the motion effect; wherein, the first process is a process that runs the image plane engine Surface Flinger of the electronic device, and the second process is a process that runs the system service System Server or application of the electronic device.
[0010] Through this method, the motion effect calculation capability of the electronic device is transferred to the Surface Flinger process, which can significantly reduce the number of cross-process communications during the motion effect display process. For example, the System Server process or the application process can complete the motion effect display by sending the motion effect information to the Surface Flinger process through one cross-process communication. Since the Surface Flinger process is a resident process at the bottom layer of the electronic device, compared with the traditional solution, this solution can avoid the System Server process or the application process from executing the motion effect calculation process. The Surface Flinger process completes the calculation process in the motion effect, which can reduce the system load of the electronic device and ensure the smoothness of the motion effect display of the electronic device, avoiding the problems of motion effect freeze, frame loss, and jitter, thereby improving the user experience.
[0011] Optionally, the second process may send the motion effect information to the first process via an inter-process communication method (eg, an IPC method, a binder method, etc.).
[0012] Optionally, during the process of displaying the animation effect by the first process, after each time the first process calculates the display data of at least one animation effect operation at a moment, it calculates the image frame data corresponding to that moment based on the display data, and then synthesizes and displays the image frame corresponding to that moment. In this way, the first process can sequentially calculate the image frame data corresponding to different moments, and ultimately sequentially display the image frames in the animation effect.
[0013] In one possible design, each time the first process receives an image refresh signal, it calculates display data for the at least one operation at the current moment based on the motion effect information, so that the first process can subsequently calculate image frame data corresponding to the current moment based on the display data for the at least one operation at the current moment, and then synthesize and display the image frame corresponding to the current moment. Exemplarily, the image refresh signal can be a Vsync signal.
[0014] Through this design, the Surface Flinger process of the electronic device can complete the calculation process in the process of displaying the dynamic effect, avoiding the System Server process or application process to perform the calculation process, thereby reducing the system load of the electronic device and avoiding the process of transmitting the image frame data in the dynamic effect across processes multiple times.
[0015] In one possible design, the motion effect information includes relevant data of the at least one motion effect operation; the relevant data of any motion effect operation includes at least one of the following: the type of the motion effect operation, the start time of the motion effect operation, the duration of the motion effect operation, the offset time of the motion effect operation, the differentiator information of the motion effect operation, or the callback information of the motion effect operation; based on this, the first process can calculate the display data of the at least one motion effect operation at the current moment according to the relevant data of the at least one motion effect operation contained in the motion effect information each time after receiving the image refresh signal.
[0016] Optionally, the second process may serialize the motion effect information and then transmit the serialized motion effect information to the first process. Based on this, the first process may first deserialize the serialized motion effect information before displaying the motion effect to obtain the original motion effect information.
[0017] In one possible design, while displaying the animation, the first process can synthesize and display the corresponding image frames at different times on a layer corresponding to the system server or application based on the image frame data at different times. This design allows the animation to be bound to the layer corresponding to the system server or application, ensuring accurate display of the animation.
[0018] In one possible design, when the first process receives a stop animation instruction from the second process, the first process stops displaying the animation, that is, the first process stops calculating the display data of the at least one animation operation at different times. Optionally, the first process can delete the animation information after stopping displaying the animation.
[0019] In one possible design, when the first process receives an animation interrupt instruction from the second process, the first process interrupts displaying the animation, that is, the first process interrupts calculating the display data of the at least one animation operation at different times; when the first process receives an animation restart instruction from the second process, the first process continues to display the animation, that is, the first process continues to calculate the display data of the at least one animation operation at different times.
[0020] Through the above design, the second process can send instructions through cross-process communication to achieve state management of the animation effect.
[0021] In one possible design, while the second process displays the motion effect, the second process does not calculate display data of the at least one motion effect operation based on the motion effect information, and the second process does not send image frame data to the first process.
[0022] This design can avoid the second process from executing the motion effect calculation process during the motion effect display process, and can also significantly reduce the number of cross-process communications during the motion effect display process.
[0023] In a second aspect, an embodiment of the present application provides an electronic device comprising a unit for executing each step of the method provided in the first aspect above. Optionally, the electronic device comprises a display screen and a processing unit; wherein the display screen is configured to display a user interface; and the processing unit is configured to execute the method provided in the first aspect. Optionally, the electronic device may further comprise a communication unit. The communication unit is configured to receive and transmit data.
[0024] For example, the module or unit running the Surface Flinger process in an electronic device includes: a Surface Flinger main module and a motion effect management module. The motion effect management module is used to: receive motion effect information from a System Server process or an application process, wherein the motion effect information is used to indicate at least one motion effect operation in the motion effect; calculate the display data of the at least one motion effect operation at different times based on the motion effect information; and calculate the image frame data corresponding to the different times based on the display data of the at least one motion effect operation at different times;
[0025] The Surface Flinger main module is used to synthesize and display image frames corresponding to different moments according to image frame data corresponding to different moments, so as to display the motion effect.
[0026] In a third aspect, an embodiment of the present application provides an electronic device comprising a display screen, at least one processor, and a memory; the at least one processor is coupled to the memory, and the at least one processor is configured to read a computer program stored in the memory to execute the method provided in the first aspect. The computer program code comprises computer instructions.
[0027] In a fourth aspect, an embodiment of the present application provides a chip, which includes a processor and a memory, wherein the processor is coupled to the memory, and the processor is used to call a computer program stored in the memory during runtime to implement the method provided in the first aspect above.
[0028] In a fifth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. When the computer program runs on an electronic device, the computer executes the method provided in the first aspect above.
[0029] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a computer program; when the computer program is run on a computer, the computer is caused to execute the method provided in the first aspect.
[0030] The technical effects that can be achieved in any of the second to sixth aspects mentioned above can refer to the description of the technical effects that can be achieved in any possible design in the first aspect mentioned above, and the repetitions will not be discussed. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] FIG1 is a schematic diagram of a traditional motion effect display solution;
[0032] FIG2 is a structural diagram of an electronic device provided in an embodiment of the present application;
[0033] FIG3 is a schematic diagram of a layered architecture of a display system provided in an embodiment of the present application;
[0034] FIG4 is a schematic diagram of a layered architecture of another display system provided in an embodiment of the present application;
[0035] FIG5 is a structural diagram of serialized motion effect information provided by an embodiment of the present application;
[0036] FIG6 is a flow chart of a method for displaying dynamic effects provided by an embodiment of the present application;
[0037] FIG7A is a flowchart of an example of a dynamic effect display provided by an embodiment of the present application;
[0038] FIG7B is a flowchart of an example of a dynamic effect display provided by an embodiment of the present application;
[0039] FIG7C is a schematic diagram of a transition effect provided in an embodiment of the present application;
[0040] FIG8 is a schematic diagram comparing a traditional dynamic effect display solution and a dynamic effect display method provided in an embodiment of the present application;
[0041] FIG9 is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] This application provides a method and electronic device for displaying motion effects, which are used to ensure smooth motion effect display in electronic devices, avoid problems such as motion freeze, frame loss, and jitter, and improve user experience. The method and electronic device are based on the same technical concept. Since the method and electronic device solve similar problems, the implementation of the electronic device and the method can refer to each other, and the repeated parts will not be repeated.
[0043] Some of the terms used in this application are explained below to facilitate understanding by those skilled in the art.
[0044] 1) Electronic devices are devices or apparatuses with display screens that can process and display images. Generally, electronic devices have display screens.
[0045] For example, the electronic devices in this application may be personal computers (PCs), laptops, computers, netbooks, tablet computers, in-vehicle computers, smart phones, smart wearable devices (such as smart watches, smart bracelets, smart glasses, etc.), etc. Of course, the electronic devices may also be various portable devices, in-vehicle terminal devices, personal digital assistants (PDAs), smart home devices (such as smart TVs, etc.), and other smart devices. This application does not limit the specific form of the electronic devices.
[0046] The electronic device can realize its functions through the operating system it carries and provide services to the user. For example, the electronic device can be equipped with, but not limited to, Or other operating systems.
[0047] 2) Animation: This refers to the animated process by which the user interface displayed on an electronic device's screen dynamically changes. To enhance the user experience, enrich the screen, and achieve better visuals, electronic devices can display animations during operation. These animations include, but are not limited to, window animations and transition effects. Below is a brief description of these two types of animations.
[0048] Window animation can be the animation generated when the user operates the window of the electronic device. Optionally, when the user opens, closes, or minimizes the window, or when the user adjusts the window size, or when the user operates the floating window (or small window, or window in split screen), the electronic device can display the corresponding window animation. For example, when the user closes the floating window displayed on the smartphone, the floating window can fly out and disappear. For another example, when the user restores the floating window displayed on the smartphone to a normal window, the floating window can slowly grow until it becomes the size of a normal window.
[0049] Transition effects are the effects produced when the user operates the user interface of an electronic device. Optionally, when an electronic device jumps from one page in an application to another, or jumps from one application to another, the electronic device can display a transition effect. For example, after the user opens the settings application on a smartphone and selects a mobile network from the homepage, the smartphone jumps from the homepage of the settings application (i.e., the first-level page) to the second-level page of the mobile network. For another example, the user opens the album application on the smartphone, selects a photo, clicks the share button and selects application, after which the smartphone switches from the user interface of the photo album application to The app's sharing UI.
[0050] 3) Process and thread.
[0051] A process is a program with certain independent functions running on a certain data set. A process is an independent unit for resource allocation and scheduling in the operating system. For example, when each APP is running, the electronic device will start the process corresponding to the APP, such as process, In addition, electronic devices also run corresponding processes for the services provided by the operating system, such as the process running the system service (System Server) (hereinafter referred to as the System Server process) and the process running the image plane engine (Surface Flinger) (hereinafter referred to as the Surface Flinger process).
[0052] A thread is an entity within a process and the basic unit of scheduling and dispatching by the central processing unit (CPU). It is smaller than a process and can run independently. A thread possesses virtually no system resources, possessing only those necessary for operation (such as the program counter, registers, and stack) and an independent execution sequence. However, it can share all resources owned by the process with other threads in the same process and can also share data with other threads in the same process.
[0053] The primary difference between processes and threads lies in their distinct ways of managing operating system resources. Processes have independent address spaces, and a process crash won't affect other processes in protected mode. Threads, on the other hand, are simply different execution paths within a process. They have their own stack and local variables, but lack independent address spaces. Therefore, the death of a thread means the entire process has crashed.
[0054] In short, a program has at least one process, and a process can contain one or more threads. Threads are smaller than processes. A process has its own independent memory unit during execution, while multiple threads in the same process share memory.
[0055] It should be noted that different processes can communicate with each other, i.e., cross-process communication. Currently, cross-process communication methods include, but are not limited to, inter-process communication (IPC), binder, and the like.
[0056] 4) Surface Flinger, an independent service provided by the electronic device's operating system, primarily implements functions such as surface creation, control, and management, ultimately synthesizing the final image to be displayed on the display, enabling the electronic device to display the user interface in real time. As a resident service at the bottom layer of the electronic device, Surface Flinger must remain operational throughout the device's operation. Therefore, the operating system of the electronic device must prioritize resources allocated to it (or the Surface Flinger process) to ensure that Surface Flinger is always ready for image synthesis.
[0057] The following is a brief explanation of the relevant concepts involved in Surface Flinger.
[0058] Surfaces generally correspond to windows. While windows focus on hierarchy and layout, they are defined from a designer's perspective. Surfaces, on the other hand, are triggered from an implementation perspective and are the class engineers care about and consider. It's important to note that some surfaces are not associated with windows but are instead created directly by the program. For example, Surface View is used to display video content from hardware outputs.
[0059] A layer is the basic unit of operation for Surface Flinger's image synthesis. A layer is created within Surface Flinger when an application or system service requests the creation of a surface. Therefore, one surface corresponds to one layer.
[0060] In summary, an application or system server can create at least one surface. Each surface corresponds to a layer on the Surface Flinger side.
[0061] 5) Applications are installed in electronic devices and can provide services to users after running. Applications can also be called application programs or application software.
[0062] The application may be developed by the manufacturer of the electronic device, or by the supplier of the operating system of the electronic device, or by a third-party application manufacturer, and this application does not limit this.
[0063] For example, applications may include: call applications, camera applications, text messaging applications, chat applications, video applications, music applications, electronic map applications, social applications, etc.
[0064] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple.
[0065] In the embodiments of the present application, descriptions such as "when...", "in the case of...", "if" and "if" all mean that the electronic device will perform corresponding processing under certain objective circumstances. It does not limit the time, nor does it require the electronic device to perform judgment actions when implemented, nor does it mean that there are other limitations.
[0066] It should be noted that in this application, "used for indication" can include being used for direct indication and being used for indirect indication. When describing that a certain information is used to indicate A, it can include that the information directly indicates A or indirectly indicates A, but it does not mean that the information must carry A. Taking the first information used to indicate the first content as an example, the first information can include the first content, and can also include a part of the first content or an identifier or index of the first content, etc., and can also include an algorithm or calculation parameters for determining the first content. This application does not limit the method of "indication".
[0067] In this application, "including / comprising A" may be equivalent to "including / comprising A information". A information is used to indicate A.
[0068] In addition, it should be understood that, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.
[0069] Currently, in traditional motion effect display solutions, as shown in Figure 1, the System Server process or application process in the electronic device is responsible for defining the motion effect and sequentially calculating the image frame data in the motion effect that needs to be displayed. Afterwards, the System Server process or application process sends the image frame data obtained each time through cross-process communication to the Surface Flinger process. The Surface Flinger process synthesizes the image based on the received image frame data and displays the image on the display screen. The System Server process or application process in the electronic device, as well as the Surface Flinger process, cyclically execute the above-mentioned process of calculating image frame data and synthesizing and displaying the image, ultimately realizing the motion effect display process of the electronic device.
[0070] Through the description of the above-mentioned traditional motion effect display scheme, it can be seen that the calculation process of the image frame data and the synthesis display process in the motion effect display process are separated and completed by different processes. Therefore, the number of cross-process communications in this process is frequent. Once the process that needs to display the motion effect (such as the System Server process or the application process) or a thread involved in the motion effect display in the process is not scheduled in time (such as the electronic device overload scenario), it may cause unstable processing delay. Furthermore, there are also unstable factors in the delay of cross-process communication. In summary, the motion effects displayed by electronic devices will have problems such as freezes, frame drops, and jitters, resulting in poor smoothness of the motion effect display of electronic devices.
[0071] Furthermore, as applications become more powerful, their size will also increase significantly. Electronic devices may experience overloads or overloads when running multiple applications simultaneously, which will increase the probability of the above-mentioned problems occurring in electronic devices. Therefore, how to ensure the smoothness of the dynamic display of electronic devices and avoid problems such as dynamic freezes, frame drops, and jitter is a problem that technicians in this field need to consider.
[0072] Based on this, the present application provides a method for displaying motion effects and an electronic device. In this method, the process running the System Server or application (referred to as the System Server process or application process for short) defines the motion effect information and sends it to the process running the Surface Flinger (referred to as the Surface Flinger process for short). The Surface Flinger process displays the motion effect based on the received motion effect information. Through this solution, it can be seen that the motion effect calculation capability of the electronic device is sunk to the Surface Flinger process, which can significantly reduce the number of cross-process communications during the motion effect display process. Since the Surface Flinger process is a resident process at the bottom layer of the electronic device, compared with the traditional solution, this solution can avoid the System Server process or the application process from executing the motion effect calculation process. The Surface Flinger process completes the calculation process in the motion effect, which can reduce the system load of the electronic device, and can ensure the smoothness of the motion effect display of the electronic device, avoid the problems of motion effect freeze, frame loss, and jitter, and improve the user experience.
[0073] The solution provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0074] The dynamic effect display method provided in the embodiment of the present application is applicable to the electronic device shown in Figure 2. Referring to Figure 2, the structure of the electronic device provided in the embodiment of the present application is introduced below.
[0075] As shown in Figure 2, the electronic device 200 may include: a processor 210, an external memory interface 220, an internal memory 221, a USB interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a SIM card interface 295, etc.
[0076] It should be understood that the electronic device 200 shown in FIG2 is merely an example and does not limit the electronic device, and the electronic device may have more or fewer components than shown in FIG2, may combine two or more components, or may have a different component configuration. The various components shown in FIG2 may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.
[0077] The processor 210 may include one or more processing units, for example: the processor 210 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. Among them, the controller can be the nerve center and command center of the electronic device 200. The controller can generate an operation control signal based on the instruction opcode and the timing signal to complete the control of instruction fetching and execution.
[0078] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or is reusing. If the processor 210 needs to use the instruction or data again, it can directly access it from the memory. By providing this memory, the number of times the processor 210 accesses data from the internal memory 221 can be reduced, reducing the processor 210's waiting time, thereby improving system efficiency.
[0079] The dynamic effect display method provided in the embodiment of the present application can be controlled by the processor 210 or completed by calling other components, such as calling the software program of the embodiment of the present application stored in the internal memory 221 to control cross-process communication and control the display screen to display the user interface.
[0080] Display screen 294 is used to display images, videos, etc. Display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 200 may include one or N display screens 294, where N is a positive integer greater than 1. Display screen 294 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces (GUIs). For example, display screen 294 can display windows, photos, videos, web pages, or files. Furthermore, it will be appreciated that in some embodiments, the status bar may also include a Bluetooth icon, a Wi-Fi icon, an external device icon, etc. In some embodiments, when the processor 210 detects a touch operation of a user's finger (or stylus, etc.) on an application icon, in response to the touch operation, the processor 210 opens a user interface corresponding to the application icon and displays the user interface on the display screen 294. In some embodiments, when the processor 210 detects a click operation of an external input device (such as a mouse, keyboard, etc.) of the electronic device 200 on an application icon, in response to the click operation, the processor 210 performs a corresponding control operation on the application.
[0081] In the embodiment of the present application, the display screen 294 can be an integrated flexible display screen, or a spliced display screen consisting of two rigid screens and a flexible screen located between the two rigid screens. When the processor 210 executes the motion effect display method provided in the embodiment of the present application, the processor 210 can control the display screen 294 to display the motion effect.
[0082] The camera 293 is used to capture still images or videos. Optionally, the electronic device 200 may include 1 to N cameras 293. For example, the camera 293 may include a front camera or a rear camera, or one camera 293 may serve as both a front camera and a rear camera. Typically, the camera 293 may include a photosensitive element such as a lens group and an image sensor. The lens group includes a plurality of lenses (convex lenses or concave lenses) for collecting light signals reflected by the object to be photographed and transmitting the collected light signals to the image sensor. The image sensor generates an original image of the object to be photographed based on the light signal.
[0083] The internal memory 221 can be used to store computer executable program codes, which include instructions. The processor 210 executes various functional applications and data processing of the electronic device 200 by running the instructions stored in the internal memory 221. The internal memory 221 may include a program storage area and a data storage area. The program storage area may store an operating system, program codes of at least one application, etc. The operating system may include, but is not limited to, The data storage area can store data created during the use of the electronic device 200 .
[0084] The internal memory 221 may also store one or more computer programs for executing the motion effect display method provided in the embodiments of the present application. The one or more computer programs are stored in the internal memory 221 and configured to be executed by the one or more processors 210. The one or more computer programs include instructions, which can be used to execute the various steps in the following embodiments.
[0085] In addition, the internal memory 221 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
[0086] The sensor module 280 may include a fingerprint sensor, a touch sensor, a pressure sensor, a magnetic sensor, an ambient light sensor, an air pressure sensor, a bone conduction sensor, and the like.
[0087] A touch sensor, also known as a "touch panel," can be provided on the display screen 294. The touch sensor is configured to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 294. In other embodiments, the touch sensor can also be provided on the surface of the electronic device 200, at a location different from that of the display screen 294.
[0088] The wireless communication function of the electronic device 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor and the baseband processor.
[0089] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.
[0090] The mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 200. The mobile communication module 250 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the same device as at least some of the modules of the processor 210.
[0091] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 270A, the receiver 270B, etc.) or displays an image or video through the display screen 294. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 210 and be set in the same device as the mobile communication module 250 or other functional modules.
[0092] The wireless communication module 260 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 200. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210. The wireless communication module 260 can also receive the signal to be sent from the processor 210, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0093] In addition, the electronic device 200 can implement audio functions through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone jack 270D, and the application processor. For example, music playback, recording, etc. The electronic device 200 can receive input from the key 290 and generate key signal input related to the user settings and function control of the electronic device 200. The electronic device 200 can use the motor 291 to generate a vibration prompt. The indicator 292 in the electronic device 200 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 295 in the electronic device 200 is used to connect the SIM card. The SIM card can be connected to and separated from the electronic device 200 by inserting it into the SIM card interface 295 or pulling it out from the SIM card interface 295.
[0094] The display system of the electronic device provided in the embodiments of the present application can adopt a layered architecture. Specifically, the display system of the electronic device can be divided into multiple layers, each with a clear role and division of labor. The layers communicate with each other via corresponding software interfaces. The following, in conjunction with Figure 3, illustrates an exemplary display architecture of an electronic device.
[0095] As shown in FIG3 , the display system of an electronic device can be divided into four layers, namely, the application layer, the system server layer, the surface flinger layer, and the hardware abstraction layer (HAL) from top to bottom.
[0096] The application layer is the top layer of the operating system, including native applications of the operating system and third-party applications, such as camera, desktop, system user interface (systemUI) gallery, settings, Music, videos, messages and more.
[0097] The System Server layer is a crucial component of an electronic device's operating system. The System Server process is a crucial process within the operating system, hosting the system's primary services, including the Window Manager Service (WMS), Activity Manager Service (AMS), and Package Manager Service (PackageManagerServer). The System Server is responsible for initiating various system services.
[0098] The Surface Flinger layer provides Surface Flinger services. Surface Flinger is an independent service provided by the operating system of an electronic device. It is a resident service at the bottom layer of the electronic device. During the operation of the electronic device, the Surface Flinger process needs to run continuously. The operating system of the electronic device will prioritize the allocation of resources to it to ensure that the Surface Flinger process can perform image synthesis at any time to ensure the normal display function of the electronic device.
[0099] The HAL is an interface layer between the operating system kernel and the hardware circuitry of an electronic device. Its purpose is to abstract the hardware. Optionally, as shown in Figure 3, the HAL includes a hardware composer (HWC). The hardware composer is primarily responsible for image synthesis and display functions.
[0100] During startup, an application in the application layer or a system server in the system server layer can request the creation of at least one Surface. Surface Flinger creates a corresponding layer for each Surface, binding the application or system server to the layer so that the user interface of the application or system server corresponding to the layer can be displayed on each layer.
[0101] Based on the layered architecture of the display system shown in Figure 3, in traditional motion effect display solutions, the System Server process or application process can define motion effects according to display requirements during operation, and calculate image frame data in sequence; then the System Server process or application process sends the image frame data obtained each time through cross-process communication to the Surface Flinger process. Based on the received image frame data, the Surface Flinger process synthesizes the image in the layer (Layer) corresponding to the System Server or application, and finally synthesizes and displays the image through the HWC in the HAL. The System Server process or application process in the electronic device, and the Surface Flinger process cyclically execute the above-mentioned process of calculating image frame data and synthesizing and displaying images, and finally realize the motion effect display process of the electronic device.
[0102] The above analysis of traditional motion effect display solutions shows that due to the separation of the image frame data calculation process and the synthesis display process, cross-process communication is frequent. Due to factors such as unstable processing delays of the System Server process or application process and unstable delays of cross-process communication, the motion effects displayed on electronic devices are more likely to experience problems such as freezes, frame drops, and jitters, which may ultimately result in poor motion effect display smoothness on electronic devices.
[0103] In order to ensure the smoothness of the motion effect display of electronic devices and avoid problems such as motion effect freeze, frame loss, and jitter, based on the layered architecture of the display system shown in Figure 3, an embodiment of the present application provides another display system architecture, as shown in Figure 4.
[0104] As shown in FIG4 , the application in the application layer may include a remote animation module, and the remote animation module is used to define animations and generate animation information during the running process of the application.
[0105] The System Server includes the SurfaceAnimRunner, which defines animations and generates animation information during System Server operation. The System Server also includes the AnimationAdaptor, which is responsible for adapting various animations defined by applications or the System Server, managing animation status, and transparently transmitting animation information.
[0106] Surface Flinger includes at least an image synthesis module and a motion effect management module.
[0107] The motion effect management module is responsible for the functions of motion effect state management, motion effect basic capabilities, layer capability docking, etc. As shown in Figure 4, the motion effect management module can start, stop, interrupt or restart the effect according to the instructions from the motion effect adapter, and can also control the parallel display of multiple motion effects. The motion effect management module can also analyze, process and calculate the motion effect information from the motion effect adapter to determine the type, attributes, object of action, differentiator, callback information, etc. of at least one motion effect operation in the motion effect to be displayed, so as to calculate the image frame data at the current moment based on the above content. As shown in the figure, the types of motion effect operations may include, but are not limited to, processing operations on layers such as displacement, cropping, scaling, rounding, rotation, shadow, transparency, blurring, etc. The properties of the motion effect operation may include the display parameters and operation target parameters of the motion effect operation. Exemplarily, the display parameters of the motion effect operation may include, but are not limited to, at least one of the following: the start time, duration, offset time, etc. of the motion effect operation. The target parameters of the animation operation are adaptively set according to the type of the animation operation, for example, the target position or target offset value of the displacement, the target size of the cropping, the target ratio of the scaling, the target angle of the rotation, the target value of the transparency, etc.
[0108] Optionally, the motion effect management module can also perform processing corresponding to the motion effect operation type on the layer, such as displacement, scaling, rotation, etc.
[0109] The image synthesis module is responsible for the traditional image synthesis function of Surface Flinger, and is used to synthesize images in the System Server or the corresponding layer of the application based on the image frame data calculated by the motion effect management module.
[0110] It should be noted that, as an example, this embodiment does not limit the names of the functional modules in each layer, and each functional module may have other names. For example, because the animation management module can provide users of electronic devices with an ultimate animation visual experience, the animation management module may also be called the extreme animation core (EACore) module.
[0111] Based on the structure of the display system shown in FIG4 , an embodiment of the present application further provides a method for displaying dynamic effects. The process of the method will be described below with reference to FIG4 .
[0112] S401: In one embodiment, while an application process is running, a remote animation module in the application process can generate animation information and send the animation information to an animation adapter in a System Server process via cross-process communication. Optionally, the remote animation module can send an animation start instruction carrying the animation information to the animation adapter. In another embodiment, while a System Server process is running, a flat animation runner in the System Server process can generate animation information and send the animation information to the animation adapter.
[0113] The motion effect information is used to indicate the motion effect display content. For example, the motion effect information can be used to indicate at least one motion effect operation in the motion effect to be displayed.
[0114] S402: The animation adapter in the System Server process sends animation information to the Surface Flinger process through cross-process communication. Optionally, the animation adapter can send an animation start instruction carrying the animation information.
[0115] Optionally, the animation adapter can directly send the animation information to the animation management module in the Surface Flinger process, as shown in S402 in Figure 4; or it can send the animation information to other modules in the Surface Flinger process, and the other modules forward the animation information to the animation management module. The embodiments of the present application do not limit the specific implementation method of the animation adapter sending the animation information to the Surface Flinger process through cross-process communication.
[0116] S403: After receiving the motion effect information (or the motion effect start instruction), the motion effect management module starts displaying the motion effect; calculates the image frame data according to the motion effect information; and sends the calculated image frame data to the image synthesis module in the same process.
[0117] S404: The image synthesis module performs image synthesis based on the received image frame to obtain an image frame, and then calls the hardware synthesizer of the HAL to display the image frame. The image synthesis module can perform image synthesis on the layer corresponding to the system server or application, and bind the animation effect to the layer corresponding to the system server or application.
[0118] After each image refresh signal (such as the vertical synchronization (Vsync) signal) is received, the Surface Flinger process calls the motion effect management module and the image synthesis module to calculate the image frame data corresponding to the current moment, and synthesizes the image frame corresponding to the current moment based on the image frame data. This process is repeated to achieve the display of motion effects.
[0119] In one embodiment, the animation information may include data related to at least one animation operation. The data related to any animation operation may include at least one of the following: the type of the animation operation, the start time of the animation operation, the duration of the animation operation, the offset time of the animation operation, the differentiator information of the animation operation, or the callback information of the animation operation.
[0120] The motion effect management module can realize the functions of motion effect basic capability management and layer capability docking management based on the content contained in the motion effect information.
[0121] It should be noted that after the above-mentioned process of starting the animation effect, the remote animation effect module or the plane animation effect runner can also send animation stop instructions, animation interrupt instructions or animation restart instructions to the animation effect management module through the cross-process communication of the animation effect adapter, so that the animation effect management module can realize the animation status management function according to the instructions of the upper layer.
[0122] Optionally, in the above process, the remote animation module, surface animation runner, or animation adapter can serialize the animation information when sending it to improve the transmission efficiency of the animation information. Based on this, after receiving the serialized animation information, the animation management module can first deserialize and parse it to obtain the original animation information; or after other modules in the Surface Flinger process receive the serialized animation information from the animation adapter, they can first deserialize and parse it to obtain the original animation information, and then send the original animation information to the animation management module in the same process.
[0123] Exemplarily, as shown in FIG5 , the serialized animation information (i.e., the animation set) may include: the size of the animation set (Size, i.e., the animation to be displayed includes several animation operations), and the relevant data of at least one serialized animation operation. Taking the relevant data of animation operation 1 as an example, the relevant data of animation operation 1 may include, but is not limited to, parameters such as the type (Type) of animation operation 1, the start time (StartTime) of animation operation 1, the duration (duration) of animation operation 1, the offset time (Offset) of animation operation 1, the values (factors) of the above parameters, and the interpolator (Interpolator) information of animation operation 1. As shown in FIG5 , the interpolator information of animation operation 1 includes: whether animation operation 1 has an interpolator, if there is an interpolator, the type of interpolator, and the values of the parameters in the interpolator information.
[0124] It should also be noted that the image synthesis module and the motion effect management module in the Surface Flinger process can be located in the same thread in the Surface Flinger process or in different threads, and this application does not limit this.
[0125] Through the hierarchical structure of the display system and the method for displaying dynamic effects provided by the embodiment shown in FIG4 , the System Server process or the application process can define dynamic effect information and send it to the Surface Flinger process. The Surface Flinger process displays the dynamic effect based on the received dynamic effect information. Through this solution, it can be seen that the dynamic effect calculation capability of the electronic device is transferred to the Surface Flinger process, which can significantly reduce the number of cross-process communications during the dynamic effect display process. For example, in the process shown in S401-S404, the System Server process or the application process can complete the dynamic effect display by sending the dynamic effect information to the Surface Flinger process through one cross-process communication. Since the Surface Flinger process is a resident process at the bottom layer of the electronic device, compared with the traditional solution, the Surface Flinger process of this solution completes the calculation process in the dynamic effect, which can reduce the system load of the electronic device and ensure the smoothness of the dynamic effect display of the electronic device, avoid the problems of dynamic effect freeze, frame loss, and jitter, and improve the user experience.
[0126] Based on the same technical concept, in order to ensure the smoothness of the dynamic effect display of the electronic device and avoid the problems of dynamic effect freezing, frame loss, and jitter, the embodiment of the present application provides a dynamic effect display method. The method provided by the embodiment of the present application is described below with reference to the flowchart shown in Figure 6.
[0127] Among them, the first process involved in the embodiment of the present application is the process of running the Surface Flinger of the electronic device, that is, the Surface Flinger process; the second process is the process of running the system service System Server or application of the electronic device, that is, the System Server process or the application process.
[0128] S601: During operation, a second process may define an animation and generate animation information based on display requirements. The second process may send the animation information to the first process via inter-process communication. The first process receives the animation information from the second process. The animation information indicates the animation display content, for example, indicating at least one animation operation in the animation to be displayed.
[0129] Optionally, in an actual application scenario, the second process may generate the animation information based on, but not limited to, at least one of the software design of the system server or application, the operation performed by the user, and the running status of the system server or application.
[0130] For example, the second process may send the animation information to the first process through, but is not limited to, an inter-process communication method such as IPC or binder.
[0131] Optionally, in S601, the second process may send motion effect information and a motion effect start instruction to the first process, or the second process may send a motion effect start instruction carrying motion effect information to the first process, so that the first process may start displaying the motion effect according to the motion effect start instruction.
[0132] S602: The first process calculates display data of at least one motion effect operation at different times according to the motion effect information; and calculates image frame data corresponding to different times according to the display data of at least one motion effect operation at different times.
[0133] In one embodiment, each time the first process receives an image refresh signal, it calculates the display data of the at least one animation operation at the current moment based on the animation information, so that the first process can subsequently calculate the image frame data corresponding to the current moment based on the display data of the at least one operation at the current moment, and then synthesize and display the image frame corresponding to the current moment. Exemplarily, the image refresh signal can be a hardware signal sent by hardware in the HAL, such as a Vsync signal.
[0134] Optionally, the animation information may include data related to the at least one animation operation. The data related to any animation operation may include at least one of the following: the type of the animation operation, the start time of the animation operation, the duration of the animation operation, the offset time of the animation operation, the differentiator information of the animation operation, or the callback information of the animation operation.
[0135] Based on this, the first process may calculate the display data of the at least one motion effect operation at the current moment according to the relevant data of the at least one motion effect operation included in the motion effect information each time the image refresh signal is received.
[0136] Optionally, when sending the motion effect information, the second process may serialize the motion effect information, as shown in Figure 5. Then, before executing S602, the first process needs to perform deserialization on the serialized motion effect information to obtain the original motion effect information.
[0137] S603: The first process synthesizes and displays image frames corresponding to different moments according to the image frame data corresponding to different moments to display motion effects.
[0138] In the embodiment of the present application, after the first process obtains the image frame data corresponding to each moment, it can synthesize and display the image frame corresponding to the moment based on the image frame data corresponding to the moment. In this way, the first process repeatedly executes S602-S603 to achieve the display of dynamic effects. Taking the first moment as an example, in one embodiment, after the first process calculates and obtains the image frame data corresponding to the first moment, it can synthesize and display the image frame corresponding to the first moment based on the image frame data corresponding to the first moment.
[0139] It should also be noted that when the first process executes S603, it can synthesize and display the image frame corresponding to each moment on the layer corresponding to the system server or application based on the image frame data corresponding to each moment. In this way, the first process can bind the animation effect to the layer corresponding to the system server or application.
[0140] It should be noted that in the process of the first process displaying the animation through S602-S603, the second process does not calculate the display data of the at least one animation operation at different times based on the animation information (nor does it calculate the image frame data corresponding to different times), and does not send the image frame data to the first process.
[0141] Optionally, after S603 , the second process may further send instructions to the first process through cross-process communication to implement state management of the animation effect, as shown in the following implementation methods:
[0142] Implementation Method 1: The second process sends a stop animation instruction to the first process. When the first process receives the stop animation instruction from the second process, it stops displaying the animation, i.e., the first process stops executing S602-S603. Optionally, after stopping displaying the animation, the first process can also delete the animation information received in S601.
[0143] Implementation method 2: The second process sends an animation interruption instruction to the first process. When the first process receives the animation interruption instruction from the second process, the first process interrupts the display of the animation, that is, the first process suspends execution of S602-S603. The second process can also send an animation restart instruction to the first process. When the first process receives the animation restart instruction from the second process, the first process can continue to display the animation, that is, the first process continues to execute S602-S603.
[0144] An embodiment of the present application provides a method for displaying motion effects. In this method, a System Server process or an application process can define motion effect information and send it to a Surface Flinger process. The Surface Flinger process displays the motion effect based on the received motion effect information. Through this solution, it can be seen that the motion effect calculation capability of the electronic device is sunk to the Surface Flinger process, which can significantly reduce the number of cross-process communications during the motion effect display process. For example, in the above process, the System Server process or the application process can complete the motion effect display by sending the motion effect information to the Surface Flinger process through one cross-process communication. Since the Surface Flinger process is a resident process at the bottom layer of the electronic device, compared with the traditional solution, the Surface Flinger process of this solution completes the calculation process in the motion effect, which can reduce the system load of the electronic device, and can ensure the smoothness of the motion effect display of the electronic device, avoid the problems of motion effect freeze, frame loss, and jitter, and improve the user experience.
[0145] Based on the motion effect display method provided in the above embodiment, this application also provides a motion effect display example. This example is described below with reference to the accompanying drawings.
[0146] As shown in Figure 7A, the System Server process can send the animation information to the Surface Flinger process through the following process:
[0147] S701: When the System Server process needs to display transition animation during operation, a start animation instruction can be sent to the window animation rule unit (WinAnimSpec) through the surface animation runner (SurfaceAnimRunner).
[0148] S702: After receiving the animation start instruction, the window animation rule unit (WinAnimSpec) obtains animation information according to the set window animation rules.
[0149] Optionally, the window animation rule unit (WinAnimSpec) can serialize the acquired animation. The specific process can be referred to the description in the embodiment shown in FIG. 4 or FIG. 6 , which will not be repeated here.
[0150] S703: The window animation rule unit (WinAnimSpec) transmits the animation information to the animation adapter (AnimationAdaptor) through the surface control unit (SurfaceControl) and the surface computing client (SurfaceComposerClient).
[0151] S704: The animation adapter (AnimationAdaptor) transmits the animation information to the Surface Flinger process through the binder method.
[0152] Based on the functional module division method in the Surface Flinger process shown in Figure 7B, optionally, the Surface Flinger main module in the Surface Flinger process can receive motion effect information through S704, and can send the motion effect information to the motion effect manager in the motion effect management module, and the motion effect manager will perform subsequent calculation and processing; or the motion effect manager in the Surface Flinger process can directly receive motion effect information from the motion effect adapter in the System Server process through S704.
[0153] Afterwards, as shown in Figure 7B, the Surface Flinger process can display the animation through the following process:
[0154] S705: The Surface Flinger main module in the Surface Flinger process receives a Vsync signal from the hardware.
[0155] The Surface Flinger main module is responsible for the main functions of Surface Flinger, including synthesizing display image frames. For example, the Surface Flinger main module and the layer state unit can be included in the image synthesis module of the Surface Flinger layer in the display system architecture shown in Figure 4.
[0156] S706: The Surface Flinger main module sends an image frame data request to the motion effect manager in the motion effect management module to request the motion effect manager to return the image frame data corresponding to the current moment.
[0157] S707: The animation manager retrieves all layers (Layer) in the current Surface Flinger and determines the layer (Layer) that needs to display the animation, that is, the animation manager can determine at least one layer (Layer) bound to at least one received animation information. This example is explained by taking the layer (Layer) corresponding to the System Server, Layer 1, as an example. The animation manager determines the animation operation that needs to be displayed on Layer 1 (that is, at least one animation operation contained in the animation information) based on the received animation information. This example is explained by taking the animation information containing animation operation 1 as an example.
[0158] S708: The animation manager obtains the display calculation result of the animation operation 1 at the current moment from the display calculation unit of the animation operation 1.
[0159] The display calculation unit of the motion effect operation 1 may calculate the display calculation result of the motion effect operation 1 at the current moment according to data other than the differentiator information and the callback information in the relevant data of the motion effect operation 1 in the motion effect information.
[0160] S709: The animation manager obtains the differentiator calculation result of the animation operation 1 at the current moment from the differentiator calculation unit of the animation operation 1.
[0161] Among them, the differentiator calculation unit of the dynamic effect operation 1 can calculate the differentiator calculation result of the dynamic effect operation 1 at the current moment according to the differentiator information in the relevant data of the dynamic effect operation 1 in the dynamic effect information.
[0162] S710: The display data for animation operation 1 at the current moment includes the display calculation result for animation operation 1 at the current moment obtained in S708 and the interpolator calculation result for animation operation 1 at the current moment obtained in S709. The animation manager can calculate the image frame data corresponding to Layer 1 at the current moment based on the display data for animation operation 1 at the current moment, and return the image frame data corresponding to Layer 1 at the current moment to the Surface Flinger main module.
[0163] S711: The Surface Flinger main module may call the layer state unit to set the layer state of Layer 1 based on the image frame data corresponding to Layer 1 at the current moment. That is, the Surface Flinger main module synthesizes and displays the current corresponding image frame on Layer 1 based on the image frame data corresponding to Layer 1 at the current moment.
[0164] Since the Vsync signal is sent according to the Vsync cycle, the Surface Flinger process can execute the above process in a loop according to the Vsync cycle, as shown in Figure 7B, thereby completing the display of the System Server transition effect.
[0165] It should be noted that the processes shown in Figures 7A and 7B are examples and do not constitute any limitation on the method provided in this application. Each unit or module in the above process can call related functions, classes, functional bodies, etc. to complete the implementation of related steps. In addition, the above process does not constitute any limitation on the division of each unit or module in the System Server process or Surface Flinger. According to the functional logic division, a unit or module can also be divided into multiple sub-units, or multiple units or modules can be coupled in one unit or module.
[0166] After going through the process shown in Figures 7A and 7B, the electronic device can display the System Server transition effect, as shown in Figure 7C: that is, the homepage of the settings application is gradually covered by the mobile network secondary page that flies out from the right.
[0167] FIG8 is a comparison diagram of the traditional solution and the solution provided by the present application, in which the black solid line represents cross-process communication and the black dotted line represents intra-process communication. In the traditional motion effect display solution, the System Server process needs to continuously refresh the software and calculate each image frame data in the motion effect frame by frame, and send each calculated image frame data to the Surface Flinger process in turn; the Surface Flinger process then synthesizes and displays each image frame frame by frame according to the physical refresh signal. With the motion effect solution provided by the present application, the System Server process only needs to send the defined motion effect information to the Surface Flinger process; the Surface Flinger main module in the Surface Flinger process instructs the motion effect management module to calculate each image frame data in the motion effect frame by frame according to the physical refresh signal; the Surface Flinger main module synthesizes and displays each image frame frame by frame according to the received image frame data. As shown in FIG8, compared with the traditional solution, the number of cross-process communications in the solution of the present application is reduced from n to 1, and the number of software refreshes of the System Server process is also reduced from n to 1. Since the Surface Flinger process is a resident process at the bottom layer of electronic devices, it is obvious that compared with traditional solutions, the Surface Flinger process in this solution can reduce the system load of electronic devices and ensure the smoothness of the animation display of electronic devices, avoiding animation freezes, frame drops, and jitter problems, thereby improving user experience.
[0168] It should also be noted that each step involved in the above embodiments can be performed by an electronic device, or by a component such as a chip, processor, or chip system within the device, and the embodiments of the present application do not limit this. In addition, the specific implementation methods or examples in the above embodiments do not limit the solutions provided in the embodiments of the present application.
[0169] It should be noted that in each of the above embodiments, some steps may be selected for implementation, and the order of the steps in the diagrams may be adjusted for implementation, and this application does not limit this. It should be understood that executing some of the steps in the diagrams, adjusting the order of the steps, or combining them for specific implementation all fall within the scope of protection of this application.
[0170] It should be noted that the "steps" in the embodiments of this application are merely illustrative and serve as a method of expression for a better understanding of the embodiments. They do not constitute a substantial limitation on the execution of the solutions of this application. For example, the "steps" can also be understood as "features." Furthermore, the steps do not constitute any limitation on the execution order of the solutions of this application. Any changes to the order of steps, or any operations such as step merging or splitting that do not affect the implementation of the overall solution, resulting in new technical solutions, are also within the scope of this application.
[0171] Based on the same concept, an embodiment of the present application also provides an electronic device. As shown in Figure 9, the electronic device includes one or more processors 901, a memory 902, a transceiver 903, and a display screen 904. For example, the above-mentioned components can be connected via one or more communication buses. The one or more computer programs are stored in the above-mentioned memory 902 and configured to be executed by the processor 901. The one or more computer programs include instructions, which can be used to enable the electronic device to perform the various steps of the methods in the above-mentioned embodiments.
[0172] For example, the processor 901 may be the processor 210 shown in FIG2 . The memory 902 may be the internal memory 221 shown in FIG2 and / or an external memory connected to the electronic device. The transceiver 903 may be the mobile communication module 250 and / or the wireless communication module 260 shown in FIG2 . The display screen 904 may be the display screen 294 shown in FIG2 , and this embodiment of the present application does not impose any limitation on this.
[0173] Based on the same concept, an embodiment of the present application further provides an electronic device, which includes a unit for executing each step of the method provided in the above embodiment. Optionally, the electronic device may include a display screen and a processing unit. Optionally, the electronic device may also include a communication unit. The display screen is used to display an interface; the communication unit is used to receive and send data; and the processing unit is used to execute the method provided in the above embodiment.
[0174] For example, the modules or units that run the Surface Flinger process in an electronic device include: the Surface Flinger main module and the motion effect management module. The main functions of each module are as follows:
[0175] The motion effect management module is configured to: receive motion effect information from a system server process or an application process, the motion effect information being used to indicate at least one motion effect operation in the motion effect; calculate display data of the at least one motion effect operation at different times based on the motion effect information; and calculate image frame data corresponding to the different times based on the display data of the at least one motion effect operation at different times;
[0176] The Surface Flinger main module is used to synthesize and display image frames corresponding to different moments according to image frame data corresponding to different moments, so as to display the motion effect.
[0177] For the specific functions of the above modules, please refer to the above embodiments and will not be repeated here.
[0178] Based on the above embodiments, the embodiments of the present application further provide a computer program product, which includes a computer program; when the computer program runs on a computer, the computer executes the method provided in the above embodiments.
[0179] Based on the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above embodiments.
[0180] Optionally, the above-mentioned computer may include, but is not limited to, electronic devices.
[0181] The storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.
[0182] Based on the above embodiments, embodiments of the present application further provide a chip configured to read a computer program stored in a memory to implement the methods provided in the above embodiments. Optionally, the chip may include a processor and a memory, wherein the processor is coupled to the memory to read the computer program stored in the memory to implement the methods provided in the above embodiments.
[0183] Based on the above embodiments, embodiments of the present application provide a chip system, which includes a processor for supporting a computer device to implement the functions involved in the electronic device in the above embodiments. In one possible design, the chip system also includes a memory for storing the necessary programs and data for the computer device. The chip system can be composed of a chip or can include a chip and other discrete devices.
[0184] In summary, the embodiments of the present application provide a method for displaying motion effects and an electronic device. In this method, the process running the System Server or the application (referred to as the System Server process or the application process for short) defines the motion effect information and sends it to the process running the Surface Flinger (referred to as the Surface Flinger process for short). The Surface Flinger process displays the motion effect based on the received motion effect information. Through this solution, it can be seen that the motion effect computing capability of the electronic device is sunk to the Surface Flinger process, which can significantly reduce the number of cross-process communications during the motion effect display process. Since the Surface Flinger process is a resident process at the bottom layer of the electronic device, therefore, compared with the traditional solution, the Surface Flinger process of this solution completes the calculation process in the motion effect, which can reduce the system load of the electronic device, and can ensure the smoothness of the motion effect display of the electronic device, avoid the problems of motion effect freeze, frame loss, and jitter, and improve the user experience.
[0185] As used in the above embodiments, the terms “when…” or “after…” may be interpreted to mean “if…” or “after…” or “in response to determining…” or “in response to detecting…”, depending on the context. Similarly, the phrases “upon determining…” or “if (stated condition or event) is detected” may be interpreted to mean “if determining…” or “in response to determining…” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.
[0186] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)). In the absence of conflict, the solutions of the above embodiments can be used in combination.
[0187] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0188] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.
[0189] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0191] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.
Claims
1. A method for displaying a dynamic effect, characterized in that: include: The first process receives the motion effect information from the second process; wherein the first process is a process for running the image plane engine Surface Flinger of the electronic device, and the second process is a process for running the system service System Server or application of the electronic device; the motion effect information is used to indicate at least one motion effect operation in the motion effect; The first process calculates display data of the at least one motion effect operation at different times according to the motion effect information; and calculates image frame data corresponding to different times according to the display data of the at least one motion effect operation at different times; The first process synthesizes and displays image frames corresponding to different moments according to the image frame data corresponding to different moments, so as to display the motion effect.
2. The method according to claim 1, characterized in that The first process calculates display data of the at least one dynamic effect operation at different times according to the dynamic effect information, including: Each time the first process receives an image refresh signal, it calculates display data of the at least one motion effect operation at the current moment according to the motion effect information.
3. The method according to claim 2, characterized in that The image refresh signal is a vertical synchronization Vsync signal.
4. The method according to any one of claims 1 to 3, characterized in that: The motion effect information includes relevant data of the at least one motion effect operation; The relevant data of any animation operation includes at least one of the following: the type of the animation operation, the start time of the animation operation, the duration of the animation operation, the offset time of the animation operation, the differentiator information of the animation operation, or the callback information of the animation operation.
5. The method according to any one of claims 1 to 4, characterized in that: The first process synthesizes and displays image frames corresponding to different moments according to image frame data corresponding to different moments, including: The first process synthesizes and displays image frames corresponding to different moments on a layer corresponding to the System Server or the application according to the image frame data corresponding to different moments.
6. The method according to any one of claims 1 to 5, characterized in that: During the process of the first process displaying the motion effect, the second process does not calculate the display data of the at least one motion effect operation according to the motion effect information, and the second process does not send the image frame data to the first process.
7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When the first process receives the motion effect stop instruction from the second process, the first process stops calculating the display data of the at least one motion effect operation at different times to stop displaying the motion effect.
8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: When the first process receives the motion effect interruption instruction from the second process, the first process interrupts the calculation of the display data of the at least one motion effect operation at different times to interrupt the display of the motion effect; When the first process receives the animation restart instruction from the second process, the first process continues to calculate the display data of the at least one animation operation at different times to continue to display the animation.
9. An electronic device, characterized in that: The electronic device includes a display screen, a memory and one or more processors; The memory is used to store computer program codes, and the computer program codes include computer instructions; when the computer instructions are executed by the one or more processors, the electronic device executes the method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 8.
11. A computer program product, characterized in that The computer program product comprises a computer program or instructions, and when the computer program or instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.
12. A chip, characterized in that: It comprises a processor and a memory; the processor is coupled to the memory, and the processor is used to call a computer program stored in the memory during operation to implement the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Image display method and device, electronic equipment and computer readable storage medium
CN111752520A
Image processing method and electronic equipment
CN114092595A
Dynamic effect calculation method and device
CN115408071A
Smooth detection method
CN116048933A