Animation frame synthesis method and device

Abstract

The embodiment of the invention provides an animation frame synthesis method and device, relates to the field of terminals, and can reduce the power consumption of electronic equipment on the basis of ensuring the display effect of an animation special effect. The method is applied to the electronic equipment, the electronic equipment comprises a GPU and an HWC, the method comprises the steps that the electronic equipment receives a first operation, the first operation is used for triggering the electronic equipment to display an animation special effect, the animation special effect comprises N animation frames, and N is an integer larger than 0; the electronic equipment synthesizes M animation frames in the N animation frames through the HWC and synthesizes N-M animation frames in the N animation frames through the GPU, and the window height of each animation frame in the M animation frames is smaller than that of each animation frame in the N-M animation frames; wherein M is an integer larger than 0 and smaller than N, each frame in the M animation frames is a rectangular image, and each frame in the N-M animation frames is a fillet image.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to an animation frame synthesis method and device. Background Art

[0002] Currently, when a user starts or exits an application on an electronic device, the electronic device can perform animation rendering and synthesis on the process of starting or exiting the application to improve the display effect of the electronic device.

[0003] The animation effect of starting or exiting an application may include multiple frames of images. In the related art, each frame of the image may be rounded to cut the rigid right-angle turning vertices of each frame of the image into smooth rounded corners, thereby achieving the purpose of improving the display effect.

[0004] However, in high frame rate (for example, 120 frames, 90 frames or 60 frames) display scenarios, electronic devices round the corners of each frame of the image, which greatly reduces the efficiency of animation rendering and synthesis, resulting in high power consumption, high load, and severe heat generation problems. It may also cause the display page to freeze, affecting the viewing experience of users of the electronic device. Summary of the invention

[0005] The embodiments of the present application provide an animation frame synthesis method and device, which can reduce the power consumption of electronic devices while ensuring the display effect of animation special effects.

[0006] In a first aspect, an embodiment of the present application provides an animation frame synthesis method, which is applied to an electronic device, wherein the electronic device includes a graphics processing unit (GPU) and a hardware composer (HWC), and the method includes: receiving a first operation, the first operation is used to trigger the electronic device to display an animation special effect, the animation special effect includes N animation frames, N is an integer greater than 0; the electronic device synthesizes M animation frames out of the N animation frames through the HWC, and synthesizes NM animation frames out of the N animation frames through the GPU, the window height of each animation frame in the M animation frames is less than the window height of each animation frame in the NM animation frames; wherein M is an integer greater than 0 and less than N, each of the M animation frames is a rectangular image, and each of the NM animation frames is a rounded image.

[0007] Based on the method provided in the embodiment of the present application, M animation frames with smaller window heights among N animation frames can be synthesized by HWC, and NM animation frames with larger window heights among N animation frames can be synthesized by GPU. Since the human eye is not sensitive to the M animation frames with smaller window heights (not easy for people to recognize), and the human eye is more sensitive to the NM animation frames with larger window heights (people can recognize), the animation frames (M animation frames) that are not easy for people to recognize are synthesized by HWC, and the animation frames (NM animation frames) that can be recognized by people are synthesized by GPU. The problems of high power consumption, high load, and severe heat generation caused by the electronic device performing rounded corner processing on each animation frame (i.e., synthesizing each animation frame by GPU) can be avoided, and the performance of electronic devices in displaying animation special effects can be improved on the basis of ensuring the display effect of animation special effects.

[0008] In a possible implementation, at a first frame rate, the window height of each animation frame in the M animation frames is less than a first threshold, and the window height of each animation frame in the NM animation frames is greater than or equal to the first threshold; at a second frame rate, the window height of each animation frame in the M animation frames is less than a second threshold, and the window height of each animation frame in the NM animation frames is greater than or equal to the second threshold; wherein, the first frame rate is greater than the second frame rate, and the first threshold is greater than the second threshold. It should be understood that when the frame rate is large, the display time of each animation frame is short, so the requirement for the window height is correspondingly high. When the frame rate is small, the display time of each animation frame is long, so the requirement for the window height is correspondingly low. In other words, when the frame rate is large, the user can only recognize the animation frame with a high window height, and when the frame rate is small, the user can also recognize the animation frame with a low window height, so setting different window heights according to different frame rates can effectively identify the animation frames that are sensitive to the human eye (recognizable by humans). Further, the animation frames that can be recognized by humans are synthesized by the GPU, and the animation frames that are not easily recognized by humans are synthesized by the HWC. It can avoid the problems of high power consumption, high load and severe heat generation caused by the electronic device performing rounded corners on each animation frame (i.e. synthesizing each animation frame through the GPU). On the basis of ensuring the display effect of animation special effects, the performance of electronic devices in displaying animation special effects can be improved.

[0009] In one possible implementation, the electronic device installs a first application, and the first operation is used to start the first application, or the first operation is used to exit the first application. In response to the user's operation of starting or exiting the first application (first operation), the electronic device can animate the process of starting or exiting the first application. The animation frame synthesis method provided in the embodiment of the present application can be applied to the scenario where the user starts or exits an application (for example, the first application).

[0010] In a possible implementation, the method further includes: obtaining the features of each of the N animation frames, the features of each of the animation frames including the frame rate and the window height; normalizing the features of each of the animation frames; inputting the normalized data into the classification model to obtain an output value, the output value being used to indicate whether each of the animation frames belongs to M animation frames or NM animation frames. That is, the classification model can be used to determine whether each of the animation frames belongs to M animation frames or NM animation frames. Then, the animation frames (M animation frames) that are not easily recognized by humans can be synthesized by the HWC, and the animation frames (NM animation frames) that can be recognized by humans can be synthesized by the GPU. The problems of high power consumption, high load, and severe heat generation caused by the rounded corners of each animation frame by the electronic device (i.e., synthesizing each animation frame by the GPU) can be avoided, and the performance of the electronic device in displaying animation effects can be improved on the basis of ensuring the display effect of the animation special effects.

[0011] In a possible implementation, the classification model is any one of a support vector machine (SVM) model, a neural network model, a random separation model, a linear regression model or a logistic regression model.

[0012] In a possible implementation, the classification model is a logistic regression model, and the logistic regression model is as follows:

[0013]

[0014] Among them, y indicates whether the current animation frame belongs to the animation frame that the human eye is sensitive to, the current animation frame is the animation frame currently input into the logistic regression model, the current animation frame is any one of the N animation frames, x represents a feature vector set, the feature vector set includes the encoding value of at least one feature of the current animation frame, at least one feature includes a frame rate and a window height, W represents a weight vector set, the weight vector set includes the weights corresponding to at least one feature of the current animation frame, T represents a transposition operation, and b represents a bias. That is, the logistic regression model can be used to determine whether each animation frame belongs to M animation frames or NM animation frames. Then, the animation frames (M animation frames) that are not easy for people to recognize can be synthesized by HWC, and the animation frames (NM animation frames) that can be recognized by people can be synthesized by GPU. The problems of high power consumption, high load, and severe heat generation caused by the electronic device performing rounded corner processing on each animation frame (i.e., synthesizing each animation frame by GPU) can be avoided. On the basis of ensuring the display effect of animation special effects, the performance of electronic devices in displaying animation special effects can be improved.

[0015] In a possible implementation, the method further includes: constructing a data set, the data set including a feature vector set of a plurality of annotated animation frame samples, the feature vector set including a frame rate and a window height; and obtaining a logistic regression model based on training of the data set. Exemplarily, the original model can be trained with the above data set to obtain a target regression coefficient. The original model can be a built logistic regression model that has not yet been trained. The logistic regression model can be obtained by replacing the regression coefficient in the original model with the target regression coefficient.

[0016] In a possible implementation, the electronic device also includes a desktop application and an image synthesis system, and the method also includes: the desktop application draws and renders the N animation frames included in the animation special effects to obtain N animation frames to be synthesized; the desktop application sends the N animation frames to be synthesized to the image synthesis system; the image synthesis system determines the animation frames that are sensitive to the human eye and the animation frames that are not sensitive to the human eye among the N animation frames to be synthesized, the animation frames that are sensitive to the human eye belong to NM animation frames, and the animation frames that are not sensitive to the human eye belong to M animation frames; the image synthesis system sends the animation frames that are not sensitive to the human eye to the HWC, and sends the animation frames that are sensitive to the human eye to the GPU. That is, the animation frames that are not easy to recognize by people (M animation frames) can be synthesized by the HWC, and the animation frames that can be recognized by people (NM animation frames) can be synthesized by the GPU. The problems of high power consumption, high load, and severe heat generation caused by the electronic device performing rounded corner processing on each frame of animation (that is, synthesizing each frame of animation by the GPU) can be avoided, and the performance of the electronic device in displaying animation special effects can be improved on the basis of ensuring the display effect of the animation special effects.

[0017] In a possible implementation, the electronic device also includes a display screen, and the method also includes: the HWC performs synthesis processing on the received animation frame to obtain a first animation frame to be displayed; the GPU performs synthesis processing on the received animation frame to obtain a second animation frame to be displayed; the HWC sends the first animation frame to be displayed to the display screen, and the display screen displays the first animation frame to be displayed; the GPU sends the second animation frame to be displayed to the display screen, and the display screen displays the second animation frame to be displayed. That is, the animation frames (M animation frames) that are not easy for people to recognize can be synthesized by the HWC, and the animation frames (NM animation frames) that can be recognized by people can be synthesized by the GPU. The problems of high power consumption, high load, and severe heat generation caused by the electronic device performing rounded corner processing on each animation frame (that is, synthesizing each animation frame by the GPU) can be avoided. On the basis of ensuring the display effect of the animation special effects, the performance of the electronic device in displaying animation special effects can be improved.

[0018] In a second aspect, the present application provides a chip system, which includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by lines. The above chip system can be applied to an electronic device including a communication module and a memory. The interface circuit is used to receive a signal from the memory of the electronic device and send the received signal to the processor, and the signal includes a computer instruction stored in the memory. When the processor executes the computer instruction, the electronic device can execute the method described in the first aspect and any possible design thereof.

[0019] In a third aspect, the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device (such as a mobile phone), the electronic device executes the method described in the first aspect and any possible design thereof.

[0020] In a fourth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the first aspect and any possible design thereof.

[0021] In a fifth aspect, an embodiment of the present application provides an animation frame synthesis device, including a processor, the processor and a memory are coupled, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the device implements the method described in the first aspect and any possible design thereof. The device can be an electronic device or a server device; or it can be a component of an electronic device or a server device, such as a chip.

[0022] In the sixth aspect, an embodiment of the present application provides an animation frame synthesis device, which can be divided into different logical units or modules according to functions, and each unit or module performs different functions, so that the device executes the method described in the above-mentioned first aspect and any possible design method thereof.

[0023] It can be understood that the beneficial effects that can be achieved by the chip system described in the second aspect, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the devices described in the fifth aspect and the sixth aspect provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1A A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0025] Figure 1B A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of interaction between modules provided in an embodiment of the present application;

[0028] Figure 4 A display schematic diagram provided for an embodiment of the present application;

[0029] Figure 5 A schematic diagram of a synthesis method of animation frames provided in an embodiment of the present application;

[0030] Figure 6 Another display schematic diagram provided for an embodiment of the present application;

[0031] Figure 7 A schematic diagram of another interaction between modules provided in an embodiment of the present application;

[0032] Figure 8 Another display schematic diagram provided for an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of another method for synthesizing animation frames provided in an embodiment of the present application;

[0034] Fig.10 Another display schematic diagram provided for the embodiment of the present application;

[0035] Fig.11 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant concepts or technologies is first given:

[0037] Frame: refers to the smallest unit of a single picture in the interface display. A frame can be understood as a still picture. Displaying multiple connected frames quickly and continuously can create the illusion of motion. For example, an animation effect consists of multiple frames (animation frames) displayed continuously.

[0038] Frame rate: refers to the number of frames that refresh an image in 1 second. It can also be understood as the number of times the graphics processor in an electronic device refreshes the screen per second. A high frame rate can produce smoother and more realistic animations. The more frames per second, the smoother the displayed action will be.

[0039] It should be noted that frames usually need to go through processes such as drawing, rendering, and synthesis before being displayed.

[0040] Frame drawing: refers to the picture drawing of the display interface. The display interface can be composed of one or more views, each view can be drawn by the visual control of the view system, each view is composed of subviews, and a subview corresponds to a small widget in the view, for example, one of the subviews corresponds to a symbol in the picture view.

[0041] Frame rendering: It is to color the drawn view or add 3D effects, etc. For example, 3D effects can be lighting effects, shadow effects, and texture effects.

[0042] Frame synthesis: is the process of synthesizing multiple or more rendered views into a display interface.

[0043] Image synthesis system (SurfaceFlinger): A system service in the Android system that manages the rendering and synthesis process of frames. When an application performs a drawing operation, it can send the drawn content to SurfaceFlinger. After SurfaceFlinger receives the graphic data from the application, it synthesizes it into the final screen image.

[0044] Among them, SurfaceFlinger can synthesize the graphics data of the application into the final screen image through image synthesis technologies such as HWC synthesis or GPU synthesis.

[0045] HWC synthesis: A hardware-based image synthesis technology that uses hardware-level image synthesis capabilities to improve image synthesis performance and efficiency. In HWC synthesis, the task of graphics synthesis is performed by the hardware module, without the need for software-level synthesis operations through the CPU or GPU. The advantages of HWC synthesis include: 1) High performance: Due to the use of hardware acceleration, HWC synthesis is generally able to provide higher frame rates and smoother animation effects. 2) Low power consumption: By directly utilizing hardware resources for synthesis, HWC synthesis can reduce the power consumption of the CPU and GPU. 3) Real-time response: HWC synthesis can display the graphics content of the application on the screen faster, providing faster user interface response time. Disadvantages of HWC synthesis: Graphics synthesis operations such as rotation, rounded corners, and color gamut are not supported.

[0046] GPU compositing: A software-based image compositing technology that uses the GPU to perform graphics compositing operations. In GPU compositing, the graphics operations of the application are sent by the CPU to the GPU, which then performs the compositing operations at the software level and finally transmits the final image data to the display for display. The advantage of GPU compositing is high flexibility. Since it is based on software implementation, GPU compositing can support more complex graphics compositing operations such as filters, transformations, and rounded corners. The disadvantages of GPU compositing are high power consumption and lower performance than HWC compositing.

[0047] At present, in order to achieve the rounded corner effect of animation frames, electronic devices can use GPU synthesis technology to round the corners of animation frames. However, in high frame rate (for example, 120 frames, 90 frames or 60 frames) display scenes, electronic devices perform rounded corner processing on each animation frame, which greatly reduces the efficiency of animation rendering synthesis, resulting in high power consumption, high load, and serious heat generation problems. It may also cause the display page to freeze, affecting the viewing experience of users of electronic devices.

[0048] For the above scenarios, an embodiment of the present application provides an animation frame synthesis method, which can reasonably use image synthesis technologies such as GPU synthesis and HWC synthesis to reduce the power consumption and load of electronic devices, avoid display page freezes, and improve the performance of electronic devices in displaying animation special effects.

[0049] The animation frame synthesis method provided in the embodiment of the present application can be applied to the scenario where the user starts or exits an application (for example, the first application).

[0050] Figure 1A FIG. 1 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. Figure 1A As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0051] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0052] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0053] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0054] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0055] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0056] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0057] It is understandable that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0058] The charging management module 140 is used to receive charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also power the electronic device through the power management module 141 .

[0059] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

[0060] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0061] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network.

[0062] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0063] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194.

[0064] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0065] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0066] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0067] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0068] The electronic device 100 can realize the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0069] The camera 193 may include 1 to N cameras. For example, the electronic device may include 2 front cameras and 4 rear cameras. NPU is a neural network (NN) computing processor. By drawing on the biological neural network structure, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0070] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in an external memory card. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 can execute various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 can execute instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0071] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0072] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 170D is used to connect wired headphones.

[0073] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The electronic device 100 can receive the button input and generate a key signal input related to the user settings and function control of the electronic device 100. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0074] The methods in the following embodiments can all be implemented in the electronic device 100 having the above hardware structure.

[0075] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0076] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, the HAL layer, and the kernel layer. It should be noted that the embodiments of the present application are illustrated by taking the Android system as an example. In other operating systems (such as Hongmeng system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those of the embodiments of the present application, the solutions of the present application can also be implemented.

[0077] Among them, the application layer can include a series of application packages.

[0078] like Figure 1B As shown, the application package may include calls, short messages, application installers, notification centers (also known as notification bars / pull-down notification bars, notification management, etc.), settings, galleries, calendars, maps, navigation, WLAN, Bluetooth, music, videos and other applications.

[0079] The application framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. The application framework layer includes some predefined functions. For example, it may include an activity manager, a window manager, a content provider, a view system, a resource manager, a notification manager, etc., and the embodiments of the present application do not impose any restrictions on this.

[0080] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0081] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0082] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0083] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0084] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0085] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0086] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0087] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0088] OpenGL ES is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0089] SGL is a graphics engine for 2D graphics.

[0090] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0091] The HAL layer is an encapsulation of the Linux kernel driver, providing an interface to the upper layer and shielding the implementation details of the low-level hardware.

[0092] The HAL layer may include Wi-Fi HAL, audio HAL, camera HAL, etc.

[0093] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, sensor driver, etc.

[0094] like Figure 2 As shown, in an embodiment of the present application, the software related to frame synthesis in the electronic device includes but is not limited to: desktop applications and image synthesis systems (SurfaceFlinger). The desktop application is located at the application layer in the layered architecture of the electronic device, and SurfaceFlinger is located at the system layer in the layered architecture of the electronic device. The hardware related to frame synthesis in the electronic device includes but is not limited to: GPU and HWC. HWC can be an independent device or integrated in a system-on-chip (SOC). The electronic device completes the synthesis of multiple animation frames through the interaction of desktop applications, SurfaceFlinger, GPU and HWC.

[0095] The technical scheme in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. Wherein, in the description of the present application, unless otherwise specified, "at least one" refers to one or more, and "a plurality of" refers to two or more than two. In addition, in order to facilitate the clear description of the technical scheme of the embodiment of the present application, in the embodiment of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit necessarily different.

[0096] For ease of understanding, the animation frame synthesis method provided in the embodiment of the present application is specifically introduced below with reference to the accompanying drawings.

[0097] like Figure 3 As shown, an embodiment of the present application provides an animation frame synthesis method, taking the scenario of starting an application (for example, a first application, which may be a camera application) as an example, the process of animation frame synthesis is described, including:

[0098] 301. The desktop application receives an operation from a user to start a camera application.

[0099] The operation of the user starting the camera application (an example of the first operation) may be an operation of the user clicking the icon of the camera application from the desktop. Figure 4 As shown in (a) of FIG. 4 , the electronic device may display a main interface 401, and the main interface 401 includes a camera application logo 402. The electronic device may receive an operation (eg, a click operation) performed by a user on the camera application logo 402. In response to the operation, the camera application is started, such as Figure 4 As shown in (b) , the electronic device may display a shooting preview interface 403 .

[0100] Optionally, the user may start the camera application by clicking on the window (card) of the camera application from the multitasking interface. That is, the camera application may be cold started (first start) or hot started (started from the background), which is not limited in this application.

[0101] 302. The desktop application sends N animation frames to be synthesized to SurfaceFlinger.

[0102] In response to a user's operation of launching an application (eg, a camera application), the desktop application may animate the process of launching the application (eg, the camera application).

[0103] The desktop application can call the drawing and rendering thread to draw and render the multiple animation frames included in the animation special effect to obtain the animation frames to be synthesized. Then, the N animation frames to be synthesized can be sent to SurfaceFlinger.

[0104] In some embodiments, the desktop application obtains an animation frame to be synthesized through rendering by the drawing rendering thread, and sends the animation frame to be synthesized to SurfaceFlinger.

[0105] 303. SurfaceFlinger determines whether the current scene is a scene that requires motion clipping.

[0106] The scene requiring motion effect cutting may include the scene of starting the application program, wherein the scene of starting the application program includes the scene of cold start and hot start of the application program.

[0107] It should be understood that in the scenario of launching an application, the electronic device can perform animation rendering synthesis on the process of launching the application. That is, the electronic device can display the animation effects of launching the application. Among them, the animation effects of launching the application can include the animation effects of cold starting the application and the animation effects of hot starting the application.

[0108] The animation effect of cold starting an application may include a plurality of animation frames that change continuously, and each animation frame may include elements such as a background window and an application icon. In the plurality of animation frames that change continuously including the animation effect of cold starting an application, the background window included in the previous animation frame may be smaller than the background window included in the next animation frame. Optionally, the application icon included in the previous animation frame may be smaller than the application icon included in the next animation frame. Alternatively, the center of the application icon included in the previous animation frame is closer to the center of the screen of the electronic device than the center of the application icon included in the next animation frame.

[0109] The animation effects of hot starting an application may include multiple animation frames that change continuously, and each animation frame may include a screenshot of the application's window. Among them, the screenshot of the application's window may refer to a screenshot of the window displayed when the application exits. Among the multiple animation frames that change continuously included in the animation effects of hot starting an application, the screenshot included in the previous animation frame is smaller than the screenshot included in the next animation frame. For example, during the startup of a camera application, a desktop application can obtain the window layout of the camera application, which defines the size and position of the camera application's window. The window layout of the camera application can be defined by an XML resource file in the Android manifest. The desktop application can obtain the window layout of the camera application from the XML resource file.

[0110] It can be understood that the operation of starting the camera application is used to start the camera application, so the current scene is the scene for starting the application, that is, the current scene belongs to the scene that requires motion effect clipping, and step 304 can be executed.

[0111] 304. SurfaceFlinger determines animation frames that are sensitive to the human eye and animation frames that are not sensitive to the human eye among the N animation frames to be synthesized.

[0112] It should be noted that among the multiple animation frames (for example, N animation frames) included in the animation special effects of launching the application, some animation frames (for example, NM animation frames out of the N animation frames) are animation frames that are sensitive to the human eye, and the other part of the animation frames are animation frames that are not sensitive to the human eye (for example, M animation frames out of the N animation frames). Animation frames that are sensitive to the human eye are animation frames that can be recognized (distinguished) by the human eye. For example, the human eye can distinguish animation frames that are enlarged to nearly the entire screen and whose motion trajectory of the motion effect curve is based on a smooth motion in the process of launching the application. Animation frames that are not sensitive to the human eye are animation frames that are difficult for the human eye to recognize. For example, it is difficult for the human eye to distinguish animation frames that are enlarged frame by frame from small ones in the process of launching the application.

[0113] In some embodiments, the window height of each animation frame in the animation frames that are not sensitive to the human eye (e.g., M animation frames out of N animation frames) is smaller than the window height of each animation frame in the animation frames that are sensitive to the human eye (e.g., NM animation frames out of N animation frames).

[0114] In some embodiments, at a first frame rate, the window height of each animation frame in the animation frames that are not sensitive to the human eye (e.g., M animation frames out of N animation frames) is less than the first threshold, and the window height of each animation frame in the animation frames that are sensitive to the human eye (e.g., NM animation frames out of N animation frames) is greater than or equal to the first threshold. At a second frame rate, the window height of each animation frame in the animation frames that are not sensitive to the human eye (e.g., M animation frames out of N animation frames) is less than the second threshold, and the window height of each animation frame in the animation frames that are sensitive to the human eye (e.g., NM animation frames out of N animation frames) is greater than or equal to the second threshold. That is, at the first frame rate, the animation frames whose window height is less than the first threshold are animation frames that are not sensitive to the human eye, and the animation frames whose window height is greater than or equal to the first threshold are animation frames that are sensitive to the human eye; at the second frame rate, the animation frames whose window height is less than the second threshold are animation frames that are not sensitive to the human eye, and the window height is greater than or equal to the second threshold is animation frames that are sensitive to the human eye. Alternatively, at a first frame rate, an animation frame whose window height is less than or equal to a first threshold is an animation frame that is not sensitive to the human eye, and an animation frame whose window height is greater than the first threshold is an animation frame that is sensitive to the human eye; at a second frame rate, an animation frame whose window height is less than or equal to a second threshold is an animation frame that is not sensitive to the human eye, and an animation frame whose window height is greater than the second threshold is an animation frame that is sensitive to the human eye. Among them, the first frame rate is greater than the second frame rate, and the first threshold is greater than the second threshold. For example, when the frame rate is 120fps, if the window height of the animation frame is greater than or equal to 1500px (px represents pixels), it indicates that the animation frame is an animation frame that is sensitive to the human eye, otherwise it is an animation frame that is not sensitive to the human eye. For another example, when the frame rate is 90fps, if the frame height of the animation frame is greater than or equal to 1400px, it indicates that the animation frame is an animation frame that is sensitive to the human eye, otherwise it is an animation frame that is not sensitive to the human eye.

[0115] In some embodiments, an intelligent decision algorithm can be used to determine animation frames that are sensitive to the human eye and animation frames that are not sensitive to the human eye, so that different synthesis processes are performed on these two different types of animation frames. For example, only animation frames that are sensitive to the human eye can be rounded, and animation frames that are not sensitive to the human eye can be left without rounding, so as to reduce the power consumption of the electronic device and improve the performance of the electronic device in performing animation rendering and synthesis.

[0116] Among them, the intelligent decision-making algorithm may include the following steps:

[0117] (1) Identify the features of each animation frame included in the animation special effect. Exemplarily, the features of each animation frame include but are not limited to at least one of the following features: frame rate, animation special effect triggering scene (e.g., sliding scene, startup scene, exit scene, etc.), window attributes, etc. Optionally, the features of each animation frame may also include at least one of the following features: animation special effect triggering event (e.g., click event), GPU attribute, CPU attribute, application animation attribute, touch attribute.

[0118] Among them, the GPU attribute may include the clock frequency of the GPU. Optionally, the GPU attribute may also include memory bandwidth, pixel fill rate, etc. The CPU attribute may include the clock frequency of the CPU, etc. Optionally, the CPU attribute may also include the type of CPU, and the type of CPU may include a large core processor, a medium core processor, and a small core processor. It should be understood that the processing speed of the large core processor is higher than that of the medium core processor, and the processing speed of the medium core processor is higher than that of the small core processor. Exemplarily, the identifier of the small core processor may include 0-3, the identifier of the medium core processor may include 4-5, and the identifier of the large core processor may include 6. A click event may include the position coordinates of the user's finger on the touch screen. The application animation attribute may include the type of animation (e.g., interface animation) and the triggering method of the animation (e.g., passive triggering). The animation curve attribute may include slow-in, slow-out, and uniform speed, etc. The touch attribute may be used to indicate whether a touch operation is received. The window attribute may include window width (WindowW), window height (WindowH), the center coordinates of the window, whether the window supports scaling, rotation, and other attributes.

[0119] (2) Encoding the features of each animation frame. For example, the features of the animation frame can be digitized. The features of the animation frame can include continuous numerical features and discontinuous category features. For continuous numerical features, normalization can be performed to obtain the corresponding numerical values. For discontinuous category features, they can be hashed or one-hot encoded. For example, the discontinuous category features can include animation special effect triggering scenes. The animation special effect triggering scenes can include sliding scenes, startup scenes or exit scenes. If the animation special effect triggering scene is not a sliding scene, the code corresponding to the sliding scene can be 0, and if the animation special effect triggering scene is a sliding scene, the code corresponding to the sliding scene can be 1. If the animation special effect triggering scene is not an exit scene, the code corresponding to the exit scene can be 0, and if the animation special effect triggering scene is an exit scene, the code corresponding to the exit scene can be 1. If the animation special effect triggering scene is not a startup scene, the code corresponding to the startup scene can be 0, and if the animation special effect triggering scene is a startup scene, the code corresponding to the startup scene can be 1.

[0120] As shown in Table 1, this is an example of digitization after feature encoding processing of an animation frame.

[0121] Table 1

[0122]

[0123] As can be seen from Table 1, the code corresponding to the startup scene is 1, the code corresponding to the exit scene is 0, and the code corresponding to the sliding scene is 0, that is, the animation special effect triggering scene is the startup scene. The code corresponding to the touch attribute is 1, indicating that a touch operation is received. The window width (WindowW) can be 1400, the window height (WindowH) can be 1920, the frame rate can be 120, the GPU operating frequency can be 200M, and the CPU type can be 0, which means that the CPU is a small-core processor.

[0124] (3) The encoded data (including the normalized data) is input into the classification model to obtain an output value, and the output value is used to determine whether the current frame (the currently input animation frame) is an animation frame that is sensitive to the human eye.

[0125] Among them, the classification model can be trained in the following manner: (1) obtaining N frames of animation frames; (2) determining the features of each frame of the N frames of animation frames; (3) constructing a classification model based on the features of each frame of the N frames of animation frames and the annotation of whether the N frames of animation frames are face-sensitive animation frames. Exemplarily, the following manner can be adopted to mark whether each frame of the N frames of animation frames is a face-sensitive animation frame: for example, when the frame rate is 120fps, if the window height of the animation frame is greater than or equal to 1500px (px represents pixels), the animation frame is marked as an animation frame that is sensitive to the human eye, otherwise it is an animation frame that is not sensitive to the human eye. For another example, when the frame rate is 90fps, if the frame height of the animation frame is greater than or equal to 1400px, the animation frame is marked as an animation frame that is sensitive to the human eye, otherwise it is an animation frame that is not sensitive to the human eye.

[0126] Among them, the classification model can be a machine learning model, and the machine learning model can include a support vector machine (SVM) model, or a neural network model, or a random separation model, a linear regression model or a logistic regression model, etc. This application does not specifically limit the classification model.

[0127] For example, taking the classification model as a logistic regression model, the logistic regression model can be shown as formula (1).

[0128]

[0129] Wherein, y represents whether the current animation frame (the animation frame currently input into the logistic regression model) is an animation frame that is sensitive to the human eye, or the probability that the current animation frame is an animation frame that is sensitive to the human eye. x represents a feature vector set, which includes the encoding value of at least one feature of the current animation frame. W represents a weight vector set, which includes the weights corresponding to at least one feature of the current animation frame, T represents a transposition operation, and b represents a bias.

[0130] The data set D of the logistic regression model can be shown as formula (2).

[0131]

[0132] Wherein, xi represents the feature vector set of S animation frame samples, and S is an integer greater than or equal to 1. xi=[x1,x2,...,xS], where x1 represents the feature vector set of animation frame sample 1 (e.g., animation frame 1), x2 represents the feature vector set of animation frame sample 2 (e.g., animation frame 2), and so on, xS represents the feature vector set of animation frame sample S (e.g., animation frame S). yi represents the annotation of S animation frame samples. yi=[y1,y2,...,yS], where y1 represents the annotation (e.g., 0 or 1) of animation frame sample 1 (e.g., animation frame 1), y2 represents the annotation (e.g., 0 or 1) of animation frame sample 2 (e.g., animation frame 2), and so on, yS represents the annotation (e.g., 0 or 1) of animation frame sample S (e.g., animation frame S). yi takes values ​​in {0,1}. For example, 1 may represent that the current animation frame is an animation frame that is sensitive to the human eye, and 0 may represent that the current animation frame is an animation frame that is not sensitive to the human eye. Alternatively, 1 may indicate that the current animation frame is an animation frame to which the human eye is not sensitive, and 0 may indicate that the current animation frame is an animation frame to which the human eye is sensitive, and this application does not make any limitation thereto.

[0133] The original model can be trained with the above data set to obtain the target regression coefficient (for example, W and b in formula (1)). The original model can be a built, untrained logistic regression model. The logistic regression model can be obtained by replacing the regression coefficient in the original model with the target regression coefficient.

[0134] It should be noted that different features of the animation frame may correspond to different weights, that is, different features of the animation frame may have different effects on the output of the logistic regression model. For example, the weights of features such as frame rate, window width (WindowW), and window height (WindowH) may be greater than the weights of other features (e.g., GPU attributes, CPU attributes, animation effect triggering scenes, animation effect triggering events (e.g., click events), application animation effect attributes, touch attributes, etc.).

[0135] Through the intelligent decision-making algorithm, animation frames that are sensitive to the human eye and animation frames that are not sensitive to the human eye can be determined, so that different synthesis processes can be performed on these two different types of animation frames. For example, animation frames that are not sensitive to the human eye can be sent to the HWC for synthesis, and animation frames that are sensitive to the human eye can be sent to the GPU for synthesis. There is no need to send all animation frames to the GPU for synthesis, which can ensure the display effect and reduce the power consumption of electronic devices, and improve the performance of electronic devices in rounding animation frames when displaying animation special effects.

[0136] In some embodiments, in each Vsync cycle, an intelligent decision algorithm may be used to determine whether an animation frame rendered in the Vsync cycle is an animation frame that is sensitive to the human eye or an animation frame that is not sensitive to the human eye.

[0137] In some embodiments, the electronic device may save motion effect clipping parameters corresponding to different scenarios (e.g., scenarios for launching an application or exiting an application) and different animation frame features (e.g., different frame rates) determined according to an intelligent decision-making algorithm. The motion effect clipping parameters are used to indicate animation frames that are sensitive to the human eye and animation frames that are not sensitive to the human eye. If the features of the current scene and the current animation frame match the features of a scene and animation frame saved by the electronic device, the animation frames that are sensitive to the human eye and animation frames that are not sensitive to the human eye in the current scene are determined based on the motion effect clipping parameters corresponding to the features of the scene and animation frame.

[0138] If SurfaceFlinger determines that the animation frame to be synthesized is an animation frame to which the human eye is not sensitive, step 305 may be executed; if SurfaceFlinger determines that the animation frame to be synthesized is an animation frame to which the human eye is sensitive, step 306 may be executed.

[0139] 305. SurfaceFlinger sends the animation frames that are not sensitive to the human eye among the N animation frames to be synthesized (for example, M animation frames among the N animation frames) to the HWC.

[0140] 306. SurfaceFlinger sends the animation frames that are sensitive to the human eye among the N animation frames to be synthesized (for example, NM animation frames among the N animation frames) to the GPU.

[0141] 307. The HWC performs synthesis processing on the received animation frames to obtain a first animation frame to be displayed.

[0142] Each of the first animation frames to be displayed is a rectangular image, and the four vertices of the rectangular image are right angles.

[0143] For example, Figure 5As shown, HWC can synthesize animation frame 1, animation frame 2 and animation frame 3, and the top angle of each animation frame is a right angle. Among them, the top angles of the animation frame can include the upper left corner, the upper right corner, the lower left corner and the lower right corner.

[0144] 308. The GPU performs synthesis processing on the received animation frame to obtain a second animation frame to be displayed.

[0145] The GPU performs synthesis processing on the received animation frames and can perform rounding processing on the animation frames. Exemplarily, the GPU can perform rounding processing on each received animation frame according to the G1 curve or the G2 curve to obtain the second animation frame to be displayed. Each frame of the second animation frame to be displayed is a rounded image, and the four top corners of the rounded image are rounded.

[0146] For example, Figure 5 As shown, the GPU can synthesize animation frame 4, animation frame 5 and animation frame 6, and the top corner of each animation frame is rounded.

[0147] 309. The HWC sends the first animation frame to be displayed to the display screen, and the display screen displays the first animation frame to be displayed.

[0148] Exemplarily, the first animation frame to be displayed may include Figure 6 In the animation frames shown in (a)-(c), each animation frame is a rectangular image, and the four vertices of the rectangular image are right angles.

[0149] It is understandable that after the operation of launching the camera application is completed (the user leaves the hand (raises the hand), that is, the user's finger (or other position (e.g., elbow)) or input tool (e.g., stylus) is detected to leave the touch screen), the electronic device can display the animation effect of launching the camera application, and the animation effect of launching the camera application can include multiple animation frames. For example, Figure 6 As shown in (a) of FIG. 1 , 100 ms after the user releases the hand, the electronic device can display an animation frame 601, such as Figure 6 As shown in (b) of FIG. 2, 200 ms after the user releases the hand, the electronic device may display an animation frame 602, such as Figure 6 As shown in (c), 300ms after the user releases the hand, the electronic device can display animation frame 603. Among them, animation frames 601-603 are synthesized by HWC.

[0150] 310. The GPU sends the second animation frame to be displayed to the display screen, and the display screen displays the second animation frame to be displayed.

[0151] Exemplarily, the second animation frame to be displayed may include Figure 6 In the animation frames shown in (d)-(e), each animation frame is a rounded image, and the four top corners of the rounded image are rounded.

[0152] It is understandable that after the operation of launching the camera application is completed (the user leaves the hand (raises the hand), that is, the user's finger (or other position (e.g., elbow)) or input tool (e.g., stylus) is detected to leave the touch screen), the electronic device can display the animation effect of launching the camera application, and the animation effect of launching the camera application can include multiple animation frames. For example, Figure 6 As shown in (d) in FIG. 400 ms after the user releases the hand, the electronic device may display an animation frame 604, such as Figure 6 As shown in (e) in FIG. 500 ms after the user releases the hand, the electronic device may display an animation frame 605, such as Figure 6 As shown in (f) in FIG. 6, 600 ms after the user releases the hand, the electronic device can display animation frame 606. Among them, animation frame 604-animation frame 606 are synthesized by the GPU. Further, after the animation effect of starting the camera application is displayed, the interface of the camera application can be displayed. For example, Figure 4 As shown in (b) , the electronic device may display a shooting preview interface 403 .

[0153] The method provided in the embodiment of the present application takes into account human perception and may not perform rounding on animation frames that cannot be recognized by humans (i.e., animation frames that are not sensitive to the human eye), and perform rounding on animation frames that can be recognized by humans (i.e., animation frames that are sensitive to the human eye). That is, animation frames that cannot be recognized by humans can be synthesized by HWC, and animation frames that can be recognized by humans can be synthesized by GPU. Thereby, the problems of high power consumption, high load, and severe heat generation caused by the electronic device performing rounding on each animation frame (i.e., synthesizing each animation frame by GPU) can be avoided. On the basis of ensuring the display effect of animation special effects, the performance of electronic devices in displaying animation special effects can be improved.

[0154] like Figure 7 As shown, the embodiment of the present application provides an animation frame synthesis method, taking the scenario of exiting an application as an example, the process of animation frame synthesis is described, including:

[0155] 701. The desktop application receives an operation from a user to exit a camera application.

[0156] When the electronic device displays the first interface of the camera application, the desktop application may receive an operation of the user to exit the camera application (an example of the first operation). Exemplarily, the first interface may be a preview interface of the camera application. Figure 8 As shown in (a) in .

[0157] In some embodiments, the operation of exiting the camera application is used to exit the camera application and return to the desktop. For example, the user can exit the first application and return to the desktop by clicking the home button (a functional button for returning to the desktop). The home button may be a physical button or a virtual button, which is not limited in this application. Alternatively, the user can exit the first application and return to the desktop by clicking the return button (for example, clicking once, or clicking twice in a row). The return button is used to return to the previous interface. Alternatively, the user can exit the first application and return to the desktop through a first shortcut gesture operation. The first shortcut gesture operation can be any one of the following operations: an operation of sliding up a first distance from the bottom of the screen, an operation of sliding right from the left end of the screen, and an operation of sliding left from the right end of the screen. That is, the operation of exiting the camera application may include a single click of the home button by the user, a double click of the return button by the user, and a first shortcut gesture operation.

[0158] For example, Figure 8 As shown in (a) of FIG. 8 , the electronic device may receive an operation of a user sliding up a first distance from the bottom of the screen (operation of exiting the camera application) when displaying the preview interface (first interface) 801 of the camera application, and respond to the operation, such as Figure 8 As shown in (b) in FIG. 8 , the mobile phone may display the main interface 802 .

[0159] In other embodiments, the operation of exiting the camera application is used to exit the camera application and enter the multitasking interface. For example, the user can exit the camera application and enter the multitasking interface by clicking the multitasking key (background key). Among them, the background key can be used to view and clean up background programs of the electronic device. Alternatively, the user can exit the first application and enter the multitasking interface through a second shortcut gesture operation. The second shortcut gesture operation can, for example, be an operation of sliding up a second distance from the bottom of the screen. Among them, the second distance is different from the first distance. That is, the operation of exiting the camera application may include a single click of the background key by the user, and a second shortcut gesture operation.

[0160] Optionally, the operation of exiting the camera application may also be a voice operation, or other types of operations, which is not limited in this embodiment of the present application.

[0161] 702. The desktop application sends N animation frames to be synthesized to SurfaceFlinger.

[0162] 703. SurfaceFlinger determines whether the current scene is a scene that requires motion clipping.

[0163] The scene requiring motion effect cutting may include a scene of exiting an application, wherein the scene of exiting an application may include a scene of exiting an application and returning to the desktop, and a scene of exiting an application and entering a multitasking interface.

[0164] It should be understood that in the scenario of exiting an application, the electronic device can perform animation rendering and synthesis on the process of exiting the application, that is, the electronic device can display the animation effects of exiting the application.

[0165] The animation effect of exiting the application may include a plurality of continuously changing animation frames, and each animation frame may include a screenshot of the application window. The screenshot of the application window may refer to a screenshot of the window (first interface) displayed when the application is exited. Among the plurality of continuously changing animation frames included in the animation effect of exiting the application and returning to the desktop, the screenshot included in the previous animation frame is larger than the screenshot included in the next animation frame.

[0166] It can be understood that the operation of exiting the camera application is used to exit the camera application, so the current scene is the scene of exiting the application, that is, the current scene belongs to the scene that requires motion effect clipping, and step 703 can be executed.

[0167] 704. SurfaceFlinger determines animation frames that are sensitive to the human eye and animation frames that are not sensitive to the human eye among the N animation frames to be synthesized.

[0168] 705. SurfaceFlinger sends the animation frames that are not sensitive to the human eye among the N animation frames to be synthesized (for example, M animation frames among the N animation frames) to the HWC.

[0169] 706. SurfaceFlinger sends the animation frames that are sensitive to the human eye among the N animation frames to be synthesized (for example, NM animation frames among the N animation frames) to the GPU.

[0170] 707. The HWC performs synthesis processing on the received animation frames to obtain a first animation frame to be displayed.

[0171] Each of the first animation frames to be displayed is a rectangular image, and the four vertices of the rectangular image are right angles.

[0172] For example, Fig. 9 As shown, HWC can synthesize animation frame 4, animation frame 5 and animation frame 6, and the top angle of each animation frame is a right angle. Among them, the top angles of the animation frame can include the upper left corner, the upper right corner, the lower left corner and the lower right corner.

[0173] 708. The GPU performs synthesis processing on the received animation frame to obtain a second animation frame to be displayed.

[0174] The GPU performs synthesis processing on the received animation frames and can perform rounding processing on the animation frames. Exemplarily, the GPU can perform rounding processing on each received animation frame according to the G1 curve or the G2 curve to obtain the second animation frame to be displayed. Each frame of the second animation frame to be displayed is a rounded image, and the four top corners of the rounded image are rounded.

[0175] For example, Fig. 9 As shown, the GPU can synthesize animation frame 1, animation frame 2, and animation frame 3, and the top corner of each animation frame is rounded.

[0176] 709. The HWC sends the first animation frame to be displayed to the display screen, and the display screen displays the first animation frame to be displayed.

[0177] Exemplarily, the first animation frame to be displayed may include Fig.10 In the animation frames shown in (d)-(e), each animation frame is a rectangular image, and the four vertices of the rectangular image are right angles.

[0178] It is understandable that after the operation of exiting the camera application is completed (the user leaves the camera application (raises the hand), that is, the user's finger (or other position (e.g., elbow)) or input tool (e.g., stylus) is detected to leave the touch screen), the electronic device can display an animation effect of exiting the camera application, and the animation effect of exiting the camera application can include multiple animation frames. For example, Fig.10 As shown in (d) in FIG. 400 ms after the user releases the hand, the electronic device may display animation frame 1004, such as Fig.10 As shown in (e) in FIG. 500 ms after the user releases the hand, the electronic device may display animation frame 1005, such as Fig.10 As shown in (f) in FIG. 600 ms after the user releases the hand, the electronic device can display animation frame 1006. Among them, animation frame 1004-animation frame 1006 are synthesized by HWC. Further, after the animation effect of exiting the camera application is displayed, the interface of the camera application can be displayed, for example, Figure 8 As shown in (b) , the electronic device can display the desktop 802.

[0179] 710. The GPU sends the second animation frame to be displayed to the display screen, and the display screen displays the second animation frame to be displayed.

[0180] Exemplarily, the first animation frame to be displayed may include Fig.10 In the animation frames shown in (a)-(c), each animation frame is a rounded image, and the four top corners of the rounded image are rounded.

[0181] It is understandable that after the operation of exiting the camera application is completed (the user leaves the camera application (raises the hand), that is, the user's finger (or other position (e.g., elbow)) or input tool (e.g., stylus) is detected to leave the touch screen), the electronic device can display an animation effect of exiting the camera application, and the animation effect of exiting the camera application can include multiple animation frames. For example, Fig.10 As shown in (a) of FIG. 1 , 100 ms after the user releases the hand, the electronic device can display the animation frame 1001, such as Fig.10 As shown in (b) of FIG. 2, 200 ms after the user releases the hand, the electronic device may display an animation frame 1002, such as Fig.10 As shown in (c), 300ms after the user releases the hand, the electronic device can display animation frame 1003. Among them, animation frames 1001 to 1003 are synthesized by the GPU.

[0182] The method provided in the embodiment of the present application takes into account human perception and may not perform rounding on animation frames that are difficult / unrecognizable to humans (i.e., users) (i.e., animation frames that are not sensitive to the human eye), and perform rounding on animation frames that are recognizable to humans (i.e., animation frames that are sensitive to the human eye). That is, animation frames that are unrecognizable to humans may be synthesized by HWC, and animation frames that are recognizable to humans may be synthesized by GPU. Thereby, the problems of high power consumption, high load, and severe heat generation caused by the electronic device performing rounding on each animation frame (i.e., synthesizing each animation frame by GPU) can be avoided. On the basis of ensuring the display effect of animation special effects, the performance of electronic devices in displaying animation special effects can be improved.

[0183] The present application also provides a chip system, such as Fig.11 As shown, the chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 and the interface circuit 1102 can be interconnected through a line. For example, the interface circuit 1102 can be used to receive signals from other devices (for example, a memory of an electronic device). For another example, the interface circuit 1102 can be used to send signals to other devices (for example, the processor 1101).

[0184] For example, the interface circuit 1102 can read the instructions stored in the memory of the electronic device and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the electronic device (such as Figure 1A The electronic device 100 shown in the figure performs each step in the above embodiment.

[0185] Of course, the chip system may also include other discrete devices, which is not specifically limited in the embodiments of the present application.

[0186] The present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are in an electronic device (such as Figure 1A When the method is executed on the electronic device 100 shown in the figure, the electronic device 100 executes each function or step executed by the electronic device in the above method embodiment.

[0187] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the electronic device in the above method embodiment.

[0188] An embodiment of the present application also provides a device, which can be divided into different logical units or modules according to functions, and each unit or module performs a different function, so that the device performs each function or step performed by the electronic device in the above method embodiment.

[0189] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0190] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0191] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0192] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0193] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0194] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for synthesizing animation frames, characterized in that: Applied to an electronic device, the electronic device includes a graphics processor GPU and a hardware hybrid renderer HWC, the method includes: Receive a first operation, where the first operation is used to trigger the electronic device to display an animation special effect, where the animation special effect includes N animation frames, where N is an integer greater than 0; The electronic device synthesizes M animation frames of the N animation frames through the HWC, and synthesizes NM animation frames of the N animation frames through the GPU, and the window height of each animation frame of the M animation frames is less than the window height of each animation frame of the NM animation frames; Wherein, M is an integer greater than 0 and less than N, each of the M animation frames is a rectangular image, and each of the NM animation frames is a rounded image.

2. The method according to claim 1, characterized in that At the first frame rate, a window height of each of the M animation frames is less than a first threshold, and a window height of each of the NM animation frames is greater than or equal to the first threshold; At the second frame rate, a window height of each of the M animation frames is less than a second threshold, and a window height of each of the NM animation frames is greater than or equal to the second threshold; The first frame rate is greater than the second frame rate, and the first threshold is greater than the second threshold.

3. The method according to claim 1 or 2, wherein the electronic device installs a first application, characterized in that: The first operation is used to start the first application, or The first operation is used to exit the first application.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Acquire the characteristics of each of the N animation frames, wherein the characteristics of each of the animation frames include a frame rate and a window height; Normalizing the features of each animation frame; The normalized data is input into the classification model to obtain an output value, where the output value is used to indicate whether each animation frame belongs to the M animation frames or the NM animation frames.

5. The method according to claim 4, characterized in that The classification model is any one of a support vector machine SVM model, a neural network model, a random separation model, a linear regression model or a logistic regression model.

6. The method according to claim 4 or 5, characterized in that: The classification model is a logistic regression model, and the logistic regression model is as follows: Among them, y indicates whether the current animation frame is an animation frame that is sensitive to the human eye, the current animation frame is the animation frame currently input into the logistic regression model, and the current animation frame is any one of the N animation frames. x represents a feature vector set, and the feature vector set includes the encoding value of at least one feature of the current animation frame, and the at least one feature includes a frame rate and a window height. W represents a weight vector set, and the weight vector set includes weights corresponding to at least one feature of the current animation frame. T represents a transpose operation, and b represents a bias.

7. The method according to claim 6, characterized in that The method further comprises: Constructing a data set, wherein the data set includes a feature vector set of a plurality of labeled animation frame samples, wherein the feature vector set includes a frame rate and a window height; The logistic regression model is obtained by training based on the data set.

8. The method according to any one of claims 1 to 7, characterized in that: The electronic device further includes a desktop application and an image synthesis system, and the method further includes: The desktop application draws and renders the N animation frames included in the animation special effect respectively to obtain N animation frames to be synthesized; The desktop application sends the N animation frames to be synthesized to the image synthesis system; The image synthesis system determines animation frames that are sensitive to human eyes and animation frames that are not sensitive to human eyes among the N animation frames to be synthesized, wherein the animation frames that are sensitive to human eyes belong to the NM animation frames, and the animation frames that are not sensitive to human eyes belong to the M animation frames; The image synthesis system sends the animation frames to which the human eye is not sensitive to the HWC, and sends the animation frames to which the human eye is sensitive to the GPU.

9. The method according to claim 8, characterized in that The electronic device further includes a display screen, and the method further includes: The HWC performs synthesis processing on the received animation frame to obtain a first animation frame to be displayed; The GPU performs synthesis processing on the received animation frame to obtain a second animation frame to be displayed; The HWC sends the first animation frame to be displayed to the display screen, and the display screen displays the first animation frame to be displayed; The GPU sends the second animation frame to be displayed to the display screen, and the display screen displays the second animation frame to be displayed.

10. An electronic device, characterized in that: The electronic device comprises a processor, and the processor is used to run a computer program stored in a memory, so that the electronic device implements the method according to any one of claims 1 to 9.

11. A chip system, characterized in that: The method comprises a processor coupled to a memory, wherein the processor executes a computer program stored in the memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a processor, the method according to any one of claims 1 to 9 is implemented.

Images