Image frame drawing method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides an image frame drawing method, electronic equipment and a computer readable storage medium, in the image frame drawing method, after a sliding operation of a user on a current display interface is detected, when a current image frame is drawn, the electronic equipment does not depend on a Vsync signal any more, but draws the current image frame according to a first moment and a current frame interval; and determining the drawing start time of the current image frame to be drawn, and then drawing the current image frame according to the drawing start time. The time interval between the drawing starting time of the current image frame determined according to the first time and the current frame interval is relatively uniform, so that the sliding displacement and the sliding speed calculated according to the drawing starting time are relatively smooth, and the sliding of an interface can be smoother and is not easy to shake.

Description

Technical Field

[0001] This application relates to the field of smart terminal technology, and in particular to a method for drawing image frames, an electronic device, and a computer-readable storage medium. Background Technology

[0002] If a user swipes on an app's interface while browsing it on an electronic device, the app's interface will scroll upwards in response. With the widespread adoption of high frame rates and the increasing complexity of applications, the probability of screen jitter during this upward scrolling process has increased, negatively impacting the user experience. Summary of the Invention

[0003] This application provides a method for drawing image frames, an electronic device, and a computer-readable storage medium to make the sliding of the current display interface smoother and reduce jitter during the sliding process after the user performs a sliding operation on the current display interface of the electronic device.

[0004] In a first aspect, embodiments of this application provide a method for drawing an image frame, comprising: after detecting a user's swiping operation on a current display interface, determining the start time for drawing the current image frame to be drawn based on a first moment and the current frame interval; wherein the current display interface is the interface of an application currently running on the electronic device, and the first moment is the start time for drawing the previous image frame; and drawing the current image frame based on the start time for drawing.

[0005] In existing related technologies, image frame rendering relies on the Vsync signal. Although the period of the Vsync signal is fixed, the time between the arrival of each Vsync signal and the rendering of the image frame is uneven. Therefore, the time interval between two adjacent Vsync times in existing related technologies is uneven, causing jitter when the interface scrolls. In the image frame rendering method described above, after detecting the user's scrolling operation on the current display interface, when rendering the current image frame, the electronic device 100 no longer relies on the Vsync signal. Instead, it determines the start time of rendering the current image frame based on the first moment and the current frame interval. Then, based on the above-mentioned start time, it uses the scrolling curve to determine the scrolling displacement and scrolling speed of the current image frame to be rendered. Finally, the current image frame is rendered based on the above-mentioned scrolling displacement and scrolling speed. Since the time interval between the start time of rendering the current image frame determined based on the first moment and the current frame interval is relatively uniform, the scrolling displacement and scrolling speed calculated based on this start time are smoother, thus making the interface scrolling smoother and less prone to jitter.

[0006] In one possible implementation, determining the start time of drawing the current image frame to be drawn based on the first time and the current frame interval includes: calculating the sum of the first time and the frame interval to obtain a second time; if the second time is greater than or equal to the end time of drawing the previous image frame, then the second time is used as the start time of drawing the current image frame; if the second time is less than the end time of drawing the previous image frame, then the end time of drawing the previous image frame is used as the start time of drawing the current image frame.

[0007] In one possible implementation, determining the start time of drawing the current image frame to be drawn based on the first moment and the current frame interval includes: within a first time interval after the sliding operation is detected, after the previous image frame has been drawn, determining the start time of drawing the current image frame to be drawn based on the first moment and the current frame interval; wherein, the start time of the first time interval is the moment when the sliding operation is detected, and the duration of the first time interval is a predetermined duration.

[0008] In this implementation, within the first time interval (the time period when the user's swipe operation is detected) during the swipe-off period, as long as the previous image frame is completed, the electronic device starts drawing the current image frame and obtains the start time of drawing the current image frame, thereby avoiding frame loss due to the long time required to draw a certain image frame.

[0009] In one possible implementation, determining the start time of drawing the current image frame to be drawn based on the first moment and the current frame interval includes: within the second time interval, if the size of the cached image frame is less than a first threshold, then after the previous image frame is drawn, determining the start time of drawing the current image frame to be drawn based on the first moment and the current frame interval; wherein, the second time interval is after the first time interval, the start time of the second time interval is the end time of the first time interval, and the end time of the second time interval is the sliding end time.

[0010] This implementation can adjust the frame insertion timing based on the number of cached image frames, which avoids frame dropping and does not increase the burden on the user interface (UI) thread.

[0011] In one possible implementation, determining the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval includes: within the second time interval, if the size of the cached image frame is greater than or equal to a first threshold, then in response to the vertical synchronization signal of the electronic device, determining the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval.

[0012] In one possible implementation, determining the start time of drawing the current image frame to be drawn based on the first moment and the current frame interval includes: in response to the vertical synchronization signal of the electronic device, obtaining the start time of drawing the current image frame to be drawn.

[0013] In one possible implementation, drawing the current image frame according to the drawing start time includes: determining the sliding displacement and sliding speed of the current image frame to be drawn using the sliding curve of the application according to the drawing start time; and drawing the current image frame according to the sliding displacement and the sliding speed.

[0014] Secondly, embodiments of this application provide an electronic device, including: one or more processors; a memory; multiple application programs; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method provided in the first aspect.

[0015] It should be understood that the second aspect of the embodiments of this application is consistent with the technical solution of the first aspect, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, so they will not be described again.

[0016] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method provided in the first aspect.

[0017] Fourthly, this application provides a computer program that, when executed by a computer, performs the method provided in the first aspect.

[0018] In one possible design, the program in the fourth aspect can be stored wholly or partially on a storage medium packaged with the processor, or it can be stored wholly or partially on a memory not packaged with the processor. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application;

[0020] Figure 2 A software structure block diagram of an electronic device 100 provided in one embodiment of this application;

[0021] Figure 3 This is a schematic diagram illustrating the uneven time intervals between Vsynctimes.

[0022] Figure 4A flowchart illustrating a method for drawing an image frame according to an embodiment of this application;

[0023] Figures 5(a) to 5(b) This is a schematic diagram illustrating a user's swiping operation according to one embodiment of this application;

[0024] Figure 5(c) is a schematic diagram of a user performing a swiping operation according to another embodiment of this application;

[0025] Figure 6 A flowchart illustrating a method for drawing an image frame according to another embodiment of this application;

[0026] Figure 7 A flowchart illustrating a method for drawing an image frame as provided in another embodiment of this application;

[0027] Figure 8 A diagram illustrating how frame dropping occurs when the time required to render a certain image frame exceeds one Vsync cycle;

[0028] Figure 9 A schematic diagram of a drawn image frame provided in one embodiment of this application;

[0029] Figure 10 A flowchart illustrating a method for drawing an image frame as provided in another embodiment of this application;

[0030] Figure 11 A schematic diagram illustrating rapid cache replenishment according to one embodiment of this application;

[0031] Figure 12 A schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0032] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0033] To address the issue of screen jitter that can occur when scrolling up on an application's interface, existing technologies offer a frame interpolation solution based on a system control (OverScroller). By using frame interpolation, an extra frame can be cached, thereby reducing the probability of frame loss and consequently reducing screen jitter.

[0034] However, this solution can only be applied to pages that use system controls (OverScroller), thus limiting its applicability.

[0035] To address the above issues, this application provides a method for drawing image frames, which can make the swiping of the current display interface smoother after a user performs a swipe operation on the electronic device's current display interface, reducing jitter during the swipe process. In this application embodiment, jitter during the swipe process refers to uneven swipe speed, with the swipe speed varying from fast to slow.

[0036] The image frame rendering method provided in this application embodiment can be applied to electronic devices, such as smartphones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs). This application embodiment does not impose any restrictions on the specific type of electronic device.

[0037] For example, Figure 1 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application, as shown below. Figure 1 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0038] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0039] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0040] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0041] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0042] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0043] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (DCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0044] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0045] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0046] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0047] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0048] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0049] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0050] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0051] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via a USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device 100 via the power management module 141.

[0052] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0053] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0054] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0055] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0056] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0057] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0058] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-CDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0059] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0060] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0061] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0062] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0063] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0064] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0065] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0066] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0067] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0068] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0069] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0070] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0071] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0072] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0073] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0074] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0075] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0076] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0077] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0078] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0079] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0080] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0081] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0082] Figure 2 This is a software structure block diagram of an electronic device 100 provided in one embodiment of this application.

[0083] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer (HAL), and the kernel layer.

[0084] The application layer can include a series of application packages, such as Figure 2 As shown, the application package may include applications such as camera, gallery, email, call, map, navigation, WLAN, Bluetooth, music playback, video, and SMS. These applications may include system applications pre-installed on the electronic device 100, such as camera, gallery, call, video, SMS, and Bluetooth applications, as well as applications installed by the user on the electronic device 100, such as music playback, map, and navigation applications.

[0085] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes a set of predefined functions.

[0086] like Figure 2 As shown, the application framework layer can include managers, such as package managers, window managers, and resource managers. The application framework layer can also include components such as widgets, iAware, and view systems.

[0087] The window manager is used to manage window programs. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture screenshots, among other things.

[0088] The package manager is responsible for managing all application packages (usually APK files) installed on the electronic device 100. The package manager is an important component of the Android system architecture, and its main functions include: application installation and uninstallation, application permission management, information retrieval of applications installed on the electronic device 100, component management, intent parsing, dynamic permission management, system updates, and / or application verification.

[0089] The resource manager is used to manage and access the application's resource files, which include, but are not limited to, images, layout files, strings, color definitions, sizes, styles, animations, and audio files.

[0090] Widgets can display information or provide simple interactive functions directly on a user's home screen without requiring the user to open the full application. Widgets make the home screen of electronic devices more practical and personalized. They can display various real-time information, such as weather forecasts, news headlines, stock quotes, music controls, calendar events, battery status, and / or time displays. They can also provide quick actions, such as turning Wi-Fi on / off, adjusting screen brightness, and launching frequently used applications.

[0091] A view system includes visual controls, such as controls for displaying text and / or controls for displaying images. View systems can be used to construct the display interface of an application, which can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images.

[0092] iAware components can enhance user experience and device security by learning user habits and intelligently optimizing system resource allocation, while protecting user privacy and data security.

[0093] In this embodiment, when a user is using an application (e.g., a music player application), in order to quickly browse the content on the interface, the user performs a swipe operation on the current display interface of the electronic device 100. In response to the swipe operation, the widget component and the view system can call the iAware component to complete calculations related to the control, such as obtaining the drawing start time of the current image frame, determining the sliding displacement and sliding speed of the current image frame, etc.

[0094] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0095] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0096] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0097] System libraries can include multiple functional modules. For example: libgui, libhwui, surface composition (surfaceflinger), 3D graphics processing libraries (e.g., OpenGL ES), input, etc.

[0098] libgui is responsible for the low-level implementation and rendering of the graphical user interface (GUI). libgui contains components responsible for drawing application interface elements, such as view layout, handling drawing commands, and scheduling interface updates. libgui works in conjunction with surfaceflinger to ensure the correct composition and display of screen content.

[0099] libhwui is responsible for implementing hardware-accelerated 2D graphics rendering. When Android enables hardware acceleration, libhwui utilizes the GPU to accelerate the graphics drawing process. This means that graphics rendering tasks originally handled by the CPU are moved to the GPU, significantly improving drawing efficiency and application smoothness.

[0100] SurfaceFlinger acts as the display engine, responsible for managing and compositing all graphical interfaces to ensure that user interfaces and application content can be displayed correctly and efficiently on the screen of electronic device 100.

[0101] In this embodiment, after the widget component and the view system call the iAware component to obtain the start time of the current image frame's drawing, determine the sliding displacement and sliding speed of the current image frame, libgui and libhwui can draw the current image frame at the start time of the current image frame's drawing based on the sliding displacement and sliding speed of the current image frame. Then, surfaceflinger composites the drawn current image frame and sends it for display. In this way, the current image frame can be displayed on the screen of the electronic device 100.

[0102] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0103] A 2D graphics engine is a graphics engine for 2D drawing.

[0104] like Figure 2 As shown, HAL sits above the kernel layer and serves as the interface between the operating system and hardware devices. HAL's primary responsibility is to hide the specific implementation details of the hardware, providing a unified interface for upper-layer applications. This allows the Android system to run on different hardware platforms without requiring extensive modifications for each hardware, thus accelerating software development and porting.

[0105] The kernel layer is the layer between hardware and software. At a minimum, the kernel layer includes Bluetooth drivers, camera drivers, audio drivers, sensor drivers, and power management.

[0106] For ease of understanding, the following embodiments of this application will be described using the following methods: Figure 1 and Figure 2 Taking the electronic device with the structure shown as an example, and in conjunction with the accompanying drawings and application scenarios, the image frame drawing method provided in this application embodiment will be specifically described.

[0107] Figure 3 This is a schematic diagram illustrating the uneven time intervals between Vsynctimes, as shown below. Figure 3 As shown, the CPU in the electronic device 100 responds to the vertical synchronization (Vsync) signal, obtains the start time of drawing the image frame, i.e., the vertical synchronization time (Vsynctime), and then draws the image frame using a sliding curve based on the Vsynctime of the image frame. Figure 3In the diagram, Vsynctime1 is the start time for drawing image frame 1, Vsynctime2 is the start time for drawing image frame 2, Vsynctime3 is the start time for drawing image frame 3, Vsynctime4 is the start time for drawing image frame 4, and Vsynctime5 is the start time for drawing image frame 5.

[0108] Specifically, taking image frame 1 as an example, the CPU responds to the Vsync signal, obtains the Vsynctime1 of image frame 1, and then uses a sliding curve to draw image frame 1 based on Vsynctime1. The drawing process of image frames 2 to 5 is the same as that of image frame 1, and will not be described again here.

[0109] from Figure 3 As can be seen, the time interval between two adjacent Vsynctimes is uneven. For example, the time interval between Vsynctime1 and Vsynctime2 is small, while the time interval between Vsynctime2 and Vsynctime3 is large. This results in uneven screen scrolling speed, sometimes fast and sometimes slow, causing jitter.

[0110] Based on the above problems, the image frame drawing method provided in this application embodiment detects a user's sliding operation on the current display interface, obtains the drawing start time of the current image frame to be drawn through an algorithm, and then draws the current image frame using a sliding curve based on this drawing start time. Since the drawing start time obtained by the algorithm has a more uniform time interval than Vsynctime, the sliding displacement and sliding speed calculated based on this drawing start time are smoother, thereby making the interface sliding smoother and less prone to jitter. The image frame drawing method provided in this application embodiment can be applied to scenarios of display interface sliding in electronic device 100 (such as... Figures 5(a) to 5(b) As shown in Figure 5(c), it can also be applied to scenarios where the interface opened by third-party controls (such as webview control and / or flutter control) is slidable in the application (as shown in Figure 5(c)).

[0111] The following is a detailed description of the image frame drawing method provided in the embodiments of this application.

[0112] Figure 4 A flowchart of an image frame drawing method provided in one embodiment of this application is shown below. Figure 4 As shown, the method for drawing the above image frame may include:

[0113] Step 401: After detecting the user's swipe operation on the current display interface, the electronic device 100 determines the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval.

[0114] The current display interface is the interface of the application currently running on the electronic device 100; the first moment is the start moment of drawing the previous image frame.

[0115] For example, in one instance, referring to Figure 5(a), after the user clicks the "Music" icon 51 displayed on the screen of the electronic device 100, the electronic device 100 runs the "Music" application and displays the interface shown in Figure 5(b). In order to quickly browse the content in the interface shown in Figure 5(b), the user performs a swipe operation on the interface shown in Figure 5(b) (as shown in 52). After detecting the above swipe operation, the "Music" application queries the operating system of the electronic device 100 for the start time of drawing the current image frame. The operating system of the electronic device 100 can determine the start time of drawing the current image frame to be drawn, and then return the start time of drawing the current image frame to the "Music" application. Figures 5(a) to 5(b) This is a schematic diagram illustrating a user's swiping operation according to one embodiment of this application. It should be noted that... Figures 5(a) to 5(b) The "music" application in the example shown can be a music playback application that comes with the electronic device 100 or a music playback application that the user installs on the electronic device 100. This embodiment does not limit this.

[0116] In another example, referring to Figure 5(c), a user chats with friend A using an instant messaging application in electronic device 100. The application currently running on electronic device 100 is an instant messaging application, as shown in the left image of Figure 5(c). Friend A sends the user a link introducing the Beijing Zoo. The user clicks the link in the message viewing interface of friend A. Electronic device 100 can display the content of the link in the interface of the instant messaging application through a third-party control (e.g., a webview control) embedded in the instant messaging application, as shown in the right image of Figure 5(c). To quickly browse the content in the interface shown in the right image of Figure 5(c), the user performs a swipe operation on the interface shown in the right image of Figure 5(c) (as shown in 53). After detecting the swipe operation, the instant messaging application queries the operating system of electronic device 100 for the start time of drawing the current image frame. The operating system of electronic device 100 can determine the start time of drawing the current image frame to be drawn, and then return the start time of drawing the current image frame to the instant messaging application. Figure 5(c) is a schematic diagram of a user performing a swipe operation according to another embodiment of this application.

[0117] In this embodiment, the electronic device 100 determines the start time of drawing the current image frame to be drawn based on the first time and the current frame interval by: calculating the sum of the first time and the frame interval to obtain the second time; if the second time is greater than or equal to the end time of drawing the previous image frame, then the second time is used as the start time of drawing the current image frame; if the second time is less than the end time of drawing the previous image frame, then the end time of drawing the previous image frame is used as the start time of drawing the current image frame.

[0118] Additionally, it should be noted that if the current image frame to be drawn is the first frame (i.e., the first image frame), then the start time of drawing the first frame can be determined to be 0.

[0119] The frame interval refers to the time interval between two adjacent image frames. The frame interval is inversely related to the frame rate, which is the number of frames per second (FPS). For example, assuming a frame rate of 120 frames per second, the frame interval is 8.33 milliseconds. The following explanation uses a frame interval of 8.33 milliseconds as an example to illustrate the process of determining the start time of rendering the current image frame.

[0120] When the current image frame to be drawn is frame 1, as mentioned above, the start time of drawing frame 1 is 0. Assuming that drawing frame 1 takes 12 milliseconds, then the end time of drawing frame 1 is 12 milliseconds. Next, the drawing of frame 2 begins. The previous frame of frame 2 is frame 1. Therefore, the sum of the start time of drawing frame 1 and the frame interval is calculated first: 0 + 8.33 milliseconds = 8.33 milliseconds, that is, the second time is 8.33 milliseconds. Then, the second time (8.33 milliseconds) is compared with the end time of drawing frame 1 (12 milliseconds). Since 8.33 milliseconds < 12 milliseconds, 12 milliseconds is used as the start time of drawing frame 2.

[0121] Assuming it takes 10 milliseconds to draw the second frame, the end time of drawing the second frame is 12 milliseconds + 10 milliseconds = 22 milliseconds. Next, we start drawing the third frame. The previous frame of the third frame is the second frame. Therefore, we first calculate the sum of the start time of drawing the second frame and the frame interval, 12 + 8.33 milliseconds = 20.33 milliseconds, that is, the second time is 20.33 milliseconds. Then we compare the second time (20.33 milliseconds) with the end time of drawing the second frame (22 milliseconds). Since 20.33 milliseconds < 22 milliseconds, we take 22 milliseconds as the start time of drawing the third frame.

[0122] Assuming it takes 6 milliseconds to draw the 3rd frame, then the end time of drawing the 3rd frame is 22 milliseconds + 6 milliseconds = 28 milliseconds. Next, we start drawing the 4th frame. The previous frame of the 4th frame is the 3rd frame. Therefore, we first calculate the sum of the start time of drawing the 3rd frame and the frame interval, 22 + 8.33 milliseconds = 30.33 milliseconds, that is, the second time is 30.33 milliseconds. Then we compare the second time (30.33 milliseconds) with the end time of drawing the 3rd frame (28 milliseconds). Since 30.33 milliseconds > 28 milliseconds, we take 30.33 milliseconds as the start time of drawing the 4th frame.

[0123] Similarly, the method for determining the start time of drawing subsequent image frames is similar to the above process, and will not be repeated here.

[0124] Step 402: The electronic device 100 draws the current image frame according to the above-mentioned drawing start time.

[0125] Specifically, the electronic device 100 draws the current image frame based on the aforementioned drawing start time by: determining the sliding displacement and sliding speed of the current image frame to be drawn using the sliding curve applied in the aforementioned application, and then drawing the current image frame based on the aforementioned sliding displacement and sliding speed. In a specific implementation, 100 sampling points can be pre-set on the aforementioned application's sliding curve, and then the drawing start time of the current image frame can be mapped to these 100 sampling points. Then, at the sampling points mapped from the aforementioned drawing start time, the slope and height of the sliding curve are calculated. The height of the sliding curve can be the distance between the sampling points mapped from the aforementioned drawing start time and the sampling points mapped from the sliding start time; the slope of the sliding curve is the sliding speed of the current image frame, and the height of the sliding curve is the sliding distance of the current image frame. After determining the sliding displacement and sliding speed of the current image frame, the electronic device 100 can draw the current image frame based on the aforementioned sliding displacement and sliding speed.

[0126] In existing related technologies, image frame rendering relies on the Vsync signal. Although the period of the Vsync signal is fixed, the time between the arrival of each Vsync signal and the rendering of the image frame is uneven. Therefore, the time interval between two adjacent Vsync times in existing related technologies is uneven, causing jitter when the interface scrolls. In the image frame rendering method described above, after detecting the user's scrolling operation on the current display interface, when rendering the current image frame, the electronic device 100 no longer relies on the Vsync signal. Instead, it determines the start time of rendering the current image frame based on the first moment and the current frame interval. Then, based on the above-mentioned start time, it uses the scrolling curve to determine the scrolling displacement and scrolling speed of the current image frame to be rendered. Finally, the current image frame is rendered based on the above-mentioned scrolling displacement and scrolling speed. Since the time interval between the start time of rendering the current image frame determined based on the first moment and the current frame interval is relatively uniform, the scrolling displacement and scrolling speed calculated based on this start time are smoother, thus making the interface scrolling smoother and less prone to jitter.

[0127] Figure 6 A flowchart of a method for drawing an image frame provided in another embodiment of this application is shown below. Figure 6 As shown, this application Figure 4 In the illustrated embodiment, step 401 may include:

[0128] Step 601: After detecting the user's swipe operation on the current display interface, in response to the Vsync signal of the electronic device 100, the electronic device 100 determines the start time of drawing the current image frame to be drawn based on the first moment and the current frame interval.

[0129] In other words, in this embodiment, the rendering of the current image frame is triggered by the Vsync signal. Only after the Vsync signal arrives will the electronic device 100 determine the start time for rendering the current image frame. The method by which the electronic device 100 determines the start time for rendering in this step can be found in this application. Figure 4 The description of step 401 in the illustrated embodiment will not be repeated here.

[0130] Figure 7 A flowchart of an image frame drawing method provided in another embodiment of this application is shown below. Figure 7 As shown, this application Figure 4 In the illustrated embodiment, step 401 may include:

[0131] Step 701: In response to the user's swiping operation on the current display interface, within the first time interval after the swiping operation is detected, after the previous image frame is drawn, the electronic device 100 determines the start time of drawing the current image frame to be drawn based on the first time interval and the current frame interval.

[0132] In this embodiment, the start time of the first time interval is the moment the sliding operation is detected, and the duration of the first time interval is a predetermined duration. The length of the predetermined duration can be set by the implementation itself; this embodiment does not limit the length of the predetermined duration. For example, the predetermined duration can be 300 milliseconds. In this embodiment, the first time interval can also be referred to as the sliding-off period.

[0133] Step 702: Within the second time interval, if the size of the cached image frame is less than the first threshold, then after the previous image frame is drawn, the electronic device 100 determines the start time for drawing the current image frame to be drawn based on the first time and the current frame interval; if the size of the cached image frame is greater than or equal to the first threshold, then the electronic device 100, in response to the Vsync signal of the electronic device 100, determines the start time for drawing the current image frame to be drawn based on the first time and the current frame interval.

[0134] In this embodiment, the second time interval follows the first time interval, with the start time of the second time interval being the end time of the first time interval, and the end time of the second time interval being the end time of the slide.

[0135] In this step, the aforementioned cached image frames refer to image frames that have been drawn and are waiting to be rendered, stored in the buffer. The size of the aforementioned first threshold can be set by the specific implementation; this embodiment does not limit the size of the aforementioned first threshold.

[0136] Similarly, in this embodiment, the method by which the electronic device 100 determines the start time of drawing can refer to this application. Figure 4 The description of step 401 in the illustrated embodiment will not be repeated here.

[0137] In existing related technologies, the Vsync period is determined by the frame rate, and one image frame is drawn in each Vsync period. If the time taken to draw a certain image frame exceeds one Vsync period, frame dropping may occur, causing jitter when scrolling the interface, such as... Figure 8 As shown, Figure 8 This diagram illustrates a frame drop caused by the time taken to render a certain image frame exceeding one Vsync cycle.

[0138] Figure 8In response to the Vsync signal at time T1, the CPU in the electronic device 100 draws image frame 1. After image frame 1 is drawn, the GPU in the electronic device 100 renders image frame 1. After image frame 1 is rendered, in response to the Vsync signal at time T3, the electronic device 100 displays image frame 1.

[0139] In response to the Vsync signal at time T2, the CPU in electronic device 100 draws image frame 2. After image frame 2 is drawn, the GPU in electronic device 100 renders image frame 2. After image frame 2 is rendered, in response to the Vsync signal at time T4, electronic device 100 displays image frame 2.

[0140] In response to the Vsync signal at time T3, the CPU in electronic device 100 draws image frame 3 from... Figure 8 It can be seen that image frame 3 was not completed within the Vsync cycle T3-T4; the rendering of image frame 3 occupied two Vsync cycles (T3-T4 and T4-T5). After image frame 3 was completed, the GPU in the electronic device 100 rendered image frame 3. Figure 8 As can be seen, when the Vsync signal arrives at time T5, image frame 3 has not yet been rendered. Therefore, during the Vsync cycle from T5 to T6, electronic device 100 has no image frames to display, resulting in frame loss and causing jitter when scrolling. After image frame 3 is rendered, electronic device 100 displays image frame 3 in response to the Vsync signal at time T6.

[0141] To address the aforementioned issues, in this embodiment, the rendering of the current image frame no longer relies on the Vsync signal for triggering. Within the first time interval after detecting a user's swipe operation (the swipe-off period), as soon as the previous image frame is completed, the electronic device 100 begins rendering the current image frame, obtaining the start time of the current image frame's rendering. This avoids frame loss due to prolonged rendering of a particular image frame. Figure 9 As shown, Figure 9 This is a schematic diagram of drawing an image frame according to an embodiment of this application.

[0142] Figure 9 In response to the Vsync signal at time T1, the CPU in the electronic device 100 draws image frame 1. After image frame 1 is drawn, the CPU draws image frame 2. After image frame 2 is drawn, the CPU draws image frame 3. After image frame 3 is drawn, image frame 4 is drawn.

[0143] After image frame 1 is drawn, the GPU in the electronic device 100 renders image frame 1; after image frame 2 is drawn, the GPU renders image frame 2; after image frame 3 is drawn, the GPU renders image frame 3; after image frame 4 is drawn, the GPU renders image frame 4. Figure 9 It can be seen that the rendering of image frame 1 is parallel to the drawing of image frame 2, the rendering of image frame 2 is parallel to the drawing of image frame 3, and the rendering of image frame 3 is parallel to the drawing of image frame 4.

[0144] Thus, after the Vsync signal arrives at time T3, electronic device 100 can display image frame 1; after the Vsync signal arrives at time T4, electronic device 100 can display image frame 2; after the Vsync signal arrives at time T5, electronic device 100 can display image frame 3; and after the Vsync signal arrives at time T6, electronic device 100 can display image frame 4.

[0145] from Figure 9 It can be seen that although the rendering of image frame 3 still takes a relatively long time, exceeding one Vsync cycle, because the CPU immediately begins rendering image frame 2 after image frame 1 is completed, without waiting for the Vsync signal, the rendering of image frame 3 is also accelerated. Thus, in Figure 9 In the example shown, when the Vsync signal arrives at time T4, the drawn image frames stored in the buffer are image frame 1, image frame 2, and image frame 3. Figure 8 In the process, when the Vsync signal arrives at time T4, the buffer only contains image frames 1 and 2 that have been drawn. Therefore, compared with... Figure 8 compared to, Figure 9 In the example shown, when the Vsync signal arrives at time T4, the buffer stores one more drawn image frame, achieving the effect of frame interpolation. Thus, starting from time T3, when each Vsync signal arrives, the electronic device 100 displays an image frame without any frame loss.

[0146] During the period from the moment the hand leaves the screen to the end of the screen, i.e., within the second time interval, the electronic device 100 needs to determine whether to trigger the drawing of the current image frame based on the size of the cached image frames. Specifically, if the size of the cached image frames is less than the first threshold, it means that there are few cached image frames. Therefore, after the previous image frame is drawn, the electronic device 100 can trigger the drawing of the current image frame and obtain the drawing start time of the current image frame. However, if the size of the cached image frames is greater than or equal to the first threshold, it means that there are many cached image frames. In this case, there is no rush to draw the current image frame. Therefore, the electronic device 100 can wait for the Vsync signal to arrive and then respond to the Vsync signal of the electronic device 100 before starting to draw the current image frame and obtaining the drawing start time of the current image frame.

[0147] In existing related technologies, since the interface shown in the right figure of Figure 5(c) is displayed through a third-party control embedded in an instant messaging application, existing related technologies do not support frame insertion in the interface shown in the right figure of Figure 5(c). Thus, as... Figure 8 As shown, if the rendering of a certain image frame takes a long time, frame dropping is likely to occur. However, the image frame rendering method provided in this embodiment can insert frames into the interface shown in the right figure of Figure 5(c), thereby increasing the number of cached image frames (e.g., Figure 9 As shown in the figure, this avoids frame dropping and improves the smoothness of interface scrolling. Furthermore, this embodiment can adjust the frame insertion timing based on the number of cached image frames, thus avoiding frame dropping without increasing the burden on the UI thread.

[0148] Figure 10 A flowchart of an image frame drawing method provided in another embodiment of this application is shown below. Figure 10 As shown, the method for drawing the above image frame may include:

[0149] Step 1001: Listen for the throw / slip event.

[0150] In this embodiment, the swipe event can be: during the process of the user browsing the current display interface of the electronic device 100, after the user performs a swipe operation on the current display interface, the current display interface of the electronic device 100 slides upward, and due to inertia, the sliding speed of the current display interface changes from fast to slow.

[0151] The currently displayed interface can be the interface of the application currently running on the electronic device 100, and the currently displayed interface can include the application's native interface (such as...). Figures 5(a) to 5(b) As shown in Figure 5(c), it can also include interfaces opened using third-party controls (e.g., webview controls and / or flutter controls) in the application (as shown in Figure 5(c)).

[0152] Step 1002: A user's swipe action on the current display screen is detected.

[0153] Step 1003: Determine if the Vsync signal has arrived.

[0154] If no, meaning the Vsync signal corresponding to the current image frame to be drawn has not yet arrived, then proceed to step 1004; if yes, meaning the Vsync signal corresponding to the current image frame to be drawn has arrived, then proceed to step 1007.

[0155] For example, see Figure 9 When the current image frame to be drawn is image frame 1, the corresponding Vsync signal is the Vsync signal at time T1.

[0156] Step 1004: Determine whether the current time is within the slide-off period; if yes, proceed to step 1005; if no, that is, the current time is no longer within the slide-off period, proceed to step 1006.

[0157] Among them, the aforementioned sliding off-hand phase is Figure 7 In the illustrated embodiment, the first time interval mentioned above, for example, can be the start time of the sliding hand-off period, which can be the moment when the sliding operation is detected, and the duration of the sliding hand-off period is 300 milliseconds.

[0158] The current time is not within the slide-off period, which means that the slide-off period has ended and the current time is in the second time interval. The start time of the second time interval is the end time of the slide-off period, and the end time of the second time interval is the end time of the slide. For example, the current time is not within the slide-off period, which can be understood as the current time being within the period from 300 milliseconds to the end of the slide.

[0159] Step 1005: After the previous image frame is drawn, the electronic device 100 determines the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval. Then, step 1008 is executed.

[0160] For example, see Figure 9 During the sliding-off period, after image frame 1 is drawn, the drawing of image frame 2 is triggered.

[0161] In this step, the method by which the electronic device 100 determines the start time of drawing can refer to this application. Figure 4 The description of step 401 in the illustrated embodiment will not be repeated here.

[0162] Step 1006: Determine whether the size of the cached image frame is less than the first threshold. If yes, proceed to step 1005; otherwise, proceed to step 1003.

[0163] In this step, if the size of the cached image frame is less than the first threshold, it means that there are few completed image frames stored in the current cache. Therefore, the current image frame can be drawn after the previous image frame is drawn, thus achieving the effect of frame interpolation. However, if the size of the cached image frame is greater than or equal to the first threshold, it means that there are already a lot of completed image frames stored in the current cache. In this case, in order to avoid increasing the burden on the UI thread, we can wait for the Vsync signal of the electronic device 100 to trigger the drawing of the current image frame, and frame interpolation is not required.

[0164] In step 1007, in response to the Vsync signal of electronic device 100, electronic device 100 determines the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval. Then, step 1008 is executed.

[0165] In this step, the method by which the electronic device 100 determines the start time of drawing can refer to this application. Figure 4 The description of step 401 in the illustrated embodiment will not be repeated here.

[0166] Step 1008: Electronic device 100 draws the current image frame according to the above-mentioned drawing start time.

[0167] In this step, the process by which the electronic device 100 draws the current image frame according to the drawing start time can be found in this application. Figure 4 The description of step 402 in the illustrated embodiment will not be repeated here.

[0168] In this embodiment, after step 1008, we can return to step 1003 and wait for the drawing of the next image frame to be triggered.

[0169] In the above-described image frame drawing method, after detecting a user's swipe operation on the current display interface, when drawing the current image frame, the electronic device 100 no longer relies on the Vsync signal. Instead, it determines the start time for drawing the current image frame based on the first moment and the current frame interval, and then draws the current image frame according to the aforementioned start time. Since the time interval between the start times for drawing the current image frame, determined based on the first moment and the current frame interval, is relatively uniform, the swipe displacement and swipe speed calculated based on this start time are relatively smooth, thus making the interface swipe smoother and less prone to jitter. Furthermore, as... Figure 11As shown, the above method can achieve frame interpolation effect and quickly fill the buffer in a short time after the swipe is released. When encountering a scene where the drawing of a certain image frame takes a long time, since there are enough drawn image frames in the buffer, frame loss can be avoided by consuming the image frames in the buffer during the rendering and compositing (surfaceflinger, SF) stages. Figure 11 This is a schematic diagram illustrating a method for quickly replenishing the cache according to one embodiment of this application.

[0170] It is understood that some or all of the steps or operations in the above embodiments are merely examples, and other operations or variations thereof can be performed in the embodiments of this application. Furthermore, the steps may be performed in different orders as presented in the above embodiments, and it is not necessary to perform all the operations in the above embodiments.

[0171] It is understood that, in order to achieve the above-mentioned functions, electronic devices include hardware and / or software modules that perform the respective functions. Based on the algorithm steps of the examples described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0172] This embodiment can divide the electronic device into functional modules according to the above method embodiment. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0173] Figure 12 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. In the case where functional modules are divided according to their respective functions, Figure 12 A schematic diagram of a possible composition of the electronic device 1200 involved in the above embodiments is shown, such as... Figure 12 As shown, the electronic device 1200 may include: a receiving unit 1201, a processing unit 1202, and a transmitting unit 1203;

[0174] The processing unit 1202 can be used to support the electronic device 1200 in executing steps 401 to 403, as well as steps 601, 701 to 702, and / or other processes used in the technical solutions described in the embodiments of this application.

[0175] It should be noted that this application Figures 4 to 11 All relevant content of each step involved in the illustrated embodiment can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0176] The electronic device 1200 provided in this embodiment is used to execute this application. Figures 4 to 11 The image frame drawing method provided in the illustrated embodiment can achieve the same effect as the method described above.

[0177] It should be understood that electronic device 1200 can correspond to Figure 1 The electronic device 100 shown. The functions of the receiving unit 1201 and the transmitting unit 1203 can be determined by... Figure 1 The processor 110, antenna 1, and mobile communication module 150 in the illustrated electronic device 100, and / or, are implemented by the processor 110, antenna 2, and wireless communication module 160; the function of the processing unit 1202 can be implemented by... Figure 1 The processor 110, touch sensor 180K, and display screen 194 in the illustrated electronic device 100 are implemented.

[0178] When using integrated units, the electronic device 1200 may include a processing module, a storage module, and a communication module.

[0179] The processing module can be used to control and manage the actions of the electronic device 1200. For example, it can support the electronic device 1200 in executing the steps performed by the receiving unit 1201, processing unit 1202, and sending unit 1203. The storage module can support the electronic device 1200 in storing program code and data. The communication module can support communication between the electronic device 1200 and other devices.

[0180] The processing module can be a processor or controller, which can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as radio frequency circuitry, a Bluetooth chip, and / or a Wi-Fi chip.

[0181] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device 1200 involved in this embodiment can be a device having... Figure 1 The device with the structure shown.

[0182] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute this application. Figures 4-9 The method provided in the illustrated embodiment.

[0183] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to execute this application. Figures 4-9 The method provided in the illustrated embodiment.

[0184] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0185] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0186] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0187] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0188] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for drawing an image frame, characterized in that, include: After detecting a user's swipe operation on the current display interface, the drawing start time of the current image frame to be drawn is determined based on the first moment and the current frame interval; wherein, the current display interface is the interface of the application currently running on the electronic device, and the first moment is the drawing start time of the previous image frame. The current image frame is drawn according to the drawing start time.

2. The method according to claim 1, characterized in that, The step of determining the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval includes: Calculate the sum of the first time point and the frame interval to obtain the second time point; If the second time is greater than or equal to the end time of the previous image frame, then the second time is taken as the start time of the current image frame. If the second time is less than the end time of drawing the previous image frame, then the end time of drawing the previous image frame is taken as the start time of drawing the current image frame.

3. The method according to claim 1, characterized in that, The step of determining the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval includes: Within the first time interval after the sliding operation is detected, after the previous image frame is drawn, the start time for drawing the current image frame to be drawn is determined based on the first moment and the current frame interval; wherein, the start time of the first time interval is the moment when the sliding operation is detected, and the duration of the first time interval is a predetermined duration.

4. The method according to claim 3, characterized in that, The step of determining the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval includes: Within the second time interval, if the size of the cached image frame is less than the first threshold, then after the previous image frame is drawn, the start time for drawing the current image frame to be drawn is determined based on the first time and the current frame interval; wherein, the second time interval is after the first time interval, the start time of the second time interval is the end time of the first time interval, and the end time of the second time interval is the end time of the sliding.

5. The method according to claim 4, characterized in that, The step of determining the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval includes: Within the second time interval, if the size of the cached image frame is greater than or equal to the first threshold, then in response to the vertical synchronization signal of the electronic device, the drawing start time of the current image frame to be drawn is determined according to the first moment and the current frame interval.

6. The method according to claim 1, characterized in that, The step of determining the start time for drawing the current image frame to be drawn based on the first moment and the current frame interval includes: In response to the vertical synchronization signal of the electronic device, the start time of drawing the current image frame to be drawn is obtained.

7. The method according to claim 1, characterized in that, Drawing the current image frame according to the drawing start time includes: Based on the drawing start time, the sliding displacement and sliding speed of the current image frame to be drawn are determined using the sliding curve of the application. The current image frame is drawn based on the sliding displacement and the sliding speed.

8. An electronic device, characterized in that, include: One or more processors; Memory; Multiple applications; And one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the electronic device, cause the electronic device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-7.

Images