Data processing method and related device

Abstract

The embodiment of the present application provides a data processing method and related devices, which are applied to the field of terminal technology. The method includes: in response to the up event of the sliding operation, calculating the sliding speed of the list control; wherein the sliding operation acts on the list control; when the sliding speed of the list control is greater than or equal to the speed threshold, determining whether the rendering result of the first image is stored in the cache queue, the first image is the image of the electronic device after receiving the up event; when the rendering result of the first image is stored in the cache queue, synthesizing the first image. By pre-drawing and rendering multiple images (predicted frames) of the list interface, when the pre-drawn rendered image (predicted frame) is matched during the next sliding operation, the corresponding predicted frame is synthesized and displayed. In this way, the drawing and rendering time of one frame is saved, thereby reducing the problem of no frame available due to the long drawing and rendering time, and reducing the jamming phenomenon.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a data processing method and related devices. Background Art

[0002] Currently, users can browse various types of content through the display screen of terminal devices. When there is a lot of content, the display screen cannot display all the content at once, and users can slide on the screen to browse related content.

[0003] The interface display of a terminal device typically requires processes such as drawing, rendering, compositing, and displaying. For example, the interface drawing process of a terminal device may include background drawing, subview drawing, scroll bar drawing, and other processes. The interface compositing process of a terminal device may include processes such as vertex processing and pixel processing.

[0004] However, the terminal device may freeze when it is released from the hand and slid. Summary of the Invention

[0005] The present invention provides a data processing method and related apparatus for use in electronic devices. The data processing method synthesizes pre-rendered images to save rendering time, thereby resolving the problem of lag caused by long rendering times.

[0006] In a first aspect, an embodiment of the present application proposes a data processing method, which is applied to an electronic device, and the method includes: calculating the sliding speed of a list control in response to an up event of a sliding operation; wherein the sliding operation acts on the list control; when the sliding speed of the list control is greater than or equal to a speed threshold, determining whether a rendering result of a first image is stored in a cache queue, the first image being an image of the electronic device after receiving the up event; when the rendering result of the first image is stored in the cache queue, synthesizing the first image.

[0007] The first image can be one or more. The rendering result of the first image can also be called a predicted frame. In this way, by pre-drawing multiple images (predicted frames) of the rendering list interface, when the pre-drawn rendered image (predicted frame) is matched during the next sliding operation, the corresponding predicted frame is synthesized and displayed. In this way, the drawing and rendering time of one or more frames is saved, thereby reducing the problem of no frame available due to long drawing and rendering time and reducing the jamming phenomenon.

[0008] Optionally, confirming whether the rendering result of the first image is stored in the cache queue includes: calculating the displacement value of the first image, where the displacement value is the displacement relative to the down event in the sliding operation; confirming whether the rendering result of the first image is stored in the cache queue based on the displacement value and a mapping relationship, where the mapping relationship is the relationship between the rendering results of multiple images stored in the cache queue and multiple hotspot displacement values; the hotspot displacement value is the top N displacement values ​​ranked in descending order of the number of occurrences in historical sliding operations.

[0009] In this way, the image is pre-rendered based on the hotspot displacement value, and the displacement value is used to determine whether the rendering result of the first image is stored in the cache queue. Then, when a pre-rendered image (predicted frame) is matched, the predicted frame is synthesized. This saves one or more frames of rendering time, thereby reducing the problem of unavailable frames caused by long rendering times and reducing lag.

[0010] Optionally, based on the displacement value and the mapping relationship, confirming whether the rendering result of the first image is stored in the cache queue includes: when the difference between the displacement value and any hotspot displacement value is less than or equal to a first threshold, confirming that the rendering result of the first image is stored in the cache queue; or, when the difference between the displacement value and multiple hotspot displacement values ​​is greater than the first threshold, confirming that the rendering result of the first image is not stored in the cache queue.

[0011] In this way, the matching range of the rendering result of the first image can be increased, so that the rendering result of the first image.

[0012] Optionally, calculating the displacement value of the first image includes: calculating the displacement value relative to the up event according to a pre-set formula in the electronic device and a pre-stored normalized displacement curve, wherein the pre-set formula satisfies: T = A × sliding speed, S = B × sliding speed, the total displacement of the list control sliding after the up event, S is the total time the list control slides after the up event, and A and B are both constants; calculating the sum of the displacement value relative to the up event and the first displacement to obtain the displacement value, the first displacement being the difference between the touch coordinates corresponding to the up event and the touch coordinates corresponding to the down event.

[0013] Optionally, the displacement value relative to the up event is calculated according to a preset formula and a pre-stored normalized displacement curve, including: substituting the sliding velocity into the formula to calculate the total displacement and total time; and calculating the displacement value relative to the up event according to the total displacement, total time and the normalized displacement curve.

[0014] Optionally, the normalized displacement curve is stored in the electronic device in the form of multiple discrete points.

[0015] Optionally, the method further includes: when the displacement value is greater than the maximum value of the hotspot displacement values, discarding the rendering results of multiple frames of images in the cache queue.

[0016] Optionally, the method further includes: updating the hotspot displacement value after the sliding operation ends; and drawing the rendered image based on the updated hotspot displacement value.

[0017] Optionally, the method further includes: when the rendering result of the first image is not stored in the cache queue, rendering the first image.

[0018] Optionally, the electronic device includes: an application and an image synthesis system; in response to an up event of a sliding operation, calculating the sliding speed of the list control, including: the application calculates the sliding speed of the list control in response to the up event of the sliding operation; confirming whether the rendering result of the first image is stored in the cache queue, including: the electronic device confirming whether the rendering result of the first image is stored in the cache queue; synthesizing the first image, including: the image synthesis system synthesizing the first image.

[0019] Optionally, confirming whether the rendering result of the first image is stored in the cache queue includes: the sliding control in the application calculates the displacement value of the first image, where the displacement value is the displacement relative to the down event in the sliding operation; the image synthesis system confirms whether the rendering result of the first image is stored in the cache queue based on the displacement value and a mapping relationship, where the mapping relationship is the relationship between the rendering results of multiple images stored in the cache queue and multiple hotspot displacement values; the hotspot displacement value is the top N displacement values ​​ranked in descending order of the number of occurrences in historical sliding operations.

[0020] Optionally, the image synthesis system confirms whether the rendering result of the first image is stored in the cache queue based on the displacement value and the mapping relationship, including: when the difference between the displacement value and any hotspot displacement value is less than or equal to a first threshold, the image synthesis system confirms that the rendering result of the first image is stored in the cache queue; or, when the difference between the displacement value and multiple hotspot displacement values ​​is greater than the first threshold, the image synthesis system confirms that the rendering result of the first image is not stored in the cache queue.

[0021] Optionally, the sliding control calculates the displacement value of the first image, including: the sliding control calculates the displacement value relative to the up event according to a formula preset in the electronic device and a pre-stored normalized displacement curve, wherein the preset formula satisfies: T=A×sliding speed, S=B×sliding speed, where S is the total displacement of the list control sliding after the up event, S is the total sliding time of the list control after the up event, and A and B are both constants;

[0022] The sliding control calculates the sum of the displacement value relative to the up event and the first displacement to obtain the displacement value, where the first displacement is the difference between the touch coordinates corresponding to the up event and the touch coordinates corresponding to the down event.

[0023] Optionally, the sliding control calculates the displacement value relative to the up event based on a preset formula and a pre-stored normalized displacement curve, including: the sliding control substituting the sliding speed into the formula to calculate the total displacement and total time; the sliding control calculates the displacement value relative to the up event based on the total displacement, total time and the normalized displacement curve.

[0024] Optionally, the normalized displacement curve is stored in the electronic device in the form of multiple discrete points.

[0025] Optionally, the method further includes: when the displacement value is greater than the maximum value of the hotspot displacement values, the image synthesis system discards the rendering results of multiple frames of images in the cache queue.

[0026] Optionally, the electronic device further includes a data manager: the data manager updates the hotspot displacement value after the sliding operation is completed; and the application draws the rendered image based on the hotspot displacement value updated by the data manager.

[0027] Optionally, the method further includes: when the rendering result of the first image is not stored in the cache queue, applying rendering to the first image.

[0028] In a second aspect, an embodiment of the present application provides an electronic device, including a terminal device, which may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc.

[0029] The electronic device includes a processor, and the processor is configured to call a computer program in a memory to execute the method according to the first aspect.

[0030] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect.

[0031] In a fourth aspect, an embodiment of the present application provides a computer program product, which, when the computer program is executed, enables the computer to execute the method of the first aspect.

[0032] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute a method such as the first aspect.

[0033] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the hardware system structure of a terminal device provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of the terminal device software system structure provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a scenario provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a terminal device interface display processing flow in a possible implementation;

[0038] Figure 5 A schematic diagram of another terminal device interface display processing flow in a possible implementation;

[0039] Figure 6 A schematic diagram of an interface provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of the interaction of modules involved in forming a prediction frame in the data processing method provided in an embodiment of the present application;

[0041] Figure 8 A schematic diagram of rendering after lifting the hand and sliding provided in an embodiment of the present application;

[0042] Figure 9 A schematic diagram of a normalized displacement curve provided in an embodiment of the present application;

[0043] Figure 10 A schematic diagram of module interactions involved in displaying predicted frames in the data processing method provided in an embodiment of the present application;

[0044] Figure 11 A schematic diagram of a process flow of hand-off sliding provided in an embodiment of the present application;

[0045] Figure 12 A flowchart of a data processing method provided in an embodiment of the present application;

[0046] Figure 13 A schematic diagram of a terminal device interface display processing flow provided in an embodiment of the present application;

[0047] Figure 14 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0048] Figure 15 A schematic diagram of the hardware structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] To facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first chip and the second chip are merely used to distinguish different chips and do not limit their order. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or execution order, and the words "first" and "second" do not necessarily mean that they are different.

[0050] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0051] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0052] The sliding display method provided in the embodiment of the present application can be applied to electronic devices with display functions.

[0053] Electronic devices include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablet computers (Pads), computers with wireless transceiver functions, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal devices.

[0054] In order to better understand the embodiments of the present application, the structure of the terminal device in the embodiments of the present application is introduced below:

[0055] Figure 1FIG1 shows a schematic diagram of the structure of a terminal device 100. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0056] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0057] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0058] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0059] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0060] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0061] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the terminal device 100.

[0062] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0063] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0064] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0065] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the terminal device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0066] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0067] The USB interface 130 is an interface that complies with USB standards and specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the terminal device 100, or to transfer data between the terminal device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as AR devices.

[0068] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is for illustrative purposes only and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0069] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 can also provide power to the terminal device via the power management module 141.

[0070] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0071] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0072] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The antennas in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0073] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0074] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0075] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0076] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the Beidou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS) and / or the Satellite Based Augmentation System (SBAS).

[0077] The terminal device 100 implements display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0078] The display screen 194 is used to display images, videos, and receive sliding operations. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0079] The terminal device 100 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0080] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0081] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0082] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0083] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. This allows terminal device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0084] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in the terminal device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0085] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0086] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0087] The terminal device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0088] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0089] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The terminal device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0090] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 100 receives a call or voice message, the user can hear the voice by placing the receiver 170B close to the ear.

[0091] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The terminal device 100 can be provided with at least one microphone 170C. In other embodiments, the terminal device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 100 can also be provided with three, four or more microphones 170C to realize sound signal collection, noise reduction, and can also identify the source of sound, realize directional recording function, etc.

[0092] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0093] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device comprising at least two parallel plates with conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 194, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The terminal device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

[0094] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0095] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0096] The magnetic sensor 180D includes a Hall effect sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Furthermore, based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0097] The accelerometer 180E can detect the magnitude of acceleration of the terminal device 100 in all directions (generally three axes). When the terminal device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the terminal device's posture, enabling applications such as landscape and portrait screen switching and pedometers.

[0098] The distance sensor 180F is used to measure distance. The terminal device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the terminal device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0099] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light outward through the light emitting diode. The terminal device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can use the proximity light sensor 180G to detect when the user holds the terminal device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0100] Ambient light sensor 180L is used to sense ambient light brightness. Terminal device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether terminal device 100 is in a pocket to prevent accidental touches.

[0101] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0102] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 heats the battery 142 to prevent the terminal device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0103] The touch sensor 180K is also called a "touch control device." The touch sensor 180K can be provided on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be provided on the surface of the terminal device 100, at a location different from that of the display screen 194.

[0104] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bones of the human body's vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse the vibration signals of the vibrating bones of the vocal cords acquired by the bone conduction sensor 180M to obtain voice signals and implement voice functions. The application processor can parse heart rate information based on the blood pressure signals acquired by the bone conduction sensor 180M to implement heart rate detection functions.

[0105] The buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons. The terminal device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the terminal device 100.

[0106] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0107] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0108] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the terminal device 100 by inserting or removing it from the SIM card interface 195. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0109] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture, etc. In the embodiment of the present application, the Android system with a layered architecture is used as an example to exemplify the software structure of the terminal device 100.

[0110] Figure 2 It is a software structure block diagram of the terminal device in an embodiment of the present application.

[0111] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0112] The application layer can include a series of application packages. Figure 2 As shown, the application package can include applications such as phone, email, calendar, and camera. The application layer can also include multiple widgets, such as an overscroller. Applications such as phone, email, calendar, and camera can call widgets.

[0113] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0114] like Figure 2As shown, the application framework layer may include an input system, an activity manager, a location manager, a data manager, a notification manager, an activity manager, an image composition system, and a view system, etc.

[0115] The input system manages programs that use input devices. For example, the input system determines input actions such as mouse clicks, keyboard inputs, and touch swipes.

[0116] The Activity Manager is used to manage the lifecycle of each application and the navigation back function. It is responsible for creating the Android main thread and maintaining the lifecycle of each application.

[0117] The location manager is used to provide location services to applications, including querying the last known location, registering and unregistering from periodic location updates, etc.

[0118] The data manager is used to store and count data related to hand-off sliding, such as activity name, displacement value, and hotspot displacement.

[0119] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating the device, or flashing indicator lights.

[0120] The Activity Manager Service (AMS) manages the lifecycle of each application and the navigation back function. It is responsible for creating the Android main thread and maintaining the lifecycle of each application.

[0121] The image synthesis system is used to control image synthesis and generate a vertical synchronization (Vsync) signal.

[0122] The image synthesis system includes: a synthesis thread, a Vsync thread, and a queue buffer thread. The synthesis thread is used to be awakened by the Vsync signal for synthesis. The Vsync thread is used to generate the next Vsync signal according to the Vsync signal request. The queue buffer thread is used to store the cache, generate Vsync signal requests, and wake up the synthesis thread, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon can include a view for displaying text and a view for displaying pictures.

[0123] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0124] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0125] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0126] The system library can include multiple functional modules, such as image rendering library, image synthesis library, function library, media library, and input processing library.

[0127] The image rendering library is used to render 2D or 3D images. The image synthesis library is used to synthesize 2D or 3D images.

[0128] In a possible implementation, the application renders the image using the image rendering library, and then sends the rendered image to the cache queue of the image compositing system. Whenever a Vsync signal arrives, the image compositing system (e.g., a surface flinger) sequentially retrieves a frame of image to be synthesized from the cache queue, and then performs image synthesis using the image compositing library.

[0129] The function library provides macros, type definitions, string operation functions, mathematical calculation functions, and input and output functions used in the C language.

[0130] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0131] The input processing library is a library used to process input devices, which can implement mouse, keyboard and touch input processing, etc.

[0132] The kernel layer is the layer between hardware and software. The kernel layer includes at least the touch panel (TP) driver, display driver, Bluetooth driver, Wi-Fi driver, keyboard driver, shared memory driver, and camera driver.

[0133] Hardware can be audio devices, Bluetooth devices, camera devices, sensor devices, etc.

[0134] The following describes the workflow of the software and hardware of the terminal device 100 by way of example, in conjunction with the scenario of application startup or interface switching in the application.

[0135] When the touch sensor 180K in the touch panel receives a touch operation, the kernel layer processes the touch operation into an original input event (including touch coordinates, touch force, timestamp of the touch operation and other information). The original input event is stored in the kernel layer. The kernel layer reports the original input event to the input system of the application framework layer through the input processing library. The input system of the application framework layer parses the information of the original input event (including: operation type, timestamp and reporting point position, etc.) and determines the focus application based on the current focus, and sends the parsed information to the focus application. The focus can be the touch point in a touch operation or the click position in a mouse click operation. The focus application is the application running in the foreground of the terminal device or the application corresponding to the touch position in the touch operation. The focus application determines the control corresponding to the original input event based on the parsed information of the original input event (for example, reporting point position).

[0136] Taking the touch operation as a touch sliding operation, and the control corresponding to the touch sliding operation as the list control of the WeChat application as an example, the WeChat application calls the image rendering library in the system library through the view system of the application framework layer to draw and render the image. The WeChat application sends the drawn and rendered image to the cache queue of the image synthesis system. The image synthesis library in the system library synthesizes the drawn and rendered image in the image synthesis system into the WeChat interface. The image synthesis system uses the display driver of the kernel layer to make the screen (display) display the corresponding interface of the WeChat application.

[0137] To facilitate understanding, some of the examples given are provided for reference to the description of concepts related to the embodiments of the present application.

[0138] 1. Frame: This refers to the smallest unit of a single image in an interface display. A frame can be thought of as a still image. Displaying multiple frames in rapid succession can create the illusion of motion. Frame rate refers to the number of frames refreshed per second, or the number of times per second the graphics processor in a terminal device refreshes the image. A higher frame rate results in smoother and more realistic animation. The more frames per second, the smoother the displayed motion.

[0139] It should be noted that before the interface displays a frame, it usually needs to go through processes such as drawing, rendering, and synthesis.

[0140] 2. Frame drawing: refers to the image drawing of the display interface. The display interface can be composed of one or more views. Each view can be drawn by the visual control of the view system. Each view is composed of subviews. A subview corresponds to a small widget in the view. For example, a subview corresponds to a symbol in the image view.

[0141] 3. Frame rendering: This is to shade the drawn view or add 3D effects, etc. For example, 3D effects can be lighting effects, shadow effects, and texture effects.

[0142] 4. Frame synthesis: It is the process of synthesizing multiple or more rendered views into a display interface.

[0143] 5. Vertical synchronization (Vsync) signal: This signal controls the start of frame drawing, rendering, synthesis, and display. The Vsync signal is a periodic signal, and the Vsync signal period can be set based on the screen refresh rate. For example, when the screen refresh rate is 60Hz, the Vsync signal period can be 16.6ms, meaning that the terminal device generates a control signal every 16.6ms to trigger the Vsync signal period. To ensure smooth display and avoid display freezes and other phenomena, terminal devices generally display based on the Vsync signal to synchronize image drawing, rendering, synthesis, and screen refresh processes.

[0144] Vsync signals include software Vsync (Vsync-APP or Vsync-SF) and hardware Vsync (Vsync-HW). Vsync-APP triggers the rendering process. Vsync-SF triggers the compositing process. The hardware Vsync signal (Vsync-HW) triggers the screen display refresh process. Typically, software Vsync and hardware Vsync maintain cycle synchronization. For example, if Vsync-HW switches from 60Hz to 120Hz, Vsync-APP and Vsync-SF change synchronously, switching from 60Hz to 120Hz.

[0145] 6. Hand-off sliding: refers to the terminal device interface continuing to slide and display at the initial speed after the touch sliding ends.

[0146] For example, in applications such as Settings and Headlines, the interface displays a list layout. After the finger on the sliding interface is lifted, the interface can continue to slide for a period of time at the initial speed after the finger is released.

[0147] The application scenarios provided by the embodiments of the present application are described below with reference to the accompanying drawings. Figure 3 Schematic diagram of the application scenario provided by the embodiment of the present application.

[0148] The terminal device can Figure 3 The interface of the social application shown in a, or Figure 3 In the settings related interface shown in b, Figure 3 The document interface shown in c, Figure 3 The product browsing interface shown in d in the figure receives the user's upward or downward operation. The terminal device can also Figure 3 In the interface shown in e, Figure 3 The e-book interface shown in f in FIG. 1 is displayed, and when a left swipe operation or a right swipe operation is received by the user, the terminal device performs frame drawing, rendering, synthesis, and other processes based on the swipe operation to display the content corresponding to the swipe operation.

[0149] The following is based on Figure 4 and Figure 5 Describes the process of displaying the interface when sliding the terminal device.

[0150] Figure 4 The following is a schematic diagram of a terminal device interface display processing flow in a possible implementation. Figure 4 The contents displayed in frames 1, 2, and 3 correspond to different interfaces. Figure 4 When the terminal device receives the user's upward sliding operation in the application interface, the content displayed by the terminal device corresponds to frame 1, frame 2 and frame 3 in sequence.

[0151] The application in the terminal device uses the view system of the application framework layer to draw and render frames 1, 2, and 3 in sequence. After frame 1 is drawn and rendered, the terminal device application sends the drawn and rendered frame 1 to the image synthesis system (for example, surface flinger). The image synthesis system synthesizes the drawn and rendered frame 1. After frame 1 is synthesized, the terminal device can start the display driver by calling the kernel layer to display the content corresponding to frame 1 on the screen (display). Frames 2 and 3 are also synthesized and displayed in a similar process to frame 1, which will not be repeated here.

[0152] Figure 4 In the display processing shown, frames 1, 2, and 3 are rendered within a single Vsync cycle. However, in actual terminal applications, the time it takes to render frames may increase due to various factors, such as heavy system load (CPU and GPU usage) or poor network connectivity. The terminal may run out of frames to synthesize, causing display lag.

[0153] For example, Figure 5 The following is a schematic diagram of another terminal device interface display processing flow in a possible implementation. Figure 5 The contents displayed in frames 1, 2, and 3 correspond to different interfaces. Figure 5 When the terminal device receives the user's upward sliding operation, the content displayed by the terminal device corresponds to frame 1, frame 2 and frame 3 in sequence. Figure 5 Frame 1, frame 2 and frame 3 are rendered, synthesized and displayed respectively. For details, please refer to Figure 4 The corresponding description.

[0154] and Figure 4 The difference is, Figure 5 In frame 2, the rendering time increases to 2 Vsync cycles due to various reasons, for example, the reasons include the complexity of graphics drawing or the fluctuation of system load due to the influence of downloaded data. Figure 5 When Vsync arrives, frame 2 rendering is not complete, the image synthesis system has no frames to take, and no image synthesis occurs. The display time of frame 1 increases, causing display stuttering and a poor user experience.

[0155] In view of this, the embodiments of the present application provide a data processing method and related apparatus. By pre-rendering multiple images (predicted frames) of a rendering list interface, when the next sliding operation matches the pre-rendered image (predicted frame), the corresponding predicted frame is synthesized and displayed. This saves one frame of rendering time, thereby reducing the problem of unavailable frames caused by long rendering time and reducing lag.

[0156] For ease of understanding, the following Figure 6 This section describes the interface changes during a sliding operation, as well as the multiple touch coordinates and displacement values ​​involved in the sliding operation.

[0157] It should be noted that, when the terminal device receives a sliding operation from the user, the types of input events corresponding to the sliding operation are press (down), move (move) and lift (up).

[0158] For example, Figure 6This is a schematic diagram of an interface provided by an embodiment of the present application. For example, a terminal device with a screen refresh rate of 120Hz and an interface display order of frames 1, 2, 3, 4, and 5 is shown. Frames 1, 2, and 3 are the display interfaces corresponding to a down event, a move event, and an up event, respectively. Frames 4 and 5 are the display interfaces corresponding to an up event.

[0159] like Figure 6 As shown, frame 1 is the initial interface. The interface corresponding to frame 1 is the list interface. When the user presses the screen with his finger, the terminal device determines that the input event is a down event, and the touch coordinates corresponding to the down event (first touch coordinates) are (60, 25). The interface displayed by the terminal device remains unchanged, which is frame 1. When the user slides his finger upward, the terminal device determines that the input event is a move event. If the touch coordinates corresponding to the move event (second touch coordinates) are (60, 100), the terminal device determines the screen layout according to the second touch coordinates for drawing and rendering, and the interface displayed by the terminal device changes from frame 1 to frame 2. The difference between the second touch coordinates and the first touch coordinates is the first displacement value.

[0160] When the user's finger leaves the screen, the terminal device determines the input event as an up event. If the touch coordinates corresponding to the up event (the third touch coordinates) are (60, 150), the terminal device determines the screen layout based on the third touch coordinates and performs rendering. The terminal device's interface changes from frame 2 to frame 3. At this time, the screen refresh rate is 120Hz, the corresponding frame interval is 8.3ms, and the frame displacement is the difference between the third touch coordinates and the second touch coordinates, that is, 75piex1. The sliding speed is the ratio of the frame displacement to the frame interval, which is 75piex1 / 8.3ms.

[0161] If the speed threshold is 10 piex1 / 8.3 ms and the sliding speed exceeds the speed threshold, the application continues to request the Vsync-APP signal for drawing and rendering. When the application receives the Vsync-APP signal, the application calculates the second displacement value and speed value based on the preset formula and normalized displacement curve. If the calculated speed value is 40 piex1 / 8.3 ms and the second displacement value is 50 piex1, the terminal device interface changes from frame 3 to frame 4.

[0162] The second displacement value is a displacement value relative to the third touch coordinate. The calculation of the second displacement value and the speed value can refer to the following related descriptions, which will not be repeated here.

[0163] Since the speed value corresponding to frame 4 is not 0, the application continues to request the Vsync-APP signal for drawing and rendering. When the application receives the Vsync-APP signal, the application continues to calculate the second displacement value according to the preset formula and normalized displacement curve. If the calculated speed value is 20piexl / 8.3ms, the second displacement value is 75piexl, and the interface of the terminal device changes from frame 4 to frame 5. Since the speed value corresponding to frame 5 is not 0, the application continues to request the Vsync-APP signal for drawing and rendering, and the above steps are executed in a loop until the speed value is 0. If frame 6 is the interface corresponding to the speed value of 0, the second displacement value corresponding to frame 6 is 90piexl, and the sliding ends.

[0164] In this sliding operation, the second displacement values ​​are 50, 75, and 90, respectively. If the second displacement values ​​corresponding to the previous sliding operation were 50, 100, 120, and 130, then the heat displacement value can be 50. In this case, the terminal device will pre-render the interface shown in frame 7 based on frame 6 and the heat displacement value, and the interface shown in frame 7 will not be synthesized or displayed.

[0165] For ease of understanding, the following Figure 7-Figure 9 The process of interaction between the various modules involved in the data processing method provided in the embodiment of the present application is explained.

[0166] Figure 7 This is a diagram illustrating the module interactions involved in forming a predicted frame in the data processing method provided in an embodiment of the present application. The modules involved include: an input system, an application, a data manager, an image synthesis system (surface flinger), a display driver, and a data manager. The application includes: an application main thread, a slider control, and a rendering thread.

[0167] When the terminal device receives a sliding operation from the user, the types of input events corresponding to the sliding operation are press (down), move (move), and lift (up). The following describes the three input events.

[0168] The following combination Figure 7 S601-S603 describe the interaction of various modules when the input event is a down event.

[0169] S601. During a user's sliding operation, the input system parses a first input event to obtain first input information, and sends the first input information to the application main thread. The first input information includes a down event and first touch coordinates.

[0170] S602: After receiving the first input information, the application main thread records the first input information and determines whether the application interface includes a list control. The list control can be a listview or a recyleview, and the present embodiment does not limit the list control.

[0171] S603: When the application main thread determines that the application interface includes a list control, the first input information is sent to the sliding control, and the sliding control records the first input information.

[0172] In a possible implementation, after receiving the first input information, the application main thread sends a Vsync-APP request to the image compositing system to request the Vsync-APP corresponding to the next input event. The image compositing system sets a timer for sending the next Vsync-APP. After the timer expires, the image compositing system feeds the next Vsync-APP back to the application main thread.

[0173] The following combination Figure 7 S604-S615 describe the interaction between modules when the input event is a move event.

[0174] S604: The input system parses the second input event to obtain second input information, and sends the second input information to the application main thread. The second input information includes a move event and second touch coordinates.

[0175] S605 . When receiving the second input information (move event), the main application thread sends a Vsync-APP request to the image synthesis system to request processing of the Vsync-APP corresponding to the input event (move event).

[0176] In a possible implementation, the image synthesis system sets a timer for indicating sending the next Vsync-App, and after the timer expires, the Vsync-APP corresponding to the next input event is fed back to the application main thread.

[0177] It can be understood that after receiving the Vsync-App request from the image synthesis system, the application main thread processes the current input event. This Vsync-App request is based on the Vsync-App request sent by the application main thread in the previous input event. It can also be understood that the Vsync-App request sent in this input event is used to request the processing of the Vsync-App corresponding to the next input event.

[0178] For example, take multiple input events as an example, namely down event, move event and up event, and if the images corresponding to the three input events are frame 1, frame 2 and frame 3. When the application main thread receives the second input information (move event), the application main thread sends a Vsync-APP request to the image synthesis system. After the timer ends, the image synthesis system generates a Vsync-App and feeds back the first Vsync-App to the application main thread. Vsync-APP wakes up the application main thread. After being woken up, the application main thread processes the move event sent by the input system.

[0179] S606: After the timer expires, the image synthesis system sends a Vsync-APP message representing the last input event request to the application main thread. This Vsync-APP message is used to wake up the application main thread. After waking up, the application main thread begins processing the move event. Optionally, the application main thread records the timestamp of the Vsync-APP message.

[0180] In a possible implementation, when the application main thread receives a Vsync-APP, it saves the Vsync-APP timestamp and calculates the frame interval, which is the difference between the Vsync-APP timestamp and the previous Vsync-APP timestamp.

[0181] S607: When receiving the second input information (move event), the application main thread records the second input information and determines whether the application interface includes a list control.

[0182] Exemplarily, the list control may be a listview or a recyleview, and the embodiment of the present application does not limit the list control.

[0183] S608: When the application main thread determines that the application interface includes a list control, the application main thread sends the second input information to the sliding control. The sliding control records the second input information. In addition, the application main thread also executes S609.

[0184] S609: Apply the main thread to calculate the difference between the second touch coordinates and the first touch coordinates to obtain a first displacement value.

[0185] S608 and S609 can be executed simultaneously, and the embodiment of the present application does not limit the order of S608 and S609.

[0186] S610: The application main thread sends the first displacement value and the timestamp of the Vsync-APP signal to the rendering thread. The timestamp of the Vsync-APP signal is used to wake up the rendering process. The first displacement value is used by the rendering thread to determine the layout of the image.

[0187] S611 : After receiving the first displacement value and Vsync-APP information, the rendering thread starts drawing a rendered image.

[0188] S612: The rendering thread sends the rendered image to the image synthesis system.

[0189] S613: After receiving the rendered image, the image synthesis system starts synthesizing the image.

[0190] S614: The image synthesis system sends the synthesized image to the display driver.

[0191] S615 , after receiving the synthesized image, the display driver drives the screen (display screen) to display the synthesized image.

[0192] In a possible implementation, the image synthesis system synthesizes an image and sends the synthesized image to a hardware compositor (HWC), and the hardware compositor sends the synthesized image to a display driver for display.

[0193] It is understandable that during the sliding process, multiple move events may be included, and the application main thread calculates the touch coordinate processing flow of two adjacent move events in a similar manner, which will not be described in detail in the embodiment of the present application.

[0194] The following combination Figure 7 S616-S653 describe the interaction between modules when the input event is an up event.

[0195] S616: The input system parses the third input event to obtain third input information, and sends the third input information to the application main thread. The third input information includes an up event and third touch coordinates.

[0196] For example, Figure 8 For example, if Figure 8 Frame 2 is the frame corresponding to the move event. When frame 2 is being rendered, the third input event (up event) is received.

[0197] S617 . When receiving the third input information (up event), the main application thread sends a Vsync-APP request to the image synthesis system to request processing of the Vsync-APP corresponding to the input event.

[0198] For example, Figure 8 For example, when the application main thread receives the third input information (up event), it sends a Vsync-APP request to the image synthesis system to request Vsync2.

[0199] S618: After the timer expires, the image synthesis system feeds back a Vsync-APP request of the input event (up event) to the application main thread. The Vsync-APP is used to wake up the application main thread.

[0200] For example, Figure 8 For example, after the timer ends, the image synthesis system feeds back Vsync2 of the move event request.

[0201] S619: When the application main thread receives the Vsync-APP, the Vsync-APP timestamp is saved, and a frame interval is calculated based on the Vsync-APP timestamp and the Vsync-APP timestamp of the last input event.

[0202] For example, Figure 8 For example, if the screen refresh rate is 120 Hz, when the application main thread receives Vsync2, it saves the timestamp of Vsync2 and calculates the frame interval based on the timestamp of Vsync2 and the timestamp of Vsync1. The frame interval is 8.3ms.

[0203] It is understood that the frame interval is the difference between the Vsync-APP timestamp and the Vsync-APP timestamp corresponding to the move event. In a possible implementation, if there are multiple move events, the frame interval is the difference between the current Vsync-APP timestamp and the Vsync-APP timestamp corresponding to the most recent move event.

[0204] S620: When receiving the third input information (up event), the application main thread records the third input information and determines whether the application interface includes a list control.

[0205] Exemplarily, the list control may be a listview or a recyleview, and the embodiment of the present application does not limit the list control.

[0206] S621: When the application main thread determines that the application interface includes a list control, the third input information is sent to the sliding control. The sliding control records the third input information and confirms that the input event is an up event.

[0207] S622: When the sliding control confirms that the input event is an up event, the sliding control calculates a frame displacement, where the frame displacement is the difference between the third touch coordinate and the second touch coordinate corresponding to the last move event.

[0208] In addition, the sliding control also sends the frame displacement through the application main thread to the rendering thread for drawing and rendering. The rendering thread draws the image corresponding to the rendering up event based on the frame displacement. The rendered image corresponding to the up event is sent to the image synthesis system. The image synthesis system synthesizes the image and sends the synthesized image corresponding to the up event to the display driver. The display driver drives the screen (display) to display the synthesized image corresponding to the up event. ( Figure 7 (not shown)

[0209] Alternatively, the application main thread further calculates the difference between the third touch coordinates and the second touch coordinates corresponding to the last move event, and sends the difference to the rendering thread for drawing and rendering. The rendering thread draws the rendered image and sends the rendered image to the image synthesis system. The image synthesis system synthesizes the image and sends the synthesized image to the display driver. The display driver drives the screen (display) to display the synthesized image. ( Figure 7 (not shown)

[0210] S623: When the sliding control confirms that the input event is an up event, the sliding control queries the application main thread for a frame interval, and the application main thread adaptively sends the frame interval to the sliding control.

[0211] S624: The sliding control calculates the hand-off sliding speed based on the frame interval and the frame displacement, and determines whether the hand-off sliding speed reaches the hand-off threshold.

[0212] The release slide speed is the ratio of the frame displacement to the frame interval. The frame displacement is the difference between the third touch coordinate and the second touch coordinate corresponding to the last move event.

[0213] S625: When the hand-off sliding speed is greater than or equal to the speed threshold, the sliding control sends a first command to the application main thread, where the first command is used to instruct the sliding control to trigger the hand-off sliding process and to request a Vsync-APP signal.

[0214] S626 . The application main thread sends a Vsync-APP request to the image synthesis system to request a Vsync-APP signal.

[0215] For example, Figure 8 For example, the application main thread sends a Vsync-APP request to the image synthesis system to request Vsync3.

[0216] S627: When the timer in the image synthesis system expires, the image synthesis system feeds back the Vsync-APP signal requested in S626 to the application main thread. Optionally, the application main thread records the timestamp of the Vsync-APP signal requested in S626 fed back by the image synthesis system.

[0217] For example, Figure 8 For example, the image synthesis system feeds back Vsync3 to the application main thread; the application main thread records the timestamp of Vsync3.

[0218] S628. After receiving the Vsync-APP signal requested in S617, the application main thread determines that a hand-off slide has been triggered, and sends a second command to the slide control, where the second command is used to instruct the slide control to call a hand-off slide module.

[0219] S629: The sliding control queries the application main thread for the timestamp of the Vsync-APP signal.

[0220] For example, Figure 8 For example, the sliding control queries the Vsync3 timestamp from the application main thread.

[0221] S630, the application main thread sends the timestamp of the Vsync-APP signal to the sliding control. Figure 8 For example, the main thread of the application sends the timestamp of Vsync3 to the sliding control.

[0222] S631. The sliding control calls the off-hand sliding module, calculates the total time and total displacement of the off-hand sliding according to the formula preset in the off-hand sliding module, and calculates the second displacement value and speed value according to the timestamp of the Vsync-APP signal and the normalized displacement curve stored in the off-hand sliding module. The second displacement value is the displacement relative to the third touch coordinate. For example, Figure 8 For example, the sliding control calculates the total time and total displacement of the hand-off sliding according to the pre-set formula in the hand-off sliding module, and calculates the second displacement value corresponding to frame 4 and the speed value corresponding to frame 4 according to the timestamp of Vsync3 and the normalized displacement curve stored in the hand-off sliding module.

[0223] It should be noted that the sliding control includes a preset formula and a normalized displacement curve. The preset formula is used to calculate the total displacement and total time corresponding to the hand-off sliding according to the hand-off sliding speed.

[0224] For example, taking the preset formula, total displacement (total distance) = 0.349 * sliding speed, total time (total fling time) = 0.125 * sliding speed, as an example, when the sliding speed is 8162 pixels (pixel) / second (S), the total displacement is 2853.56 pixels, and the total time is 1020 milliseconds (ms).

[0225] The normalized displacement curve is used to calculate a corresponding second displacement value and a corresponding velocity value according to the current frame time.

[0226] In one possible implementation, the normalized displacement curve is stored in a slider control as discrete points. For example, the slider control is pre-set with 100 discrete points, each ranging from 0 to 1. The curve formed by the discrete points has a trend of a cubic curve.

[0227] For example, Figure 9 A schematic diagram of a normalized displacement curve provided in an embodiment of the present application. Figure 9 The normalized displacement curve consists of 100 discrete points. The normalized displacement curve increases with the increase of the horizontal coordinate. The normalized displacement curve composed of 100 discrete points can be fitted as y = 0.000001x 3 -0.000330x 2 +0.030640x+0.000216.

[0228] For example, Table 1 shows Figure 9 The 100 discrete points in the normalized displacement curve are shown in Table 1. The discrete points are represented in the terminal device as index numbers and the normalized displacement values ​​corresponding to the index numbers. For example, the normalized displacement value corresponding to index number 1 is 0.028561; the normalized displacement value corresponding to index number 2 is 0.057052.

[0229] Table 1 100 discrete points

[0230]

[0231] The calculation of the second displacement value and the corresponding velocity value is described below.

[0232] The second displacement value is the product of the normalized displacement corresponding to the frame and the total displacement. The normalized displacement corresponding to the frame is the displacement value corresponding to the normalized time corresponding to the frame on the normalized displacement curve. The normalized time corresponding to the frame is the ratio of the first time difference to the total time. The first time difference is the difference between the Vsync-APP timestamp corresponding to the frame and the time of the up event. The speed value is the product of the normalized speed and the average speed. The normalized speed is the slope corresponding to the normalized time of the frame on the normalized displacement curve. The average speed is the ratio of the total displacement to the total time.

[0233] Since the normalized curve consists of 100 discrete points, the normalized time corresponding to that frame is the first time difference / total time*100. The calculated normalized time may not correspond to a discrete point. Calculate the normalized velocity corresponding to that frame and then calculate the normalized displacement.

[0234] For example, taking the total displacement as 2853.56 pixels and the total time as 1020 ms as an example, if the first time difference is 11 ms, the normalized time corresponding to the frame is That is, 1.0784314. 1.0784314 is between index 1 and index 2.

[0235] The normalized speed corresponding to this frame is the average speed between index1 and index2. That is 0.028490958.

[0236] The normalized displacement is: 0.028561+0.028490958×(1.0784314-1), which is 0.030795585.

[0237] The second displacement value is: 0.030795585×2853.56, which is 87.8906021118164 pixels.

[0238] The speed value is the ratio of the product of normalized speed and total displacement to total time. That is, the speed value of the current frame is Right now That is 7.971882 pixels / ms.

[0239] S632: The sliding control sends the window (activity) name and the second displacement value to the data manager for storage.

[0240] For example, Figure 8 For example, the sliding control sends the second displacement value corresponding to frame 4 to the data manager for storage.

[0241] For example, taking the sliding interface as the settings interface, the window name can be the main layer name of the settings interface (for example, com.android.settings / com.android.settings.HNSettings#0), the speed value can be 7.971882pixel / ms, and the second displacement value can be 87.8906021118164pixel.

[0242] In a possible implementation, the sliding control further sends the speed value to the data manager for storage.

[0243] S633: The data manager saves the window name and the second offset value.

[0244] In a possible implementation, the slider control can obtain the window name from the application main thread. The slider control can also obtain the window name from the activity manager.

[0245] S634: The sliding control sends the second displacement value and speed value to the application main thread.

[0246] For example, Figure 8For example, the sliding control sends the second displacement value corresponding to frame 4 and the speed value corresponding to frame 4 to the main thread of the application.

[0247] S635: The application main thread sends the second displacement value to the rendering thread.

[0248] For example, Figure 8 For example, the application main thread sends the second displacement value corresponding to frame 4 to the rendering thread. S636: The rendering thread draws a rendered image according to the second displacement value.

[0249] For example, Figure 8 For example, the rendering thread draws the image of frame 4 according to the second displacement value corresponding to frame 4.

[0250] S637: The rendering thread sends the rendered image to the image synthesis system.

[0251] S638: The image synthesis system synthesizes the rendered image.

[0252] S639: The image synthesis system sends the synthesized image to the display driver for screen display.

[0253] S640: The display driver drives the screen (display screen) to display the synthesized image.

[0254] S641: When the speed value sent by the slider control to the main application thread is not zero or the second displacement value changes, the main application thread continues to request Vsync-APP and loops through the above corresponding operations until the speed value sent by the slider control to the main application thread is zero or the second displacement value remains unchanged. The slider control calculates the speed value and the second displacement value for each frame.

[0255] For example, Figure 8 For example, the speed value corresponding to frame 4 is not 0, and the main thread of the application sends a Vsync-APP request to request Vsync4. After the timer expires, the image synthesis system feeds back the requested Vsync4. The image synthesis system feeds back Vsync4 to the main thread of the application; the main thread of the application records the timestamp of Vsync4. The sliding control calculates the total time and total displacement of the hand-off sliding according to the pre-set formula in the hand-off sliding module, and calculates the second displacement value corresponding to frame 5 and the speed value corresponding to frame 5 according to the timestamp of Vsync4 and the normalized displacement curve stored in the hand-off sliding module. The main thread of the application sends the second displacement value corresponding to frame 5 to the rendering thread. The rendering thread draws the image of frame 5 according to the second displacement value corresponding to frame 5, and sends the rendered image to the image synthesis system for synthesis, and sends it to the display after synthesis.

[0256] When the speed value sent by the sliding control to the application main thread is zero or the second displacement value remains unchanged, the application main thread stops sending the Vsync-APP request.

[0257] S642: When the second speed value calculated by the sliding control is zero or the second displacement value remains unchanged, the sliding control sends a command to the data manager for indicating the end of sliding.

[0258] S643. The data manager counts the number of off-hand sliding of the window corresponding to the window name and the N hotspot displacement values ​​of the window corresponding to the window name according to the window name. The N hotspot displacement values ​​are the first N second displacement values ​​in the multiple off-hand sliding, sorted from high to low according to the number of occurrences. For example, for example, the second displacement values ​​in the first off-hand sliding counted are 50, 75, 95, 105, 110, and 115 respectively; and the second displacement values ​​in the first off-hand sliding counted are 50, 80, 100, 110, 115, and 118 respectively. If N is 3, the hotspot displacement values ​​are 50, 110, and 115 respectively. The rendered image is subsequently pre-drawn based on 50, 110, and 115.

[0259] S644: The sliding control further queries the data manager whether there are N hotspot displacement values.

[0260] In a possible implementation, S643 and S644 may be executed simultaneously.

[0261] S645: If the number of hand-off slides for the activity corresponding to the activity name counted by the data manager reaches a threshold, N hotspot displacement values ​​are fed back to the slide control.

[0262] In a possible implementation, if the number of hand-off sliding counted by the data manager does not reach a threshold number, a command indicating that there is no hotspot displacement value is fed back to the sliding control.

[0263] S646: When the sliding control receives N hotspot displacement values, it sends a command for instructing to increase the cache to the image synthesis system.

[0264] S647: The image synthesis system expands the buffer queue and adds a flag in the expanded buffer, the flag being used to indicate not to send for display. It is understandable that the expanded buffer is used to store pre-rendered frames, which can also be called predicted frames.

[0265] S648. The sliding control adds the N hotspot displacement values ​​and the second displacement value corresponding to zero speed to obtain N third displacement values.

[0266] S649: The sliding control sends the N third displacement values ​​to the list control in the main thread of the application. It is understandable that the list control in the main thread of the application is pre-rendered based on the still image after the sliding is released.

[0267] S650: The list control in the application main thread sends N third displacement values ​​to the rendering thread.

[0268] S651: The rendering thread renders N images in sequence.

[0269] In a possible implementation, the rendering thread sequentially draws and renders 2N images. It is understood that the rendering thread draws images in two directions. For example, the list control can slide up or down, and the rendering thread draws the N images that slide up and the N images that slide down.

[0270] For example, Figure 8 As shown, after the sliding is completed, the rendering thread in the application sequentially draws the images corresponding to the hotspot displacement values. Taking the hotspot displacement values ​​of 50, 110, and 115, and the second displacement value of 120 when the speed is zero as an example, after the sliding is completed, the images corresponding to 170, 230, and 235 are drawn and rendered respectively. The drawn and rendered images corresponding to 170, 230, and 235 are stored in the cache queue and are not synthesized. The drawn and rendered images corresponding to 170, 230, and 235 correspond to the three hotspot displacement values ​​of 50, 110, and 115 respectively.

[0271] S652: The rendering thread sends N rendered images to the image synthesis system.

[0272] S653: The image synthesis system saves the rendered image as a prediction frame in a cache queue, and the image synthesis system also saves a mapping table of hotspot displacement values ​​and prediction frames.

[0273] In a possible implementation, the rendering thread may convert the third displacement value into a hotspot displacement value and mark the N images to be rendered. The image synthesis system stores a mapping table of the hotspot displacement value and the predicted frame according to the mark.

[0274] In a possible implementation, after the application's main thread sends N predicted frames to the image synthesis system, it resets the coordinates of the sliding module. This can be understood as resetting the displacement corresponding to the current interface to 0. This way, the base displacement used during the next sliding operation remains unchanged, reducing the computational effort involved.

[0275] In a possible implementation, when an application receives an input event during pre-rendering, the application stops pre-rendering.

[0276] For ease of understanding, the following Figure 8This section describes the pre-rendering process after the up event.

[0277] like Figure 8 As shown, when the application is rendering frame 2, it receives an up event and renders frame 3 according to the touch coordinates corresponding to the up event. When the sliding speed of the up event is greater than or equal to the speed threshold, the application calculates multiple images after the up event according to the preset formula and normalized displacement curve. The application renders frames 4 and 5, etc. When the hand-off sliding ends, the application pre-draws and renders multiple images based on the hotspot displacement value, and saves the pre-drawn and rendered images as predicted frames in the cache queue. The pre-drawn and rendered images are not synthesized.

[0278] Figure 10 This is a diagram of the module interactions involved in displaying predicted frames in conjunction with the data processing method provided in an embodiment of the present application. The modules involved include: the input system, the application, the data manager, the image synthesis system (surface flinger), the display driver, and the data manager. The application includes: the application main thread, the slider control, and the rendering thread.

[0279] When the terminal device receives the user's sliding operation,

[0280] S1001: If the input information obtained by parsing the input event by the input system is third input information, the input system sends the third input information to the main thread of the application. The third input information includes an up event and third touch coordinates.

[0281] S1002 : When receiving the third input information (up event), the main application thread sends a Vsync-APP request to the image synthesis system to request processing of the Vsync-APP corresponding to the input event (up event).

[0282] S1003: After the timer expires, the image synthesis system feeds back a Vsync-APP request of an input event (up event) to the application main thread. The Vsync-APP is used to wake up the application main thread.

[0283] S1004: When the application main thread receives Vsync-APP, it saves the Vsync-APP timestamp and calculates the frame interval.

[0284] S1005. The application main thread records the third input information and determines whether the application interface includes a list control. The list control can be a listview or a recyleview. The embodiment of the present application does not limit the list control.

[0285] S1006: When the application main thread determines that the application interface includes a list control, the third input information is sent to the sliding control, and the sliding control records the third input information.

[0286] S1007: When the sliding control confirms that the input event is an up event, the sliding control queries the application main thread for a frame interval.

[0287] S1008. The application main thread sends the frame interval to the sliding control.

[0288] S1009: The sliding control calculates the sliding speed according to the frame interval and the frame displacement, and determines whether the sliding speed reaches the hand-off threshold.

[0289] The explanation of S1001-S1009 can refer to the relevant explanation of S616-S624 mentioned above, which will not be repeated here.

[0290] S1010: When the hand-off sliding speed is greater than or equal to the speed threshold, the sliding control queries the image synthesis system whether there is a predicted frame.

[0291] For example, Figure 11 There are multiple predicted frames in the cache queue of the image synthesis system, and each predicted frame corresponds to a hotspot displacement value. The hotspot displacement value can be 50, 110, and 115.

[0292] S1011 : When there is a predicted frame in the cache queue of the image synthesis system, an instruction indicating that there is a predicted frame is fed back to the sliding control.

[0293] S1012: When the sliding speed is greater than or equal to the speed threshold, the sliding control further sends a first command to the application main thread, where the first command is used to instruct the sliding control to trigger a hand-off sliding process and request a Vsync-APP signal.

[0294] S1013. The application main thread sends a Vsync-APP request to the image synthesis system.

[0295] S1014: When the timer in the image synthesis system expires, the image synthesis system feeds back the Vsync-APP signal requested in S1013 to the application main thread. Optionally, the application main thread records the timestamp of the Vsync-APP signal requested in S1013 fed back by the image synthesis system.

[0296] For example, Figure 11 For example, the image synthesis system feeds back Vsync3 to the application main thread; the application main thread records the timestamp of Vsync3.

[0297] S1015. After receiving the Vsync-APP signal requested in S1002, the application main thread determines that a hand-off slide has been triggered, and sends a second command to the slide control, where the second command is used to instruct the slide control to call a hand-off slide module.

[0298] S1016: The sliding control queries the application main thread for the timestamp of the Vsync-APP signal.

[0299] For example, Figure 11 For example, the sliding control queries the Vsync3 timestamp from the application main thread.

[0300] S1017. The application main thread sends the timestamp of the Vsync-APP signal to the sliding control.

[0301] For example, Figure 11 For example, the main thread of the application sends the timestamp of Vsync3 to the sliding control.

[0302] S1018. The sliding control calls the off-hand sliding module, and calculates the total time and total displacement of the off-hand sliding according to the formula pre-set in the off-hand sliding module, and calculates the second displacement value and speed value according to the timestamp of the Vsync-APP signal and the normalized displacement curve.

[0303] The second displacement value and the speed value can refer to the description of S631 above, which will not be repeated here.

[0304] For example, Figure 11 For example, the sliding control calculates the second displacement value corresponding to frame 4 and the speed value corresponding to frame 4.

[0305] S1019: The sliding control sends the window (activity) name and the second displacement value to the data manager for storage.

[0306] For example, Figure 11 For example, the sliding control sends the window (activity) name and the second offset value to the data manager for storage.

[0307] S1020: The data manager saves the window (activity) name and the second offset value.

[0308] For example, Figure 11 As shown, the rendering thread draws the image of frame 4 according to the second displacement value corresponding to frame 4.

[0309] S1021. After calculating the second displacement value and the speed value, the sliding control calculates a fourth displacement value and sends the fourth displacement value to the image synthesis system.

[0310] The fourth displacement value is the sum of the second displacement value and the fifth displacement value, and the fifth displacement value is the difference between the third touch coordinate corresponding to the up event in this sliding operation and the first touch coordinate corresponding to the down event in this sliding operation.

[0311] For example, Figure 11 For example, the sliding control sends the fourth displacement value corresponding to frame 4 to the image synthesis system.

[0312] S1022. Determine whether there is a hotspot displacement value matching the fourth displacement value in the cache queue of the image synthesis system. When there is a matching hotspot displacement value in the cache queue, the image synthesis system synthesizes a predicted frame corresponding to the hotspot displacement value matching the fourth displacement value.

[0313] For example, Figure 11 As shown, if the fourth displacement value corresponding to frame 4 is 50, there is a predicted frame with a hotspot displacement value of 50 in the cache queue, and the image synthesis system synthesizes the predicted frame with the hotspot displacement value of 50.

[0314] In a possible implementation, when the difference between the fourth displacement value and any hotspot displacement value is less than or equal to a first threshold, a matching hotspot displacement value exists in the cache queue of the image synthesis system. When the difference between the fourth displacement value and any hotspot displacement value is greater than the first threshold, no matching hotspot displacement value exists in the cache queue of the image synthesis system.

[0315] In a possible implementation, there are multiple frames to be synthesized in the cache queue, and the image synthesis system synthesizes the frames to be synthesized and the predicted frames corresponding to the hotspot displacement value matching the fourth displacement value in sequence.

[0316] S1023: When there is a matching hotspot displacement value in the cache queue of the image synthesis system, the image synthesis system sends an instruction to the sliding control for indicating that there is a matching prediction frame.

[0317] S1024: The sliding control calculates a second displacement value and a speed value of the next frame according to a preset formula and the normalized displacement curve. The sliding control calculates a fourth displacement value of the next frame and sends the fourth displacement value to the image synthesis system.

[0318] It is understandable that the timestamp of the Vsync signal increases by one Vsync signal cycle time, that is, the first time difference increases by one Vsync signal cycle time. The calculation process can refer to the above related description.

[0319] It is understood that the slider sends the fourth displacement value corresponding to the next frame to the image synthesis system. When a hotspot displacement value matching the fourth displacement value of the next frame is found in the image synthesis system's cache queue, the image synthesis system synthesizes a predicted frame corresponding to the hotspot displacement value matching the fourth displacement value of the next frame. The calculation loop executes S1018-S1023 until the second displacement value remains unchanged, the speed value is zero, or there are no matching hotspot displacement values ​​in the image synthesis system's cache queue.

[0320] For example, Figure 11 As shown, if the fourth displacement value corresponding to frame 4 is 50, there is a predicted frame with a hotspot displacement value of 50 in the cache queue, and the image synthesis system synthesizes the predicted frame with the hotspot displacement value of 50.

[0321] The slider control calculates the second displacement value corresponding to frame 5, the velocity value corresponding to frame 5, and the fourth displacement value corresponding to frame 5 based on a preset formula and the normalized displacement curve. The slider control sends the second displacement value corresponding to the activity name frame 5 to the data manager. The data manager saves the second displacement value corresponding to the activity name frame 5.

[0322] The sliding control also sends the second displacement value corresponding to frame 5 to the image synthesis system to confirm whether there is a hotspot displacement value that matches the second displacement value of frame 5.

[0323] If there is a hotspot displacement value matching the fourth displacement value of frame 6 in the cache queue of the image synthesis system, the sliding control calculates the fourth displacement value of frame 6 and the speed value of frame 6 according to the preset formula and the normalized displacement curve.

[0324] If there is no hotspot displacement value matching the fourth displacement value of frame 5 in the cache queue of the image synthesis system, the image synthesis system executes S1025.

[0325] It is understandable that the sliding control sends the fourth displacement value corresponding to the next frame to the image synthesis system. When there is no hotspot displacement value matching the second displacement value corresponding to the next frame in the cache queue of the image synthesis system, S1025 is executed.

[0326] If there is no matching hotspot displacement value in the cache queue of the image synthesis system, the image synthesis system executes S1025.

[0327] S1025. The image synthesis system sends an instruction to the sliding control to indicate that there is no matching prediction frame.

[0328] S1026 . When the sliding control receives the instruction of no matching predicted frame, the sliding control sends the second displacement value of the next frame and the speed value of the next frame to the main thread of the application.

[0329] S1027. The application main thread sends the received second displacement value of the next frame to the rendering thread.

[0330] In a possible implementation, a list control in the application's main thread sends the second displacement value for the next frame to the rendering thread. The rendering thread then draws a rendered image based on the second displacement value for the next frame. The rendering thread then sends the rendered image to the image compositing system for compositing. The image compositing system then sends the compositing image to the display driver for screen display. The display driver then drives the screen (display) to display the compositing image.

[0331] For example, Figure 11 As shown, the application main thread sends the received second displacement value of frame 5 to the rendering thread, and the rendering thread draws the rendering frame 5.

[0332] In a possible implementation, when the fourth displacement value calculated by the slider control is greater than the maximum of the hotspot displacement values, the slider control notifies the image synthesis system to discard the predicted frame. Alternatively, when the hand is released from the slider control, the slider control notifies the image synthesis system to discard the predicted frame.

[0333] S1028. When the speed value sent by the sliding control to the application main thread is not zero or the second displacement value changes, the application main thread continues to request Vsync-APP and loops through the above corresponding operations (S1014-S1024) until the speed value sent by the sliding control to the application main thread is zero or the second displacement value remains unchanged.

[0334] S1029: The sliding control calculates the speed value and the second displacement value of each frame. When the second speed value calculated by the sliding control is zero or the second displacement value remains unchanged, the sliding control sends a command to the data manager for indicating the end of sliding.

[0335] S1030: When the second displacement value remains unchanged or the speed is zero, the data manager updates the hotspot displacement value.

[0336] In a possible implementation, if the terminal device cannot receive an up event in a sliding operation within a preset time, the application notifies the image synthesis system to discard the predicted frame.

[0337] The data processing method of the embodiment of the present application is described in detail below through specific embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0338] For example, Figure 12 Schematic diagram of the data processing method provided in the embodiment of the present application. Figure 12 As shown, the method may include:

[0339] S1201: Receive a sliding operation.

[0340] In a possible implementation, the terminal device may receive a sliding operation by the user on the touch screen using a hand or a stylus, or may receive a sliding operation by the user scrolling or dragging with a mouse.

[0341] S1202: Obtain position information of the sliding operation on the display screen.

[0342] Specifically, the input system of the application framework layer in the terminal device receives and analyzes the input event to obtain the position information of the sliding operation on the display screen.

[0343] S1203: Determine whether the interface corresponding to the sliding operation includes a list control.

[0344] Specifically, the application of the terminal device determines that the interface corresponding to the sliding operation includes a list control.

[0345] In the embodiment of the present application, the list control is a collection of ordered content, which is used for horizontal or vertical sliding. Specifically, the list control can also be divided into a horizontal list control or a vertical list control. The list control can include any of the following: controls in the application interface, controls in the setting interface, controls in the e-book reading interface or controls in the document interface, etc. For example, the list control can be Figure 3 The part of the social application interface shown in a that can slide along with the user's finger, or Figure 3 The part of the setting related interface shown in b in the figure that can slide along with the user's finger, Figure 3 The part of the document interface shown in c that can slide along with the user's finger, or Figure 3 The part of the product browsing interface shown in d in FIG. 1 can slide along with the user's finger.

[0346] It is understandable that the terminal device can determine whether the instantiation of the object is a list control such as a list view (listview) or a recycler view (recyclerview). If the instantiation of the object is a list control such as a list view (listview) or a recycler view (recyclerview), the terminal device determines that the interface corresponding to the sliding operation includes a list control.

[0347] S1204: When the hand-off sliding speed reaches a speed threshold, determine to initiate the hand-off sliding.

[0348] Specifically, the application of the terminal device determines to initiate the hands-off sliding when the hands-off sliding speed reaches a speed threshold.

[0349] In the embodiment of the present application, the hand-off sliding speed is the sliding speed when the user's finger or stylus leaves the screen. Specifically, the terminal device calculates the hand-off sliding speed when the input event is an up event.

[0350] In a possible implementation, the release sliding speed is the ratio of the frame displacement to the frame interval. The frame displacement is the difference between the touch coordinates corresponding to the up event and the touch coordinates corresponding to the most recent move event.

[0351] It can be understood that when the hand-off sliding speed is greater than or equal to the hand-off threshold, it is determined that the hand-off sliding is initiated.

[0352] S1205 : Calculate the displacement value according to a preset formula and a pre-stored normalized displacement curve.

[0353] In an embodiment of the present application, the calculated displacement value is the displacement value relative to the down event. A pre-set formula is used to calculate the total displacement and total time based on the hand-off sliding speed. The terminal device calculates the displacement value relative to the up event based on the total displacement and total time, as well as a pre-stored normalized displacement curve, and then calculates the displacement value relative to the down event. In a possible implementation method, the terminal device inputs the hand-off sliding speed into a pre-set formula to obtain the total displacement and total time. The terminal device inputs the total displacement and total time, as well as the time difference between the current Vsync signal time and the up event, into a pre-stored normalized displacement curve to obtain the displacement value of the frame relative to the up event.

[0354] S1206. When the displacement value relative to the down event matches the hotspot displacement value corresponding to the predicted frame, synthesize the predicted frame. The hotspot displacement value is the top N displacement values ​​ranked in descending order based on the number of occurrences determined based on the historical sliding operations. The predicted frame is a frame pre-rendered based on the historical sliding operations. N is a natural number.

[0355] It can be understood that when the terminal device slides away from the hand, it saves multiple displacement values ​​corresponding to the sliding. After the sliding is completed, the multiple displacement values ​​corresponding to the multiple sliding are statistically sorted from large to small according to the number of occurrences, and the top N displacement values ​​are ranked as hotspot displacement values.

[0356] In a possible implementation, the hotspot displacement value is an integer. For example, the finger sliding distance is generally 10 Pixel / s–150 Pixel / s. If the hotspot displacement value is 71, the unit digit can be discarded and the hotspot displacement value can be rounded to 70.

[0357] For example, Figure 13 A schematic diagram of a display process provided in an embodiment of the present application. Figure 13 The contents displayed in frames 0, 1, 2, 3, 4, and 5 correspond to different interfaces. Figure 11 When the terminal device receives an upward swipe from the user within the application interface, the content displayed by the terminal device upon lifting the swipe corresponds to frames 0, 1, 2, predicted frame 3, 4, and 5, respectively. Before receiving the swipe operation, the terminal device caches N predicted frames, namely predicted frame 1 through predicted frame N. Each predicted frame corresponds to a hotspot displacement value. For example, the hotspot displacement values ​​for predicted frames 1, 2, and 3 are 50, 110, and 115, respectively.

[0358] Figure 13 In the example, after frame 2 is rendered, the terminal device calculates the displacement value of frame 3 relative to the down event as 115. This displacement value matches the predicted hotspot displacement value for frame 3. The device then calculates the displacement value for the next frame (frame 4) and renders the next frame (i.e., frame 4). While frame 4 is being rendered, the terminal device's cache queue contains a frame to be synthesized (i.e., predicted frame 3). When the Vsync signal arrives, predicted frame 3 is synthesized.

[0359] If the rendering of frame 5 times out, it will be completed within 2 Vsync cycles. Since one frame of rendering time is saved, there is a frame to be synthesized (predicted frame 4) in the cache queue of the terminal device, and the display of the terminal device is smooth without any lag.

[0360] It is understood that if there is a matching predicted frame, one frame of rendering time can be saved, allowing the application to timeout and reduce the number of frames without available frames. If multiple predicted frames are matched, multiple frames of rendering time can be saved, allowing multiple frames of rendering timeout.

[0361] In this way, the image is pre-rendered to reduce the situation where no frames are available due to rendering timeout, reduce display jams, and increase display smoothness.

[0362] In a possible implementation, when the displacement value and the hotspot displacement value are within a first threshold range, a predicted frame is synthesized. The embodiment of the present application does not limit the value of the first threshold.

[0363] In a possible implementation, there are frames to be synthesized in the buffer queue of the terminal device. After synthesizing the frames to be synthesized, the predicted frames are synthesized. This reduces the need for sequential synthesis, reduces interface display confusion, and improves user experience.

[0364] In a possible implementation, if the calculated displacement value is greater than the hotspot displacement value, the terminal device discards the predicted frame.

[0365] S1207: Display the predicted frame.

[0366] In summary, the present embodiment pre-renders the interface based on the hotspot displacement values ​​obtained from historical sliding operations to generate a predicted frame. When the displacement point of the release slide matches the hotspot displacement value, the predicted frame is synthesized. This reduces the lag caused by rendering timeouts and improves the user experience.

[0367] The data processing method according to the embodiment of the present application has been described above. The electronic device provided in the embodiment of the present application for executing the above data processing method is described below. Those skilled in the art will appreciate that the methods and apparatuses may be combined and referenced with each other, and the electronic device provided in the embodiment of the present application may execute the steps in the above data processing method.

[0368] like Figure 14 As shown, Figure 14 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application is shown. The data processing device may be an electronic device in an embodiment of the present application. The data processing device includes: a display screen 1401 for displaying images; one or more processors 1402; a memory 1403; multiple application programs; and one or more computer programs, wherein the one or more computer programs are stored in the memory 1404. The one or more computer programs include instructions that, when executed by the data processing device, cause the data processing device to perform the steps of the above-described data processing method.

[0369] Figure 15 This is a hardware structure diagram of a data processing device provided in an embodiment of the present application. Figure 15 The device includes: a memory 1501, a processor 1502, and an interface circuit 1503. The device may also include a display screen 1504, wherein the memory 1501, the processor 1502, the interface circuit 1503, and the display screen 1504 can communicate; illustratively, the memory 1501, the processor 1502, the interface circuit 1503, and the display screen 1504 can communicate via a communication bus, the memory 1501 is used to store computer-executable instructions, the execution of which is controlled by the processor 1502, and the communication is performed by the interface circuit 1503, thereby implementing the data processing method provided in the embodiment of the present application.

[0370] Optionally, the interface circuit 1503 may further include a transmitter and / or a receiver. Optionally, the processor 1502 may include one or more CPUs, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in this application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0371] In possible implementations, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0372] The data processing device provided in the embodiment of the present application is used to execute the data processing method of the above embodiment. The technical principles and technical effects are similar and will not be repeated here.

[0373] The present application provides an electronic device, the structure of which is shown in FIG. Figure 1 The memory of the electronic device may be used to store at least one program instruction, and the processor may be used to execute at least one program instruction to implement the technical solution of the above method embodiment. The implementation principle and technical effects thereof are similar to those of the above method-related embodiments and will not be described in detail here.

[0374] The present embodiment provides a chip. The chip includes a processor configured to invoke a computer program stored in a memory to execute the technical solution of the above embodiment. The implementation principles and technical effects are similar to those of the above-mentioned related embodiments and will not be further described here.

[0375] The present application provides a computer program product that, when executed on an electronic device, enables the electronic device to execute the technical solution in the above embodiment. The implementation principle and technical effects are similar to those of the above-mentioned related embodiments and will not be described in detail here.

[0376] The present embodiment provides a computer-readable storage medium having program instructions stored thereon. When the program instructions are executed by an electronic device, the electronic device executes the technical solution of the above embodiment. The implementation principle and technical effects thereof are similar to those of the above-mentioned related embodiments and will not be further described here.

[0377] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0378] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: Applied to electronic equipment, the method includes: In response to an up event of a sliding operation, calculating a sliding speed of the list control; wherein the sliding operation acts on the list control; When the sliding speed of the list control is greater than or equal to a speed threshold, calculating a displacement value of the first image, and confirming whether a rendering result of the first image is stored in a cache queue based on the displacement value and a mapping relationship, the first image being an image of the electronic device after receiving the up event, the displacement value being a displacement relative to a press-down event in the sliding operation, the mapping relationship being a relationship between the rendering results of multiple images stored in the cache queue and multiple hotspot displacement values, the hotspot displacement values ​​being the top N displacement values ​​in descending order of appearance in historical sliding operations; When the rendering result of the first image is stored in the cache queue, the first image is synthesized.

2. The method according to claim 1, characterized in that The confirming, based on the displacement value and the mapping relationship, whether the rendering result of the first image is stored in the cache queue includes: When a difference between the displacement value and any of the hotspot displacement values ​​is less than or equal to a first threshold, confirming that a rendering result of the first image is stored in the cache queue; Alternatively, when the differences between the displacement value and the multiple hotspot displacement values ​​are all greater than the first threshold, it is confirmed that no rendering result of the first image is stored in the cache queue.

3. The method according to claim 1 or 2, characterized in that The calculating the displacement value of the first image includes: Calculating a displacement value relative to the up event based on a pre-set formula in the electronic device and a pre-stored normalized displacement curve, wherein the pre-set formula satisfies: T = A × sliding speed, S = B × sliding speed, where S is the total displacement of the list control after the up event, S is the total time the list control slides after the up event, and A and B are both constants; The displacement value is obtained by calculating a sum of the displacement value relative to the up event and a first displacement, where the first displacement is a difference between the touch coordinates corresponding to the up event and the touch coordinates corresponding to the down event.

4. The method according to claim 3, characterized in that Calculating the displacement value relative to the up event according to a preset formula and a pre-stored normalized displacement curve includes: Substituting the sliding velocity into the formula to calculate the total displacement and the total time; The displacement value relative to the up event is calculated according to the total displacement, the total time and the normalized displacement curve.

5. The method according to claim 4, characterized in that The normalized displacement curve is stored in the electronic device in the form of a plurality of discrete points.

6. The method according to any one of claims 1-2, 4-5, characterized in that: Also includes: When the displacement value is greater than the maximum value of the hotspot displacement values, the rendering results of the multiple frames of images in the cache queue are discarded.

7. The method according to any one of claims 1-2, 4-5, characterized in that: Also includes: After the sliding operation ends, updating the hotspot displacement value; Draw the rendered image based on the updated hotspot displacement value.

8. The method according to any one of claims 1-2, 4-5, characterized in that: Also includes: When the rendering result of the first image is not stored in the cache queue, the first image is rendered.

9. The method according to any one of claims 1-2, 4-5, characterized in that: The electronic device includes: an application and an image synthesis system; The calculating the sliding speed of the list control in response to the up event of the sliding operation includes: The application calculates a sliding speed of the list control in response to an up event of the sliding operation; The step of confirming whether the rendering result of the first image is stored in the cache queue includes: The electronic device confirms whether a rendering result of the first image is stored in the cache queue; Synthesizing the first image includes: synthesizing the first image by the image synthesis system.

10. The method according to claim 9, characterized in that The step of confirming whether the rendering result of the first image is stored in the cache queue includes: The sliding control in the application calculates a displacement value of the first image, where the displacement value is a displacement relative to a down event in the sliding operation; The image synthesis system confirms whether the rendering result of the first image is stored in the cache queue based on the displacement value and the mapping relationship, wherein the mapping relationship is the relationship between the rendering results of multiple images stored in the cache queue and multiple hotspot displacement values; the hotspot displacement value is the top N displacement values ​​in the historical sliding operation sorted from large to small in terms of the number of occurrences.

11. The method according to claim 10, characterized in that The image synthesis system confirms whether the rendering result of the first image is stored in the cache queue based on the displacement value and the mapping relationship, including: When a difference between the displacement value and any of the hotspot displacement values ​​is less than or equal to a first threshold, the image synthesis system confirms that a rendering result of the first image is stored in a cache queue; Alternatively, when the differences between the displacement value and the multiple hotspot displacement values ​​are all greater than the first threshold, the image synthesis system confirms that no rendering result of the first image is stored in the cache queue.

12. The method according to claim 10 or 11, characterized in that The sliding control calculates the displacement value, including: The sliding control calculates a displacement value relative to the up event based on a pre-set formula in the electronic device and a pre-stored normalized displacement curve, wherein the pre-set formula satisfies: T=A×sliding speed, S=B×sliding speed, where S is the total displacement of the list control sliding after the up event, S is the total sliding time of the list control after the up event, and A and B are both constants; The sliding control calculates the sum of the displacement value relative to the up event and a first displacement to obtain the displacement value, where the first displacement is the difference between the touch coordinates corresponding to the up event and the touch coordinates corresponding to the down event.

13. The method according to claim 12, characterized in that The sliding control calculates the displacement value relative to the up event according to a preset formula and a pre-stored normalized displacement curve, including: The sliding control substitutes the sliding speed into the formula to calculate the total displacement and the total time; The sliding control calculates the displacement value relative to the up event according to the total displacement, the total time and the normalized displacement curve.

14. The method according to claim 13, characterized in that The normalized displacement curve is stored in the electronic device in the form of a plurality of discrete points.

15. The method according to any one of claims 10-11, 13-14, characterized in that: Also includes: When the displacement value is greater than the maximum value of the hotspot displacement values, the image synthesis system discards the rendering results of multiple frames of images in the cache queue.

16. The method according to any one of claims 10-11, 13-14, characterized in that: The electronic device further comprises a data manager: The data manager updates the hotspot displacement value after the sliding operation ends; The application draws a rendered image based on the hotspot displacement value updated by the data manager.

17. The method according to claim 9, characterized in that Also includes: When the rendering result of the first image is not stored in the cache queue, the application renders the first image.

18. An electronic device, characterized in that: The electronic device comprises a processor, and the processor is configured to call a computer program in a memory to execute the method according to any one of claims 1 to 17.

19. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 17.

20. A computer program product, characterized in that The device comprises a computer program, which, when executed, causes an electronic device to perform the method according to any one of claims 1 to 17.

21. A chip, characterized in that: The chip includes a processor, and the processor is configured to call a computer program in a memory to execute the method according to any one of claims 1 to 17.

Images