Image prediction method, electronic device, and storage medium

Abstract

The present application provides an image prediction method, electronic device, and storage medium. The prediction method includes: obtaining a drawing instruction stream of a first image frame, obtaining one or more first objects of the first image frame based on a first type of drawing instruction in the drawing instruction stream of the first image frame, wherein the first type of drawing instruction is a scene drawing instruction, and calculating a third motion vector based on the first motion vector and the second motion vector; obtaining a first predicted image frame based on the first motion vector, the third motion vector, and the one or more first objects, and merging the first predicted image frame with the one or more second objects to obtain a second predicted image frame. The prediction method shown in the present application can reduce the resource consumption of a GPU and the GPU when executing drawing instructions, thereby reducing the load on the electronic device.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image frame prediction method, electronic device, and storage medium. Background Art

[0002] As game graphics and special effects become increasingly refined, the performance requirements placed on terminal devices are also increasing. Running games on terminal devices, especially those with heavy loads (such as shooting games), requires a large amount of resources. This can hinder the smooth operation of some games on mobile devices, leading to issues such as frame drops, rapid battery drain, and overheating. Summary of the Invention

[0003] In order to solve the above technical problems, in a first aspect, the present application provides an image prediction method, which is applied to an electronic device, comprising:

[0004] Obtain a drawing instruction stream of a first image frame; illustratively, the first image frame may be attached Figure 2 and the real frame N shown in 5.

[0005] Based on the first type of drawing instructions in the drawing instruction stream of the first image frame, one or more first objects of the first image frame are obtained, where the first type of drawing instructions are scene drawing instructions; the first objects may be object 502, object 504, object 506, and object 508, etc. shown in the real frame N.

[0006] Retrieve one or more second objects in the first image frame based on a second type of drawing instruction in the drawing instruction stream of the first image frame. The second type of drawing instruction is a control drawing instruction. A control drawing instruction is also called a UI drawing instruction. The second object can be a control, a health bar, etc., and the position of the second object in the image frame is generally fixed.

[0007] Acquire one or more third objects in the second image frame; illustratively, the second image frame may be attached Figure 2 and the real frame N-2 shown in 5 , the one or more third objects may be the object 502 , the object 504 , the object 506 , and the object 508 in the real frame N-2;

[0008] Calculating a first motion vector between one or more first objects and one or more third objects, where the one or more third objects match the one or more first objects, and the second image frame is an image frame preceding the first image frame; illustratively, object 502 in real frame N-2 matches object 502 shown in real frame N, and object 504 in real frame N-2 matches object 504 shown in real frame N, and the first motion vector may include a motion vector between object 502 in real frame N-2 and object 502 in real frame N, and a motion vector between object 504 in real frame N-2 and object 504 in real frame N;

[0009] Obtain a second motion vector, the second motion vector being a motion vector between one or more third objects and one or more fourth objects in a third image frame, the one or more third objects being matched with the one or more fourth objects, and the third image frame being an image frame before the second image frame; exemplarily, the third image frame may be an image frame attached to Figure 2 5 , the one or more fourth objects may be object 502, object 504, object 506, and object 508 in the real frame N-4. The second motion vector may include a motion vector between object 502 in the real frame N-4 and object 502 in the real frame N-2, and a motion vector between object 504 in the real frame N-4 and object 504 in the real frame N-2.

[0010] Based on the first motion vector and the second motion vector, a third motion vector is calculated. Exemplarily, the third motion vector may be half of the difference between the first motion vector and the second motion vector.

[0011] Based on the first motion vector, the third motion vector and one or more first objects, a first predicted image frame is obtained, wherein the first predicted image frame is attached Figure 2 The predicted scene image corresponding to the real frame N+1 shown;

[0012] The first predicted image frame is merged with one or more second objects to obtain a second predicted image frame, which may indicate predicted frame N+1. Specifically, the position of the second object is generally fixed in the image frame, so the second object can be directly added to the fixed position of the first predicted image frame to obtain predicted frame N+1.

[0013] In one embodiment, before obtaining the drawing instruction stream of the first image frame, the method includes replacing a first pointer in a pointer list with a second pointer, wherein the first pointer points to a first function and the second pointer points to a second function, wherein the first function is used to draw the first image frame and the second function is used to identify the drawing instructions in the drawing instruction stream of the first image frame. For example, the first pointer may be an original function pointer P1 in a graphics library (where pointer P1 points to an implementation function corresponding to the original function in the graphics library), and the second pointer may be an intercepted function pointer P2 (where pointer P2 may point to an implementation function corresponding to the original function in the identification module).

[0014] In one embodiment of the present application, before obtaining one or more first objects in the first image frame based on the first type of drawing instructions in the drawing instruction stream, the method further includes: identifying the drawing instructions in the drawing instruction stream of the first image frame based on the second function to determine the first type of drawing instructions, the second type of drawing instructions, and the third type of drawing instructions in the drawing instruction stream of the first image frame.

[0015] In one embodiment of the present application, before obtaining one or more third objects in the second image frame, the method further includes: obtaining a drawing instruction stream of the second image frame; obtaining one or more third objects in the second image frame based on the first type of drawing instructions in the drawing instruction stream of the second image frame; and storing the one or more third objects.

[0016] In one embodiment of the present application, acquiring one or more third objects in the second image frame includes acquiring the one or more third objects based on a third type of drawing instruction in the drawing instruction stream of the first image frame, where the third type of drawing instruction is an image display instruction. For example, when the electronic device recognizes an image display instruction in the drawing instruction stream of real frame N, the one or more third objects in real frame N-2 may be acquired.

[0017] In one embodiment of the present application, one or more first objects include a fifth object, one or more third objects include a sixth object, and one or more fourth objects include a seventh object. The one or more first objects are matched with the one or more third objects so that the fifth and sixth objects have the same identifier, and the one or more third objects are matched with the one or more fourth objects so that the fifth and seventh objects have the same identifier. For example, the fifth object may be object 502 in real frame N, the sixth object may be object 502 in real frame N-2, and the seventh object may be object 502 in real frame N-4. The one or more first objects are matched with the one or more third objects so that the objects 502 in real frame N and 502 in real frame N-2 have the same identifier, and the one or more third objects are matched with the one or more fourth objects so that the objects 502 in real frame N-2 and 502 in real frame N-4 have the same identifier.

[0018] In one embodiment of the present application, there are P image frames between the first image frame and the second image frame, and there are Q image frames between the second image frame and the third image frame, where P and Q are positive integers, and P and Q can be the same or different.

[0019] In one embodiment of the present application, the one or more first objects include a fifth object and an eighth object, the one or more third objects include a sixth object and a ninth object, the fifth object and the sixth object have the same identifier, and the eighth object and the ninth object have the same identifier. For example, the eighth object may be object 504 in real frame N, and the ninth object may be object 504 in real frame N-2. After acquiring the one or more first objects in the first image frame, the method further includes:

[0020] Based on the identifier of the fifth object, vertex information of the sixth object is obtained. Exemplarily, vertex information of the object 502 in the real frame N is obtained by the identifier of the object 502 in the real frame N. Exemplarily, vertex information of the corresponding person in the real frame N-2 can be obtained by the identifier of the object 502 in the real frame N, that is, the identifier of the person;

[0021] Based on the difference between the vertex information of the fifth object and the vertex information of the sixth object, the fifth object is determined to be a dynamic object, by comparing the vertex information of the object 502 in the real frame N with the vertex information of the object 502 in the real frame N-2. If the vertex information is different, the object 502 in the real frame N is a dynamic object.

[0022] Based on the identifier of the eighth object, vertex information of the ninth object is obtained. As described above, the eighth object may be object 504 in real frame N, and the ninth object may be object 504 in real frame N-2. For example, object 504 may be a tree. Based on the identifier of object 504 (i.e., the tree) in real frame N, vertex information of object 504 in real frame N-2 may be obtained. For example, the vertex information of object 504 in real frame N-2 may be obtained from a cache.

[0023] Based on the fact that the vertex information of the eighth object is the same as the vertex information of the ninth object, the eighth object is determined to be a static object. For example, based on the fact that the vertex information of the object 504 in the real frame N is consistent with the vertex information of the object 504 in the real frame N-2, the object 504 in the real frame N is a static object.

[0024] The fifth object is recorded as a dynamic object; the eighth object is recorded as a static object.

[0025] In one embodiment of the present application, calculating a first motion vector between one or more first objects and one or more third objects includes:

[0026] Calculating a first motion component of the first motion vector based on the coordinates of the fifth object and the sixth object; illustratively, the first motion components of the objects 502 in the real frame N and the real frame N-2 may be calculated based on the coordinates of the object 502 in the real frame N and the coordinates of the object 502 in the real frame N-2;

[0027] Based on the coordinates of the eighth object and the ninth object, the second motion component of the first motion vector is calculated; for example, the second motion components of the object 504 in the real frame N and the object 504 in the real frame N-2 can be calculated based on the coordinates of the object 504 in the real frame N and the coordinates of the object 504 in the real frame N-2.

[0028] In one embodiment of the present application, before obtaining the second motion vector, the method includes:

[0029] Obtain a drawing instruction stream of the third image frame; based on the first type of drawing instructions in the drawing instruction stream of the third image frame, obtain one or more fourth objects in the third image frame; illustratively, based on the scene drawing instructions in the drawing instruction stream of the real frame N-4, objects 502, objects 504, 508, etc. in the real frame N-4 can be obtained.

[0030] storing one or more fourth objects;

[0031] After acquiring one or more third objects in the second image frame, the method further includes: calculating second motion vectors between the one or more third objects and the one or more fourth objects; exemplarily, it has been shown above that the second motion vectors may include a motion vector between the object 502 in the real frame N-4 and the object 502 in the real frame N-2, a motion vector between the object 508 in the real frame N-4 and the object 508 in the real frame N-2, and a motion vector between the object 504 in the real frame N-4 and the object 504 in the real frame N-2;

[0032] The second motion vector is stored.

[0033] In one embodiment of the present application, the one or more fourth objects include a seventh object and a tenth object, the sixth object and the seventh object have the same identifier, and the ninth object and the tenth object have the same identifier. For example, as shown above, the seventh object may be the object 502 in the real frame N-4, the sixth object may be the object 502 in the real frame N-2, the ninth object may be the object 504 in the real frame N-2, and the tenth object may be the object 504 in the real frame N-4.

[0034] Calculating a second motion vector between one or more third objects and one or more fourth objects includes:

[0035] Calculating a first motion component of the second motion vector based on the coordinates of the sixth object and the seventh object; illustratively, the first motion component between the object 502 in the real frame N-4 and the object 502 in the real frame N-2 may be calculated based on the coordinates of the object 502 in the real frame N-2 and the coordinates of the object 502 in the real frame N-4;

[0036] Based on the coordinates of the ninth object and the tenth object, the second motion component of the second motion vector is calculated; for example, based on the coordinates of the object 504 in the real frame N-2 and the coordinates of the object 504 in the real frame N-4, the second motion component between the object 504 in the real frame N-4 and the object 504 in the real frame N-2 can be calculated.

[0037] In one embodiment of the present application, calculating the third motion vector based on the first motion vector and the second motion vector includes:

[0038] Half of the difference between the second motion vector and the first motion vector is used as the third motion vector. For example, the third motion vector may include half of the difference between the first motion component between object 502 in real frame N-4 and object 502 in real frame N-2, and the first motion component between object 502 in real frame N-2 and object 502 in real frame N. The third motion vector may also include half of the difference between the second motion component between object 504 in real frame N-4 and object 504 in real frame N-2, and the second motion component between object 504 in real frame N-2 and object 504 in real frame N.

[0039] In one embodiment of the present application, obtaining a first predicted image frame based on the first motion vector, the third motion vector, and one or more first objects includes:

[0040] Obtain predicted coordinates of a fifth object based on a sum of half the first motion component of the first motion vector, the first motion component of the third motion vector, and the coordinates of the fifth object; Obtain predicted coordinates of an eighth object based on a sum of half the second motion component of the first motion vector, the second motion component of the third motion vector, and the coordinates of the eighth object;

[0041] A first predicted image frame is obtained based on the predicted coordinates of the fifth object and the predicted coordinates of the eighth object.

[0042] In one embodiment of the present application, it further includes:

[0043] discarding the drawing instruction stream of the image frames between the first image frame and the second image frame;

[0044] The drawing instruction streams of the image frames between the second image frame and the third image frame are discarded.

[0045] In one embodiment of the present application, it further includes:

[0046] The drawing instruction stream of the first image frame is drawn.

[0047] In one embodiment of the present application, it further includes:

[0048] After the first image frame is displayed, the second predicted image frame is displayed.

[0049] In a second aspect, the present application provides an image prediction method, which is applied to an electronic device. The electronic device includes a recognition module 316, a matching module 322, a calculation module 324, an estimation module 326, and a merging module 330. The method includes:

[0050] The identification module 316 obtains the drawing instruction stream of the first image frame;

[0051] The identification module 316 obtains a first type of drawing instruction in a drawing instruction stream based on the first image frame, and obtains one or more first objects of the first image frame, where the first type of drawing instruction is a scene drawing instruction;

[0052] The identification module 316 obtains a second type of drawing instruction based on the drawing instruction stream, and obtains one or more second objects in the first image frame, where the second type of drawing instruction is a control drawing instruction;

[0053] The matching module 322 obtains one or more third objects in the second image frame;

[0054] The calculation module 324 calculates a first motion vector between the one or more first objects and the one or more third objects, the one or more third objects are matched with the one or more first objects, and the second image frame is an image frame before the first image frame;

[0055] The calculation module 324 obtains a second motion vector, where the second motion vector is a motion vector between one or more third objects and one or more fourth objects in a third image frame, where the one or more third objects match the one or more fourth objects, and the third image frame is an image frame preceding the second image frame.

[0056] The calculation module 324 calculates a third motion vector based on the first motion vector and the second motion vector;

[0057] The estimation module 326 obtains a first predicted image frame based on the first motion vector, the third motion vector and the one or more first objects;

[0058] The merging module 330 merges the first predicted image frame with one or more second objects to obtain a second predicted image frame.

[0059] In one embodiment of the present application, the electronic device also includes an interception module 314. Before the identification module 316 obtains the drawing instruction stream of the first image frame, the interception module 314 replaces the first pointer in the pointer list with a second pointer. The first pointer points to a first function, and the second pointer points to a second function. The first function is used to draw the first image frame, and the second function is used by the identification module to identify the drawing instructions of the drawing instruction stream of the first image frame.

[0060] In one embodiment of the present application, before the identification module 316 obtains one or more first objects in the first image frame based on the first type of drawing instructions in the drawing instruction stream, the identification module identifies the drawing instructions in the drawing instruction stream of the first image frame based on the second function to determine the first type of drawing instructions, the second type of drawing instructions, and the third type of drawing instructions in the drawing instruction stream of the first image frame.

[0061] In one embodiment of the present application, before the matching module 322 obtains one or more third objects in the second image frame, the recognition module also obtains a drawing instruction stream of the second image frame, obtains one or more third objects in the second image frame based on the first type of drawing instructions in the drawing instruction stream of the second image frame, and the recognition module stores the one or more third objects in the cache of the electronic device.

[0062] In one embodiment of the present application, the electronic device also includes a separation module 318, one or more first objects include a fifth object and an eighth object, one or more third objects include a sixth object and a ninth object, the fifth object and the sixth object have the same identifier, and the eighth object and the ninth object have the same identifier. After the recognition module obtains one or more first objects in the first image frame, the separation module obtains the vertex information of the sixth object based on the identifier of the fifth object; the separation module determines that the fifth object is a dynamic object based on the difference between the vertex information of the fifth object and the vertex information of the sixth object; the separation module obtains the vertex information of the ninth object based on the identifier of the eighth object; the separation module determines that the eighth object is a static object based on the same vertex information of the eighth object and the vertex information of the ninth object; the separation module records the fifth object as a dynamic object; and records the eighth object as a static object.

[0063] In one embodiment of the present application, the calculation module 324 calculates a first motion vector between one or more first objects and one or more third objects, including: the calculation module 324 calculates a first motion component of the first motion vector based on the coordinates of the fifth object and the sixth object; the calculation module calculates a second motion component of the first motion vector based on the coordinates of the eighth object and the ninth object.

[0064] In one embodiment of the present application, before the calculation module 324 obtains the second motion vector, the calculation module 324 includes:

[0065] The calculation module obtains a drawing instruction stream of the third image frame;

[0066] The calculation module obtains one or more fourth objects in the third image frame based on the first type of drawing instructions in the drawing instruction stream of the third image frame;

[0067] The computing module stores the one or more fourth objects through a cache of the electronic device;

[0068] After the computing module acquires the one or more third objects in the second image frame, the method further includes:

[0069] The calculation module calculates a second motion vector between one or more third objects and one or more fourth objects;

[0070] The calculation module stores the second motion vector in a buffer of the electronic device.

[0071] In one embodiment of the present application, the one or more fourth objects include a seventh object and a tenth object, the sixth object and the seventh object have the same identifier, and the ninth object and the tenth object have the same identifier. The calculation module calculates the second motion vector between the one or more third objects and the one or more fourth objects, including:

[0072] The calculation module calculates a first motion component of the second motion vector based on the coordinates of the sixth object and the seventh object;

[0073] The calculation module calculates a second motion component of the second motion vector based on the coordinates of the ninth object and the tenth object.

[0074] In one embodiment of the present application, the calculation module calculates the third motion vector based on the first motion vector and the second motion vector, including:

[0075] The calculation module uses half of the difference between the second motion vector and the first motion vector as the third motion vector.

[0076] In one embodiment of the present application, the estimation module 326 obtains a first predicted image frame based on the first motion vector, the third motion vector, and one or more first objects, including:

[0077] The estimation module obtains predicted coordinates of the fifth object based on a sum of half the first motion component of the first motion vector, the first motion component of the third motion vector, and the coordinates of the fifth object;

[0078] The estimation module obtains predicted coordinates of the eighth object based on a sum of half the second motion component of the first motion vector, the second motion component of the third motion vector, and the coordinates of the eighth object;

[0079] The estimation module obtains a first predicted image frame based on the predicted coordinates of the fifth object and the predicted coordinates of the eighth object.

[0080] In one embodiment of the present application, the electronic device further includes a discarding module, which discards the drawing instruction stream of the image frames between the first image frame and the second image frame, and further discards the drawing instruction stream of the image frames between the second image frame and the third image frame.

[0081] In one embodiment of the present application, the electronic device further includes a display module 332 , which displays the second predicted image frame after the display of the first image frame is completed.

[0082] In a third aspect, the present application provides an electronic device, characterized in that it includes a processor and a storage device, wherein the storage device stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the prediction method described in the first aspect.

[0083] In a fourth aspect, the present application provides an electronic device, characterized in that it includes a processor and a storage device, wherein the storage device stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the prediction method described in the second aspect.

[0084] In a fifth aspect, the present application provides a computer-readable storage medium, characterized in that it includes computer instructions, which, when the computer instructions are executed on the electronic device, enable the electronic device to execute the prediction method as described in the first aspect.

[0085] In a sixth aspect, the present application provides a computer-readable storage medium, characterized in that it includes computer instructions, which, when the computer instructions are executed on the electronic device, enable the electronic device to execute the prediction method described in the second aspect.

[0086] The prediction method shown in this application can reduce the resource consumption of the GPU and the GPU when executing drawing instructions, thereby reducing the load on the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 A schematic structural diagram of an electronic device 100 provided in an embodiment of the present invention;

[0088] Figure 2 is a schematic diagram of an image frame prediction method according to an embodiment of the present invention;

[0089] Figure 3 The hardware layer and software layer for executing the image frame prediction method of the present application in the embodiment of the present invention;

[0090] Figure 4A schematic diagram of discarding some drawing instructions in a drawing instruction stream according to an embodiment of the present invention;

[0091] Figures 5A-5D A schematic diagram of obtaining a motion vector map between adjacent frames in an embodiment of the present invention;

[0092] Figure 6 Schematic diagram of storage changes of image frames and motion vector maps in a buffer in an embodiment of the present invention;

[0093] Figure 7 is a flowchart of a method for predicting an image frame according to an embodiment of the present invention;

[0094] Figure 8 A framework diagram of the software layer and the hardware layer in an embodiment of the present invention;

[0095] Figure 9 2 is an example diagram of object trajectory changes in an embodiment of the present invention. DETAILED DESCRIPTION

[0096] The following describes the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents "or." For example, A / B can represent A or B. "And / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, or B exists alone.

[0097] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0098] See also Figure 1 , which specifically shows a structural diagram of an electronic device 100, and the method provided in this application can be applied to the electronic device 100.

[0099] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0100] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0101] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0102] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0103] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0104] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0105] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0106] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0107] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0108] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0109] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0110] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0111] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0112] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0113] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0114] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0115] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0116] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0117] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0118] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0119] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0120] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0121] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0122] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0123] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0124] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization for image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0125] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0126] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0127] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0128] The NPU is a neural-network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU enables intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0130] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0131] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0132] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0133] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0134] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0135] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0136] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0137] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0138] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0139] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0140] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0141] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0142] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0143] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0144] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0145] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0146] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0147] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0148] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0149] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0150] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0151] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0152] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0153] The principles of the image frame prediction method shown in this application are explained and illustrated below with reference to the accompanying drawings.

[0154] For ease of understanding, some professional and technical terms in this application are explained and illustrated below:

[0155] About the Graphics Library

[0156] The graphics library is also called a drawing library. The graphics library is used to define an application programming interface (API) that is cross-programming language and cross-platform, which contains many functions for processing graphics. Taking OpenGL (Open Graphics Library) as an example, the API defined by OpenGL includes an interface for drawing two-dimensional images or three-dimensional images (the interface includes drawing functions, such as the drawing function glDrawElements()), and also includes an interface for presenting the image drawn by the drawing function to the display interface (the interface includes a rendering function, such as the function eglSwapBuffers()), etc. The embodiments of this application do not give examples one by one here. Among them, the functions in OpenGL can be called through instructions. For example, a drawing function can be called through a drawing instruction to draw a two-dimensional image or a three-dimensional image. The drawing instruction is a command written by the developer according to the function in the graphics library during the development of the game application, which is used to call the interface of the graphics library corresponding to the drawing instruction.

[0157] About game image frames:

[0158] As shown above, the two-dimensional image or three-dimensional image drawn by the drawing function called by the drawing instruction may include game image frames, as well as other types of image frames. Specifically, during the running process of the game application, it is displayed by continuously rendering and quickly playing frames of images. A frame of image is a frame of static image displayed by the game application. Each static image frame may be composed of scene images and UI images, etc. For example, the scene image may include in-game scenery, game characters, background objects, special effects, skills, etc., and the UI image may include images such as rendering control buttons, small maps, floating text, etc. In some games, the character health bar is also included in the UI image. It should be noted that whether it is the game character in the scene image or the rendering control button in the UI image, it can be considered as an object in the game image frame. It can be understood that each game image frame is composed of individual objects.

[0159] About drawing instructions:

[0160] Each object in the game image frame is obtained by executing drawing instructions through specific software or hardware of the electronic device. An object can be drawn by one or more drawing instructions. Generally, an object corresponds to a drawing instruction one by one. It should be noted that each drawing instruction also includes specific parameters it carries, such as vertex information, etc. When the electronic device executes the drawing instruction, the object corresponding to the drawing instruction is drawn based on its specific parameters. It should be noted that the drawing instructions corresponding to the same object in different frames are consistent. For example, a large tree displayed in multiple consecutive game image frames of the game screen is drawn by the electronic device hardware based on the same drawing instruction. However, this application does not limit this "same" to no difference. In some embodiments, the same can also be "approximate" or "similar".

[0161] About drawing instruction flow:

[0162] A drawing instruction stream is an instruction stream consisting of one or more drawing instructions. A drawing instruction stream is generally used to draw a frame of image. Specifically, the GPU can execute one or more drawing instructions in the drawing instruction stream and call one or more interfaces of the graphics library to draw one or more objects in the image frame. It should be noted that each object drawn by the drawing instruction can be represented by data stored in memory. For example, the collection of drawing objects generated according to the drawing instruction stream can constitute the display data of the corresponding game image frame.

[0163] See attached Figure 2 , specifically relating to a schematic diagram of a method for predicting image frames in this application.

[0164] like Figure 2As shown, real frame N-4, real frame N-3, real frame N-2, real frame N-1, real frame N, real frame N+1 and real frame N+2 are image frames displayed by the game application in chronological order. Real frames are relative to predicted frames, and predicted frames can be predicted based on real frames. Specifically, each real frame includes one or more objects, and the objects of each real frame can be obtained by the GPU executing the drawing instructions corresponding to the objects in the drawing instruction stream. For example, real frame N includes four objects, namely object 502, object 504, object 506 and object 508. Object 502, object 504, object 506 and object 508 can be drawn by drawing instruction 01, drawing instruction 11, drawing instruction 21 and drawing instruction 31 respectively. The real frame N-2 includes four objects, namely object 502, object 504, object 506 and object 508. Object 502, object 504, object 506 and object 508 are also drawn by drawing instruction 01, drawing instruction 11, drawing instruction 21 and drawing instruction 31 respectively.

[0165] Since real frames N-4, N-3, N-2, N-1, N, N+1, and N+2 are consecutive image frames in a gaming application, at a certain frame rate, such as 60 fps, 60 frames of the gaming application can be displayed per second. It is understood that adjacent frames generally contain the same objects. For example, real frames N-2 and N contain four identical objects (object 502, object 504, object 506, and object 508). Since identical objects are generally drawn using the same drawing instructions, the motion trajectories of objects between adjacent frames are traceable and predictable. Specifically, the present application identifies drawing instructions in the drawing instruction stream of each image frame and classifies and labels the objects within the image frame, thereby predicting the motion trajectories of one or more objects between adjacent frames to form predicted frames. Derived from predictions, predicted frames reduce resource consumption by the GPU and the GPU when executing drawing instructions, thereby reducing the load on the electronic device.

[0166] Continue to see Figure 2, when the electronic device 100 displays the image frames of the game application, the drawing instruction streams corresponding to the real frames such as real frame N-3, real frame N-1, and real frame N+1 can be discarded first. It should be noted that discarding the drawing instruction stream can be understood as not executing the drawing instruction stream. Since the drawing instruction streams corresponding to the real frames N-3, real frame N-1, and real frame N+1 have been discarded, they will not be processed, thereby reducing the resource consumption of the GPU when executing the drawing instructions, and also reducing the load on the electronic device. It should be noted that the embodiments of the present application are not limited to discarding one real frame every other frame, and the drawing instruction streams of the real frames can also be discarded in more ways, which is not limited in the present application.

[0167] Continue to see Figure 2 , the electronic device 100 can predict the position of the object in real frame N+1 based on the motion trajectory of the same object included in real frame N-4, real frame N-2, and real frame N, thereby forming a predicted frame N+1, which is displayed after real frame N. In one embodiment, the number of predicted frames can be consistent with the number of discarded frames. In this way, the display frame rate of the game image frame can be guaranteed while reducing the load on the electronic device. Figure 2 After the real frame N-3, the real frame N-1, and the real frame N+1 are discarded, the electronic device 100 generates corresponding prediction frames at the positions of the discarded real frames. It can be understood that Figure 2 In the illustrated embodiment, the frame stream display order of the electronic device is real frame N-4, predicted frame N-3, real frame N-2, predicted frame N-1, real frame N, predicted frame N+1 and real frame N+2.

[0168] It should be noted that the image frame prediction method shown in this application is not equivalent to the traditional video frame prediction method. This application is mainly used for the display of game image frames, which are different from traditional video frames. Specifically, game image frames have high real-time requirements, and can only use the previous frame to predict the next frame, which means that frames cannot be inserted before and after. However, the real-time requirements for video images are not high, and the previous and next frames can be used to predict the intermediate frames, and the intermediate frames can be inserted between the previous and next frames. In addition, video frames are obtained by calculation based on the pixel data in the video frames, while game image frames can be obtained by calculation by the terminal device based on the game's rendering instructions and the parameters corresponding to the rendering instructions. These parameters corresponding to the rendering instructions may include vertex information, etc., while the video frames do not contain vertex information.

[0169] See also Figure 3 , specifically a framework diagram of the software layer and hardware layer of the electronic device 100 performing the image frame prediction method provided in the embodiment of the present application. Figure 2 and Figure 3 and Figures 5A-5D, further explain and illustrate the method shown in this application. Figure 5A This is an example diagram of a scene image of a real frame N-4 provided in one embodiment of the present application. Figure 5B This is an example diagram of a scene image of a real frame N-2 provided in one embodiment of the present application. Figure 5C This is an example diagram of a scene image of a real frame N provided in one embodiment of the present application. Figure 5D This is a motion vector diagram between objects among real frame N-4, real frame N-2, and real frame N provided in one embodiment of the present application.

[0170] See also Figure 3 The software layer of the electronic device 100 includes an application layer 302 and a system framework layer 304 , and the hardware layer 306 includes a GPU, a CPU, a cache 320 , and the like.

[0171] The application layer 302 includes one or more applications that can be run on the electronic device 100, such as a game application 308. For ease of understanding, the method shown in this application will be explained and illustrated below using the game application 308 as an example.

[0172] The game application 308 includes a game engine 310 . The game engine 310 can call a drawing function in a graphics library 312 through a graphics library interface to draw images of the game application.

[0173] The system framework layer 304 may include various graphics libraries 312, such as Open Graphics Library for Embedded Systems (OpenGL ES), EGL, etc.

[0174] In related art, when a user opens a game application 308, the electronic device 100 launches the game application 308 in response to the user's game launch. Based on the drawing instruction stream for the image frame issued by the game application, the game engine 310 calls the drawing functions within the graphics library through the graphics library interface to draw the image frame. After the graphics library generates the image data for the graphics frame, it calls the display interface (such as eglSwapBuffers()) to send the image data to the surfacefinger buffer queue. Based on the periodic display signal, the graphics library sends the image data in the buffer queue to the hardware (such as the CPU) for synthesis. Finally, the synthesized image data is sent to the display screen of the electronic device 100 for display.

[0175] In one embodiment of the present application, graphics library 312 includes an interception module 314. Graphics library 312 allows modification of the library's function pointer list. Interception module 314 replaces the pointers in the library's function pointer list, causing the replaced function pointers to point to functions in identification module 316 outside the library. Thus, when a game engine issues a stream of drawing instructions for an image frame through a game application, when a function in the graphics library is called, the instructions are first sent to identification module 316 for instruction recognition. In other words, interception module 314 can intercept the image frame's drawing instruction stream by replacing the pointers in the library's function pointer list.

[0176] In one embodiment of the present application, the drawing instruction stream currently intercepted by the interception module 314 is the drawing instruction stream of the real frame N. The identification module 316 may perform instruction identification on each drawing instruction in the drawing instruction stream of the real frame N issued by the game application.

[0177] The drawing instruction stream generally includes three types of drawing instructions: scene drawing instructions, UI drawing instructions, and image display instructions. Scene drawing instructions are used to draw in-game scenery, characters, special effects, skills, and other images. UI drawing instructions are used to draw images such as control buttons, mini-maps, and floating text. In some games, character health bars are also drawn using UI drawing instructions. It should be noted that UI drawing instructions can also be called control drawing instructions. Image display instructions are used to place the completed image data into a system-specified location (such as framebuffer0 in the Android system) for actual display.

[0178] The image display instruction in the drawing instruction stream is generally the last instruction in the drawing instruction stream of an image frame. The identification module 316 can determine whether the drawing instruction is an image display instruction by determining whether the drawing instruction calls the display interface. For example, if the identification module 316 determines that the drawing instruction is used to call the swapbuffer interface, it determines that the drawing instruction is an image display instruction.

[0179] UI images are typically placed at the top of an image frame. For example, the control wheel, buttons, and minimap border within a UI image are semi-transparent. When drawing UI images, the graphics library must ensure that the UI images are placed at the top of the entire image. Specifically, gaming applications can achieve this by disabling depth testing and enabling blend testing, placing the UI images at the top of the image frame. Therefore, identification module 316 can determine whether an identified drawing instruction is a UI drawing instruction by detecting whether depth testing is disabled and blend testing is enabled. Specifically, when identification module 316 identifies a drawing instruction stream containing both a blending enable command and a depth testing disable command, it can determine that all subsequent instructions intercepted by interception module 314 are UI drawing instructions until an image display instruction is received. For example, the blending enable command can be glEnable(GL_BLEND), and the depth testing disable command can be glDisable(GL_DEPTH_TEST). In some games, in addition to turning off depth detection and turning on the hybrid detection function, other methods can be used to ensure that the UI image is located at the top layer of the image frame. For example, the game application can assign the depth value of the object in the UI image to the maximum value, so that the UI image is located at the top layer of the image frame. Therefore, after the recognition module 316 recognizes the instruction for setting the depth value of the object to the maximum value and the hybrid enable command, it can be determined that the instructions intercepted by the interception module 314 thereafter are all UI drawing instructions until the image display instruction is received. Specifically, the order of the instruction for setting the depth value of the object to the maximum value and the hybrid enable command is not limited.

[0180] When the identification module 316 identifies that the drawing instruction is a UI drawing instruction, it stores the UI image data drawn by the UI drawing instruction into the cache 320 so as to be merged with the predicted scene image. For details, please refer to the following.

[0181] In one embodiment, drawing instructions in the drawing instruction stream, excluding UI drawing instructions and image display instructions, can be considered scene drawing instructions. Specifically, in general, the drawing instruction stream for a single image frame is issued in the order of scene drawing instructions, UI drawing instructions, and image display instructions. The recognition module can consider all drawing instructions preceding the aforementioned blending enable command and depth detection off command to be scene drawing instructions. For example, the blending enable command and depth detection off command are sequential, for example, the blending enable command precedes the depth detection off command. When the recognition module 316 recognizes the blending enable command, it records a flag 1 indicating that the blending enable command has been received. Subsequently, when the recognition module recognizes the depth detection off command, it records a flag 2 indicating that the depth detection off command has been received. At this point, the recognition module determines that both flags 1 and 2 have been recorded, indicating that the recognition module has received both the blending enable command and the depth detection off command. The recognition module 316 can then determine that the instructions following the depth detection off command are UI drawing instructions, and the instructions preceding the blending enable command are scene drawing instructions. Furthermore, a global variable can be set in the recognition module 316, and the initial value can be set to 0. If the recognition module determines that the value of the global variable is 0, the recognition module 316 confirms that the recognized drawing instruction is a scene drawing instruction. When the preset condition is met, the recognition module sets the value of the global variable to 1. When the subsequent recognition module determines that the value of the global variable is 1, the recognition module 316 confirms that the recognized drawing instruction is a UI drawing instruction. Specifically, the recognition module 316 can determine whether the above-mentioned preset conditions are met based on the above-mentioned mark 1 and mark 2. For example, the recognition module 316 can assign the global variable a value of 1 when it determines that mark 1 and mark 2 have been recorded. In this way, when the recognition module 316 sends the drawing instruction to the recognition module in the order of scene drawing instruction, UI drawing instruction and image display instruction, the recognition module 316 can identify the scene drawing instruction and UI drawing instruction based on the value of the global variable. It should be noted that, in the above example, the identification module 316 is not limited to assigning values ​​to global variables by detecting the flags indicating that the hybrid enable command and the depth detection off command have been received, but can also assign values ​​to global variables by detecting the flags indicating that the hybrid enable command, the depth detection off command and the depth buffer clear command have been received.

[0182] The system framework layer 304 also includes a separation module 318, which is used to separate objects in the scene image corresponding to the scene drawing instruction. When the identification module 316 identifies the drawing instruction as a scene drawing instruction, the identification module 316 can call the separation module 318 to separate dynamic objects from static objects in the scene image corresponding to the scene drawing instruction. Dynamic objects are objects whose form, position, or size changes between adjacent frames, while static objects are objects whose form, position, or size does not change between adjacent frames.

[0183] The separation module 318 may separate dynamic objects from static objects based on the parameters carried by the scene drawing instruction and the identifier of each object, and store the dynamic objects and the data of the static objects in the separated scene image into the cache 320 .

[0184] Each object drawn by a drawing instruction includes a unique identifier for representing the object. The parameters carried by the drawing instruction may include vertex information of the object. The format of the vertex information can be as follows:

[0185] float vertices[]={0.5f,0.5f, 0.0f, / / upper right corner 0.5f,-0.5f,0.0f, / / lower right corner -0.5f,-0.5f,0.0f, / / lower left corner -0.5f,0.5f,0.0f / / upper left corner}; vertex coordinates;

[0186] unsigned int indices[] = { / / Note that the index starts from 0! 0,1,3, / / the first triangle 1,2,3 / / the second triangle}; The connection line of the vertices;

[0187] In one example, when interception module 314 intercepts a scene drawing instruction for real frame N, separation module 318 may retrieve object data for real frame N-2 from cache 320 based on an object identifier in real frame N, and determine whether real frame N-2 also includes the object. Assuming that real frame N-2 does not include the object, separation module 318 may mark the object in real frame N as a dynamic object. Furthermore, if real frame N-2 does include the object, separation module 318 retrieves vertex information for the object in real frame N-2 and checks whether the vertex information for the object in real frame N-2 and real frame N is consistent. If not, the vertex information for the object in real frame N is added to the record and the object is marked as a dynamic object. If they are consistent, the object is marked as a static object. In one embodiment, a flag bit for marking dynamic and static objects may be set in the object's stencil area (a part of the object data structure), e.g., a static object is marked as 0 and a dynamic object is marked as 1. In this way, when separating the object of real frame N, the separation module 318 can read the object data of real frame N-2 from the cache 320. If real frame N-2 includes the object, the stencil area data of the object in real frame N-2 can be directly read to determine whether the object is a static object or a dynamic object.

[0188] In one example, when the image frame intercepted by the interception module 314 is the first frame of a game application, each object in the first frame may be marked as a dynamic object.

[0189] For example, see Figure 5B and Figure 5C When separating object 502 in real frame N, separation module 318 can directly obtain the stencil region data of object 502 in real frame N-2 from cache 320 based on the identifier of object 502. If the stencil region data is 1, object 502 in real frame N is determined to be a dynamic object. If the stencil region data is 0, object 502 in real frame N is determined to be a static object. Optionally, to ensure the accuracy of the separation result, after determining whether object 502 is a static object or a dynamic object based on the stencil region data of object 502 in real frame N-2, separation module 318 can further verify whether the vertex information of object 502 in real frame N is consistent with the vertex information of object 502 in real frame N-2. If they are consistent, even if the stencil region data of real frame N-2 is 1, the separation module determines that object 502 in real frame N is a static object.

[0190] In one example, there may be multiple instructions associated with an object, but generally there is only one instruction associated with the object's vertices. Since drawing instructions draw the corresponding object based on vertex information, the recognition module 316 can only send scene drawing instructions that include vertex information to the separation module 318, allowing the separation module 318 to separate dynamic objects from static objects based on the vertex information carried by the scene drawing instructions. In another example, the recognition module 316 can call back the drawing instructions that do not include vertex information to the graphics library, allowing the graphics library to continue calling the relevant drawing functions based on the called-back drawing instructions. Alternatively, the recognition module 316 can perform scene recognition based on the drawing instructions that do not carry vertex information.

[0191] In one example, when the interception module 314 intercepts the drawing instruction stream of real frame N, the cache 320 may store data of dynamic objects and static objects in the scene image of real frame N-2. It is understood that the data of dynamic objects and static objects in the scene image of real frame N-2 is separated by the separation module 318 from real frame N-2 when the electronic device 100 displays real frame N-2 and stored in the cache 320.

[0192] The system framework layer 304 also includes a matching module 322. When the drawing instruction identified by the recognition module 316 is an image display instruction of the real frame N, the recognition module 316 calls the matching module 322. The matching module 322 obtains the object data in the scene image of the real frame N and the real frame N-2 from the cache 320, and matches the objects between the real frame N and the real frame N-2. For example, the matching module 322 can perform matching based on object identification and / or object vertex information. Matching objects between the real frame N and the real frame N-2 refers to matching dynamic objects in the real frame N with dynamic objects in the real frame N-2, and matching static objects in the real frame N with static objects in the real frame N-2. For example, see Figure 5B and Figure 5C , object 502 in real frame N is a dynamic object, object 506 is a static object, and object 502 in real frame N-2 is a dynamic object, while object 506 is a static object. The matching module 322 may match the dynamic object 502 between real frame N and real frame N-2 based on the identifier of object 502, and may match the static object 506 between real frame N and real frame N-2 based on the identifier of object 506. In one example, the matching module may send the matched objects between real frame N and real frame N-2 to the calculation module 324. It will be appreciated that the matched objects between real frame N and real frame N-2 include matched dynamic objects and static objects. For example, object 502 may be the matched dynamic object between real frame N and real frame N-2, and object 506 may be the matched static object between real frame N and real frame N-2.

[0193] The calculation module 324 is configured to calculate the motion vectors between the objects matched in the real frame N and the real frame N-2 to obtain a motion vector map Y2 between the real frame N and the real frame N-2. The motion vector map Y2 between the real frame N and the real frame N-2 can be considered as a set of motion vectors between the matching objects in the real frame N and the real frame N-2. For example, the motion vector map Y2 can also be referred to as a first motion vector. The first motion vector includes one or more motion components. The first motion component of the first motion vector can be a motion vector between the object 502 in the real frame N and the real frame N-2. The second motion component of the first motion vector can be a motion vector between the object 506 in the real frame N and the real frame N-2.

[0194] The calculation module 324 is further configured to calculate motion vectors between matching objects in real frame N-2 and real frame N-4 to form a motion vector map Y1 between real frame N-2 and real frame N-4. Specifically, the motion vectors between matching objects in real frame N-2 and real frame N-4, as well as the motion vector map Y1, are calculated and obtained by the calculation module 324 when the electronic device 100 displays real frame N-2. Exemplarily, the motion vector map Y1 may also be referred to as a second motion vector, which includes one or more motion components, such as a first motion component and a second motion component. The first motion component of the second motion vector may be a motion vector between objects 502 in real frame N-2 and real frame N-4, and the second motion component of the second motion vector may be a motion vector between objects 506 in real frame N-2 and real frame N-4.

[0195] Exemplarily, half the difference between the first motion vector and the second motion vector is defined as the third motion vector. It can be understood that the third motion vector also includes a first motion component and a second motion component. Exemplarily, the first motion component of the third motion vector is half the difference between the first motion component of the first motion vector and the first motion component of the second motion vector, and the second motion component of the third motion vector is half the difference between the second motion component of the first motion vector and the second motion component of the second motion vector.

[0196] The calculation module 324 may store the motion vector map Y1 between the real frame N and the real frame N-2 in the buffer 320. The calculation module 324 may also calculate an estimated motion vector map Y3 between the real frame N and the real frame N+1 based on the motion vector map Y2 between the real frame N and the real frame N-2 and the motion vector map Y1 between the real frame N-2 and the real frame N-4. The calculation module 324 may store the estimated motion vector map Y3 in the buffer 320. In one example, after calculating the estimated motion vector map Y3, the calculation module 324 may send the estimated motion vector map Y3 to the estimation module 326 to perform motion estimation on the object in the scene image in the real frame N.

[0197] See below Figures 5A-5D , the calculation module 324 calculates the estimated motion vector map to further illustrate.

[0198] Figures 5A-5C The objects 502, 504, 506 and 508 are included. Figure 5A 、 Figure 5B and Figure 5C These correspond to real frame N-4, real frame N-2, and real frame N, respectively. When the electronic device displays real frame N-2, the separation module can obtain the object data of real frame N-4 from cache 320 and read the values ​​of specific regions of the object data structures of objects 502, 508, 504, and 506, such as the stencil region. This can determine that objects 502 and 508 are dynamic objects, and objects 504 and 506 are static images. When the electronic device displays real frame N-2, it can determine that from real frame N-4 to real frame N-2, object 502 moves in direction 510, object 508 moves in direction 512, and other static images in the scene image as a whole move in direction 514. When the electronic device displays real frame N, it can determine that from real frame N-2 to real frame N, object 502 moves in direction 510, object 508 moves in direction 512, and other static images in the scene image as a whole move in direction 514.

[0199] See also Figure 5DAfter the matching module 322 matches the objects 502 and 508 in the real frame N-4 and the real frame N-2, the calculation module 324 can calculate the motion vector y1(N-4, N-2) of the object 502 between the real frame N-4 and the real frame N-2, and the motion vector y2(N-4, N-2) of the object 508 between the real frame N-4 and the real frame N-2. Because the motion vectors between the static objects matched between the real frame N-4 and the real frame N-2 are equal to the motion vector of the entire scene, the calculation module 324 can calculate the motion vector y3(N-4, N-2) of any static object (such as the object 504) matched between the real frame N-4 and the real frame N-2, thereby determining the motion vectors y3(N-4, N-2) of all static objects matched between the real frame N-4 and the real frame N-2. The set of motion vectors Y1 {y1(N-4, N-2), y2(N-4, N-2), y3(N-4, N-2)} between all matching objects in the real frame N-4 and the real frame N-2 calculated by the calculation module 324 is a motion vector map between the real frame N-4 and the real frame N-2. Figure 5D In FIG, the motion vector map Y1 is the motion vector map between the real frame N-4 and the real frame N-2.

[0200] Based on the same principle, after the matching module matches objects 502 and 508 in real frame N-2 and real frame N, the calculation module 324 can calculate the motion vector y1(N-2, N) of object 502 between real frame N-2 and real frame N, as well as the motion vector y2(N-2, N) between real frame N-2 and object 508 in real frame N. The calculation module 324 can also calculate the motion vector y3(N-2, N) of any matching static object (such as object 504) between real frame N-2 and real frame N to determine the motion vector y3(N-2, N) of all static objects between real frame N-2 and real frame N. The set Y2{y1(N-2, N), y2(N-2, N), y3(N-2, N)} of motion vectors between all matching objects in real frame N-2 and real frame N calculated by the calculation module 324 is the motion vector map between real frame N-2 and real frame N. Figure 5D In the figure, the motion vector map Y2 is the motion vector map between the real frame N-2 and the real frame N.

[0201] The calculation module 324 can also calculate an estimated motion vector map Y3 between real frame N and real frame N+1 based on the motion vector map Y1 and the motion vector map Y2. For example, assuming that each object is uniformly accelerated, the calculation module 324 can calculate the motion acceleration of the object between real frame N and real frame N+1 using the motion vector map Y1 and the motion vector map Y2. For example, object 502 included in real frame N-4, real frame N-2, and real frame N indicates that object 502 is a matching object between real frame N-4 and real frame N-2, and also a matching object between real frame N-2 and real frame N. The calculation module 324 can calculate the motion vector y1(N,N+1) of the object 502 between the real frame N and the real frame N+1 using the motion vector y1(N-4,N-2) of the object 502 between the real frame N-4 and the real frame N-2, and the motion vector y1(N-2,N) of the object 502 between the real frame N-2 and the real frame N. For example, y1(N,N+1) = y1(N-2,N) / 2 + (y1(N-2,N) - y1(N-4,N-2)) / 2. Based on the same principle, the motion vector y2(N,N+1) of the object 508 between the real frame N and the real frame N+1 can be calculated. For example, y2(N,N+1) = y2(N-2,N) / 2 + (y2(N-2,N) - y2(N-4,N-2)) / 2. Based on the same principle, the motion vector y3(N,N+1) of all static objects between the real frame N and the real frame N+1 can be calculated, thereby forming an estimated motion vector map Y3{y1(N,N+1), y2(N,N+1), y3(N,N+1)} between the real frame N and the real frame N+1.

[0202] The system framework layer also includes an estimation module 326, which can perform motion estimation on the object of the real frame N through the estimated motion vector map Y3 calculated by the calculation module 324 and the scene image data of the real frame N and the motion vector map Y2 between the real frame N and the real frame-2.

[0203] For example, the estimation module 326 performs motion estimation on the motion trajectory of object 502 in real frame N based on the estimated motion vector y1(N, N+1) of object 502 in Y3. Specifically, the coordinates of object 502 in predicted frame N+1 = the coordinates of object 502 in real frame N + y1(N, N+1). Based on the same principle, the estimation module 326 performs motion estimation on the motion trajectory of object 508 based on the estimated motion vector y2(N, N+1) of object 508 in Y3 and the motion vector y2(N-2, N) of object 508 in Y2. Specifically, the coordinates of object 508 in predicted frame N+1 = the coordinates of object 508 in real frame N + y2(N, N+1). Based on the same principle, the positions of other objects in the real frame can also be estimated.

[0204] That is, the estimation module 326 displaces each dynamic and static object using the coordinates of the object in the real frame N and the vector of the object in the estimated motion vector map Y3 to obtain a predicted scene image for the predicted frame N+1. In one example, the estimation module 326 may store the data of the predicted scene image in the buffer 320.

[0205] After obtaining the predicted scene image of the prediction frame N+1, the estimation module 326 calls the verification module 328 in the system framework layer to verify the predicted scene graph.

[0206] The predicted scene image estimated by the estimation module may contain certain pixels or pixel blocks that do not appear in the previous real frame N, real frame N-2, and real frame N-4. This is due to reasons such as mutual occlusion between objects in the scene image and the display of a new image when the frame moves. Therefore, it is necessary to verify the predicted scene image. Specifically, the scene image can be verified using a variety of methods. For example, the size of the newly appearing pixels in the entire scene image can be calculated. The newly appearing pixels are then compared with a preset verification threshold. If the threshold is exceeded, the predicted frame is discarded.

[0207] In one embodiment, for the predicted frame N+1 that passes the verification, blur processing may be performed on the predicted scene image.

[0208] After verifying the predicted scene image, the verification module 328 calls the merging module 330 , which is used to merge the predicted scene image with the UI image.

[0209] After verifying the predicted scene image, the verification module 328 calls the merging module 330 to merge the UI image with the predicted scene image to form the final predicted image frame N+1. Specifically, the merging module 330 can retrieve the predicted scene image data and the UI image data from the cache 320. The UI image is generally fixed in position within the entire image frame. Therefore, the UI image can be merged into the predicted scene image to form the final predicted image frame N+1. After the merging module obtains the final predicted image frame, the cache can be cleared of unnecessary data, such as invalid motion vector maps and object data from the real frame. For example, in the above embodiment, after predicting predicted frame N+1, the motion vector map between real frame N-4 and real frame N-2 is no longer used to predict real frame N+3. The motion vector map between real frame N-4 and real frame N-2 is therefore an invalid motion vector map. Similarly, the object data in the cache for real frame N-4 is also invalid. To conserve cache 320, invalid motion vector maps and invalid scene image object data can be cleared.

[0210] The following combination Figure 2-Figure 6 The changes of data stored in the cache 320 are described. Figure 6 This is a schematic diagram of changes in the motion vector map stored in the cache 320 and the object data in the real frame scene image provided in one embodiment of the present application.

[0211] When the electronic device displays real frame N, cache 320 stores object data from the scene image of real frame N, as well as motion vector map Y1 between real frame N-4 and real frame N-2, and motion vector map Y2 between real frame N-2 and real frame N. When the electronic device displays predicted frame N+1, the cache clears the motion vector map Y1 between real frame N-4 and real frame N-2. When the electronic device displays real frame N+2, after calculating the motion vector map between real frame N and real frame N+2, the scene image data of real frame N is cleared. At this point, cache 320 stores object data from the scene image of real frame N+2, motion vector map Y2 between real frame N-2 and real frame N, and motion vector map Y4 between real frame N and real frame N+2. When the electronic device displays predicted frame N+3, the cache clears the motion vector map Y2 between real frame N-2 and real frame N.

[0212] That is, when the electronic device 100 displays the real frame N, the cache stores the scene image data of the real frame N and the motion vector maps between the real frames N-4 and N-2, and between the real frames N-2 and N. When the predicted frame N+1 is displayed, the cache stores the scene image data of the real frame N and the motion vector maps between the real frames N-2 and N. This allows for greater savings in memory storage area.

[0213] The system framework layer further includes a display sending module 332 , which is used to send the predicted image frame N+1 for display after the real frame N is displayed.

[0214] In one embodiment of the present application, the system framework layer also includes a callback module 334. The callback module 334 is used to call back the scene drawing instruction after the separation module 318 stores the data of the dynamic objects and static objects of the scene image of the real frame N into the cache 320, so that the graphics library draws the scene image of the real frame N. The callback module 334 is also used to call back the UI drawing instruction after the recognition module 316 stores the UI image data drawn by the UI drawing instruction into the cache, so that the graphics library draws the UI image of the real frame N. The callback module 334 is also used to call back the image display instruction when the recognition module 316 recognizes the image display instruction, so that the graphics library displays the image of the real frame N. After the graphics library completes drawing the real frame N and displays it, the display module 332 displays the predicted image frame N+1.

[0215] Specifically, when the interception module 314 replaces the graphics library function list pointer, the callback module 334 can back up the original function pointer P1 in the graphics library (where pointer P1 points to the implementation function corresponding to the original function in the image library), the interception function pointer P2 (where pointer P2 can point to the implementation function corresponding to the original function in the recognition module), and the callback function pointer P3 (where P3 = P1, i.e., P3 also points to the implementation function corresponding to the original function in the image library). Exemplarily, the interception module can replace pointer P1 with interception function pointer P2 to intercept the drawing instruction stream of the real frame. The callback module 334 can call back the drawing instruction stream of the image frame to the graphics library via callback function pointer P3, so that the graphics library can draw the image frame. Exemplarily, when the recognition module 316 identifies a scene image instruction, the callback module 334 calls back the scene drawing instruction via pointer P3. Alternatively, after the recognition module 316 stores the object data in the cache, the callback module 334 calls back the scene image instruction via pointer P3. Alternatively, when the recognition module 316 recognizes a UI drawing instruction, the callback module 334 calls back the UI drawing instruction through the pointer P3. Alternatively, after the recognition module 316 stores the UI drawing image in the cache, the callback module 334 calls back the scene image instruction through the pointer P3. Alternatively, when the recognition module 316 recognizes an image display instruction, the callback module 334 calls back the image display instruction through the pointer P3.

[0216] In one example, the callback module 334 may also call back the image display instruction after the merging module forms the predicted image frame.

[0217] The following combination Figure 2 、 Figure 3 and Figure 4 , further explain and illustrate the prediction method shown in this application, wherein Figure 4 for Figure 3 Schematic diagram of the interaction between modules in the system framework layer.

[0218] The following specifically takes the drawing instruction stream of the image frame intercepted by the interception module 314 in the graphics library as the drawing instruction stream of the real frame N as an example to illustrate the prediction method shown in this application.

[0219] In step A1 , the interception module 314 in the graphics library 312 calls the identification module 316 , so that the identification module 316 identifies the drawing instructions in the drawing instruction stream of the real frame N intercepted by the interception module 314 .

[0220] As shown above, the drawing instruction stream generally includes three types of drawing instructions, such as scene drawing instructions, UI drawing instructions, and image display instructions.

[0221] In step A2 , when the drawing instruction identified by the recognition module 316 is a scene drawing instruction, the recognition module 316 may call the separation module 318 (step A3 ), so that the separation module 318 separates the dynamic objects and the static objects in the scene image.

[0222] In step A4 , the separation module 318 separates the dynamic objects and static objects in the scene image corresponding to the scene rendering instruction, and then stores the separated dynamic objects and static objects in the cache 320 (step A5 ).

[0223] In step A6, after the separation module 318 stores the separated dynamic objects and static objects in the cache 320, the callback module 334 may be called. The callback module 334 calls back the scene drawing instruction (step A7), so that the graphics library draws the scene image of the real frame N.

[0224] In step A8 , when the drawing instruction identified by the recognition module 316 is a UI drawing instruction, the recognition module 316 caches the UI image corresponding to the UI drawing instruction (step A9 ).

[0225] In step A10 , the recognition module 316 caches the UI image corresponding to the UI drawing instruction, and then calls the callback module 334 . The callback module 334 calls back the UI drawing instruction (step A11 ), so that the graphics library draws the UI image of the real frame N.

[0226] In step A12, when the drawing instruction recognized by the recognition module 316 is an image display instruction, the recognition module 316 calls the callback module 334 (step A13), and the callback module 334 calls back the image display instruction (step A14), so that the graphics library displays the image of the drawn real frame N.

[0227] In step B1, when the drawing instruction recognized by the recognition module 316 is an image display instruction, the matching module 322 may be called to match the object in the real frame N with the object in the real frame N-2 (step B2).

[0228] In step B2, after matching the object in the real frame N with the object in the real frame N-2, the matching module 322 may cache the data of the matched object.

[0229] In step B3, after matching the object in the real frame N with the object in the real frame N-2, the matching module 322 calls the calculation module 324, and the calculation module 324 calculates the motion vector of the matched object between the real frame N and the real frame N-2 to form a motion vector map between the real frame N and the real frame N-2, and calculates the estimated motion vector map between the real frame N and the real frame N+1 based on the motion vector map between the real frame N and the real frame N-2 and the motion vector map between the real frame N-2 and the real frame N-4 (step B5).

[0230] In step B6 , the calculation module 324 stores the estimated motion vector map in the buffer 320 .

[0231] In step B7, after the calculation module 324 estimates the motion vector map and stores it in the cache 320, the estimation module 326 is called. The estimation module 326 performs motion estimation on the object of the real frame N based on the estimated motion vector map and the scene image data of the real frame N to obtain the predicted scene image of the predicted frame N+1 (step B8).

[0232] In step B9, after the estimation module 326 obtains the predicted scene image of the prediction frame N+1, it calls the verification module 328, and the verification module 328 verifies the predicted scene image (step B10).

[0233] In step B11, after the verification module 328 verifies the predicted scene image, it calls the merging module 330. The merging module 330 obtains the predicted scene image from the cache (step B12). The merging module 330 merges the predicted scene image with the UI image to obtain the predicted image frame N+1 (step B13).

[0234] In step B14, the merging module 330 sends the predicted image frame N+1 to the display module 332. After the real frame N is displayed, the display module 332 displays the predicted image frame.

[0235] By using the prediction method shown in the above embodiment and displaying the predicted frame, the resource consumption of the GPU when executing the drawing instructions can be reduced, thereby reducing the load on the electronic device.

[0236] See also Figure 7 , specifically relates to an image frame prediction method provided by an embodiment of the present application, which can be applied to the above-mentioned electronic device 100, and specifically includes:

[0237] Step 702: Obtain a drawing instruction stream of the first image frame;

[0238] Step 704: acquiring one or more first objects of the first image frame based on a first type of drawing instructions in the drawing instruction stream of the first image frame, where the first type of drawing instructions are scene drawing instructions;

[0239] Step 706: Acquire one or more second objects in the first image frame based on a second type of drawing instructions in the drawing instruction stream of the first image frame, where the second type of drawing instructions are control drawing instructions.

[0240] Step 708, acquiring one or more third objects in the second image frame;

[0241] Step 710: Calculate a first motion vector between the one or more first objects and the one or more third objects, wherein the one or more third objects match the one or more first objects, and the second image frame is an image frame preceding the first image frame.

[0242] Step 712: Obtain a second motion vector, where the second motion vector is a motion vector between the one or more third objects and one or more fourth objects in a third image frame, where the one or more third objects match the one or more fourth objects, and the third image frame is an image frame preceding the second image frame.

[0243] Step 714: Calculate a third motion vector based on the first motion vector and the second motion vector;

[0244] Step 716: Obtain a first predicted image frame based on the first motion vector, the third motion vector, and the one or more first objects;

[0245] Step 720: Merge the first predicted image frame with the one or more second objects to obtain a second predicted image frame.

[0246] See also Figure 8 , a framework diagram of the software layer and hardware layer of the electronic device 100 performing the image frame calibration method provided in an embodiment of the present application. It can be understood that, Figure 8 The framework diagram shown can also be used to implement the prediction method shown in the above embodiment.

[0247] In the prediction method shown in the above embodiment, real frames and predicted frames are displayed alternately, and the predicted frames are formed by predicting the real frames. Games have very high requirements for the real-time display of the screen, and display lag will have a great impact on the game display effect, so the predicted frame can only be obtained by predicting the real frame before the predicted frame, but this also makes the game application lack a post-frame feedback correction mechanism. For example, the predicted frame N+1 is predicted by the real frame N, the real frame N-2, and the real frame N-4. Because the generated predicted frame N+1 picture cannot be corrected according to the subsequent real frame N+2, the estimated motion vector used in the motion estimation process may have a large deviation from the actual motion vector. This may cause picture degradation during the display of the predicted frame, affecting the user's gaming experience on the terminal device.

[0248] Based on this, this application also proposes another image frame calibration method.

[0249] In the above embodiment, when the electronic device 100 forms a predicted frame N+1 based on the real frame N, the real frame N-2, and the real frame N-4, the calculation module 324 can obtain an estimated motion vector map Y3 {y1 (N, N+1), y2 (N, N+1), y3 (N, N+1)} by using the motion vector map Y1 {y1 (N-4, N-2), y2 (N-4, N-2), y3 (N-4, N-2)} of the object between the real frame N and the real frame N-2, and the motion vector map Y2 {y1 (N-2, N), y2 (N-2, N), y3 (N-2, N)} of the object between the real frame N-4 and the real frame N-2. The electronic device can displace the object in the real frame N based on the vectors in the estimated motion vector map to obtain the predicted frame N+1.

[0250] In one embodiment of the present application, after calculating the estimated motion vector map Y3 {y1(N,N+1), y2(N,N+1), y3(N,N+1)}, the calculation module 324 may store the estimated motion vector map Y3 in the cache 320. When the drawing instruction stream intercepted by the interception module 314 is the drawing instruction stream of real frame N+2, the identification module 316 identifies the drawing instruction stream of real frame N+2. The separation module 318 separates static objects from dynamic objects in the scene image of real frame N+2 based on the scene drawing instructions in the drawing instruction stream of real frame N+2, and stores the separated objects in the cache. The identification module 316 stores the UI image data in the cache based on the UI drawing instructions in the drawing instruction stream.

[0251] Furthermore, the matching module 322 matches dynamic objects between the real frame N+2 and the real frame N, and matches static objects between the real frame N+2 and the real frame N. The calculation module 324 can calculate the motion vector map Y4 { y1 (N, N+2), y2 (N, N+2), y3 (N, N+2)} of the objects between the real frame N+2 and the real frame N.

[0252] It will be appreciated that, based on the embodiment described above, calculation module 324 can calculate an estimated motion vector map Y5 between real frame N+2 and predicted frame N+3 based on the motion vector map Y1 of the object between real frame N and real frame N-2, and the motion vector map Y4 of the object between real frame N+2 and real frame N. However, as described above, this may result in image degradation. In one embodiment of the present application, before calculating the estimated motion vector map Y5, it is possible to first determine whether the prediction result of predicted frame N+1 has a significant error. If there is a significant error difference, a check can be performed in advance when predicting the estimated motion vector map between real frame N+2 and predicted frame N+3 to prevent the formation of a poor-quality predicted frame N+3.

[0253] In one embodiment, calculation module 324 may calculate a motion vector map X1 between each object between predicted frame N+1 and real frame N, and then determine whether a significant error exists in the prediction result of predicted frame N+1 based on the motion vector map Y4 of the objects between real frame N+2 and real frame N. For example, calculation module 324 may calculate a first difference between each corresponding vector of motion vector map X1 and motion vector map Y4, and compare the first difference with half of each corresponding vector of motion vector map Y4, such as by subtracting the difference, to obtain a second difference. A larger absolute value of the second difference indicates an inaccurate prediction result. Therefore, when calculating an estimated motion vector map between real frame N+2 and predicted frame N+3, the second difference may be used as a parameter to participate in the calculation of the estimated motion vector map between real frame N+2 and predicted frame N+3.

[0254] See also Figure 2 and see Figure 9 The calibration method of the above image frame is described. Figure 9 This is an example diagram of the object trajectory change in the calibration method according to an embodiment of the present application.

[0255] Figure 9The middle circle represents the same object in consecutively displayed real frames. Positions 1, 2, and 3 are the positions of the object in three consecutive real frames. For example, positions 1, 2, and 3 are the positions in real frame N-4, real frame N-2, and real frame N, respectively. The dashed arrow indicates the actual direction of motion, and position 4 is the position of the object in predicted frame N+1. Using the calibration method described above, calculation module 324 can calculate a first difference between the corresponding vectors of motion vector map X1 and motion vector map Y4, subtract this first difference from half of the corresponding vectors of motion vector map Y4, and obtain a second difference. Based on the magnitude of the second difference, it is determined that position 4 deviates significantly from the actual motion position of the object. Calculation module 324 uses the second difference as a calibration parameter to calculate the estimated motion vector map between real frame N+2 and predicted frame N+3, thereby performing a first calibration of the object's position in predicted frame N+3. Exemplarily, 5 represents the object position of a real frame N+2, 6 represents the position of object 6 in prediction N+3 predicted according to a prediction method of the above embodiment, and 7 represents the position of position 7 in prediction N+3 after data calibration of the real frame N+2, that is, the position of 7 after the first calibration.

[0256] Because user actions can affect the position of objects within the image frame, for example, the frequency and location of user control operations can affect the angle and speed of changes in each object within the image frame. Therefore, the calibration method can further calibrate the generated predicted frame based on the positional relationship between the user's touch location and the UI image (e.g., a game wheel).

[0257] Continue to see Figure 8 The system framework layer also includes a touch module 336. The touch module 336 can obtain the location information of the user touching the electronic device display screen when the electronic device displays the real frame N. The location information may also include the distance between the user's touch location and a preset UI image (such as a game wheel). After obtaining the location information, the touch module 336 sends the location information to the calculation module. The calculation module 336 can further calibrate the predicted frame N+3 based on the location information and the above-mentioned second difference. For example, the calculation module 324 can participate in calculating the estimated motion vector map between the real frame N+2 and the predicted frame N+3 based on the second difference and the above-mentioned location information to verify the predicted position of the object.

[0258] Based on the above calibration method, the accuracy of the predicted frame can be further improved.

[0259] An embodiment of the present application further provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the electronic device, the electronic device executes the prediction method provided in the present application.

[0260] The functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0261] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk, and other media that can store program code.

[0262] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for predicting an image, characterized in that: Used in electronic equipment, including: Obtaining a drawing instruction stream of a first image frame, the drawing instruction stream including one or more first-category drawing instructions for drawing one or more first objects, and one or more second-category drawing instructions for drawing one or more second objects, the first-category drawing instructions being scene drawing instructions, and the second-category drawing instructions being control drawing instructions; acquiring the one or more first objects based on the drawing instruction stream of the first image frame; Acquire one or more third objects in a second image frame, where the second image frame is an image frame preceding the first image frame; calculating a first motion vector between the one or more first objects and the one or more third objects, the one or more third objects being matched with the one or more first objects; Obtaining a second motion vector, where the second motion vector is a motion vector between the one or more third objects and one or more fourth objects in a third image frame, where the one or more third objects match the one or more fourth objects, and the third image frame is an image frame preceding the second image frame; Calculating a third motion vector based on the first motion vector and the second motion vector; A first predicted image frame is obtained based on the first motion vector, the third motion vector, and the one or more first objects.

2. The prediction method according to claim 1, characterized in that Before obtaining the drawing instruction stream of the first image frame, the method includes: The first pointer in the replacement pointer list is replaced with the second pointer, the first pointer points to a first function, and the second pointer points to a second function, wherein the first function is used to draw the first image frame, and the second function is used to identify the drawing instructions of the drawing instruction stream of the first image frame.

3. The prediction method according to claim 2, characterized in that Before acquiring the one or more first objects based on the drawing instruction stream of the first image frame, the method further includes: Based on the second function, the drawing instructions in the drawing instruction stream of the first image frame are identified to determine the first type of drawing instructions, the second type of drawing instructions, and the third type of drawing instructions in the drawing instruction stream of the first image frame.

4. The prediction method according to claim 1, wherein: Before acquiring the one or more third objects in the second image frame, the method further includes: Obtaining a drawing instruction stream of the second image frame; acquiring the one or more third objects in the second image frame based on the first type of drawing instructions in the drawing instruction stream of the second image frame; The one or more third objects are stored.

5. The prediction method according to claim 1, wherein: The acquiring one or more third objects in the second image frame includes: The one or more third objects are acquired based on a third type of drawing instructions in the drawing instructions of the drawing instruction stream of the first image frame, wherein the third type of drawing instructions are image display instructions.

6. The prediction method according to claim 1, characterized in that The one or more first objects include a fifth object, the one or more third objects include a sixth object, the one or more fourth objects include a seventh object, the one or more first objects are matched with the one or more third objects so that the fifth object and the sixth object have the same identifier, and the one or more third objects are matched with the one or more fourth objects so that the sixth object and the seventh object have the same identifier.

7. The prediction method according to any one of claims 1 to 6, characterized in that: There are P image frames between the first image frame and the second image frame, and there are Q image frames between the second image frame and the third image frame, where P and Q are positive integers.

8. The prediction method according to claim 7, characterized in that The one or more first objects include a fifth object and an eighth object, the one or more third objects include a sixth object and a ninth object, the fifth object and the sixth object have the same identifier, and the eighth object and the ninth object have the same identifier. After acquiring the one or more first objects in the first image frame, the method further includes: Based on the identifier of the fifth object, obtaining vertex information of the sixth object; determining that the fifth object is a dynamic object based on that vertex information of the fifth object is different from vertex information of the sixth object; Based on the identifier of the eighth object, acquiring vertex information of the ninth object; determining that the eighth object is a static object based on that the vertex information of the eighth object is the same as the vertex information of the ninth object; Recording that the fifth object is a dynamic object; The eighth object is recorded as a static object.

9. The prediction method according to claim 8, characterized in that The calculating a first motion vector between the one or more first objects and the one or more third objects includes: calculating a first motion component of the first motion vector based on the coordinates of the fifth object and the sixth object; A second motion component of the first motion vector is calculated based on the coordinates of the eighth object and the ninth object.

10. The prediction method according to claim 1, wherein: Before obtaining the second motion vector, the method includes: Obtaining a drawing instruction stream of the third image frame; acquiring the one or more fourth objects in the third image frame based on the first type of drawing instructions in the drawing instruction stream of the third image frame; storing the one or more fourth objects; After acquiring the one or more third objects in the second image frame, the method further includes: calculating a second motion vector between the one or more third objects and the one or more fourth objects; The second motion vector is stored.

11. The prediction method according to claim 8, characterized in that The one or more fourth objects include a seventh object and a tenth object, the sixth object and the seventh object have the same identifier, the ninth object and the tenth object have the same identifier, and calculating the second motion vector between the one or more third objects and the one or more fourth objects includes: calculating a first motion component of the second motion vector based on the coordinates of the sixth object and the seventh object; A second motion component of the second motion vector is calculated based on the coordinates of the ninth object and the tenth object.

12. The prediction method according to claim 8, characterized in that The step of calculating the third motion vector based on the first motion vector and the second motion vector, wherein P=Q=1, includes: A half of the difference between the second motion vector and the first motion vector is used as the third motion vector.

13. The prediction method according to claim 12, characterized in that: The obtaining a first predicted image frame based on the first motion vector, the third motion vector, and the one or more first objects includes: obtaining predicted coordinates of the fifth object based on a sum of half the first motion component of the first motion vector, the first motion component of the third motion vector, and the coordinates of the fifth object; obtaining predicted coordinates of the eighth object based on a sum of half the second motion component of the first motion vector, the second motion component of the third motion vector, and the coordinates of the eighth object; The first predicted image frame is obtained based on the predicted coordinates of the fifth object and the predicted coordinates of the eighth object.

14. The prediction method according to claim 1, characterized in that Also includes: discarding a drawing instruction stream of image frames between the first image frame and the second image frame; The drawing instruction stream of the image frames between the second image frame and the third image frame is discarded.

15. The prediction method according to claim 1, characterized in that Also includes: The drawing instruction stream of the first image frame is drawn.

16. A method for predicting an image, characterized in that: Applied to an electronic device, the electronic device includes an identification module, a matching module, a calculation module, and an estimation module, and the method includes: The recognition module obtains a drawing instruction stream of the first image frame, the drawing instruction stream including one or more first-category drawing instructions for drawing one or more first objects, and one or more second-category drawing instructions for drawing one or more second objects, the first-category drawing instructions being scene drawing instructions, and the second-category drawing instructions being control drawing instructions; The recognition module obtains the one or more first objects by obtaining a drawing instruction based on the first image frame; The matching module acquires one or more third objects in a second image frame, where the second image frame is an image frame before the first image frame; The calculation module calculates a first motion vector between the one or more first objects and the one or more third objects, the one or more third objects being matched with the one or more first objects; The calculation module obtains a second motion vector, where the second motion vector is a motion vector between the one or more third objects and one or more fourth objects in a third image frame, where the one or more third objects match the one or more fourth objects, and the third image frame is an image frame before the second image frame; The calculation module calculates a third motion vector based on the first motion vector and the second motion vector; The estimation module obtains a first predicted image frame based on the first motion vector, the third motion vector and the one or more first objects.

17. The prediction method according to claim 16, characterized in that: The electronic device also includes an interception module and a graphics library. Before the identification module obtains the drawing instruction stream of the first image frame, the interception module replaces the first pointer in the pointer list in the graphics library with a second pointer, where the first pointer points to a first function and the second pointer points to a second function, wherein the first function is used to draw the first image frame and the second function is used by the identification module to identify the drawing instructions in the drawing instruction stream of the first image frame.

18. The prediction method according to claim 17, characterized in that: Before the recognition module obtains the one or more first pairs based on the drawing instructions of the first image frame, the recognition module recognizes the drawing instructions in the drawing instruction stream of the first image frame based on the second function to determine the first category of drawing instructions, the second category of drawing instructions, and the third category of drawing instructions in the drawing instruction stream of the first image frame.

19. The prediction method according to claim 16, characterized in that: Before the matching module obtains one or more third objects in the second image frame, the recognition module also obtains a drawing instruction stream of the second image frame, obtains the one or more third objects in the second image frame based on the first type of drawing instructions in the drawing instruction stream of the second image frame, and the recognition module stores the one or more third objects in the cache of the electronic device.

20. The prediction method according to claim 16, characterized in that The electronic device further includes a separation module, wherein the one or more first objects include a fifth object and an eighth object, the one or more third objects include a sixth object and a ninth object, the fifth object and the sixth object have the same identifier, and the eighth object and the ninth object have the same identifier, and after the recognition module obtains the one or more first objects in the first image frame, the separation module obtains vertex information of the sixth object based on the identifier of the fifth object; The separation module determines that the fifth object is a dynamic object based on the difference between the vertex information of the fifth object and the vertex information of the sixth object; the separation module obtains the vertex information of the ninth object based on the identifier of the eighth object; The separation module determines that the eighth object is a static object based on that the vertex information of the eighth object is the same as the vertex information of the ninth object; The separation module records the fifth object as a dynamic object; and records the eighth object as a static object.

21. The prediction method according to claim 20, characterized in that: The calculation module calculates the first motion vector between the one or more first objects and the one or more third objects, including: the calculation module calculates the first motion component of the first motion vector based on the coordinates of the fifth object and the sixth object; the calculation module calculates the second motion component of the first motion vector based on the coordinates of the eighth object and the ninth object.

22. The prediction method according to claim 16, characterized in that: Before the calculation module obtains the second motion vector, it includes The calculation module obtains a drawing instruction stream of the third image frame; The calculation module obtains the one or more fourth objects in the third image frame based on the first type of drawing instructions in the drawing instruction stream of the third image frame; The computing module stores the one or more fourth objects through a cache of the electronic device; After the computing module acquires the one or more third objects in the second image frame, the method further includes: The calculation module calculates a second motion vector between the one or more third objects and the one or more fourth objects; The calculation module stores the second motion vector in a buffer of the electronic device.

23. The prediction method according to claim 20, characterized in that: The one or more fourth objects include a seventh object and a tenth object, the sixth object and the seventh object have the same identifier, and the ninth object and the tenth object have the same identifier, and the calculation module calculates the second motion vector between the one or more third objects and the one or more fourth objects, including: The calculation module calculates a first motion component of the second motion vector based on the coordinates of the sixth object and the seventh object; The calculation module calculates a second motion component of the second motion vector based on coordinates of the ninth object and the tenth object.

24. The prediction method according to claim 23, characterized in that The calculation module calculates a third motion vector based on the first motion vector and the second motion vector, including: The calculation module uses half of the difference between the second motion vector and the first motion vector as the third motion vector.

25. The prediction method according to claim 24, characterized in that: The estimating module obtains a first predicted image frame based on the first motion vector, the third motion vector, and the one or more first objects, including: The estimation module obtains the predicted coordinates of the fifth object based on a sum of half the first motion component of the first motion vector, the first motion component of the third motion vector, and the coordinates of the fifth object; The estimation module obtains the predicted coordinates of the eighth object based on a sum of half the second motion component of the first motion vector, the second motion component of the third motion vector, and the coordinates of the eighth object; The estimation module obtains the first predicted image frame based on the predicted coordinates of the fifth object and the predicted coordinates of the eighth object.

26. The prediction method according to any one of claims 16 to 25, characterized in that: The electronic device further includes a discarding module configured to discard the drawing instruction stream of the image frames between the first image frame and the second image frame. The discarding module further discards the drawing instruction stream of the image frames between the second image frame and the third image frame.

27. An electronic device, characterized in that: The electronic device comprises a processor and a storage device, wherein the storage device stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the prediction method according to any one of claims 1 to 15.

28. An electronic device, characterized in that: The electronic device comprises a processor and a storage device, wherein the storage device stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the prediction method according to any one of claims 16 to 26.

29. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on the electronic device, enable the electronic device to perform the prediction method according to any one of claims 1 to 15.

30. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on the electronic device, enable the electronic device to perform the prediction method according to any one of claims 16 to 26.

Images