Frame delivery method and apparatus, electronic device, and readable storage medium

Abstract

The present application relates to the field of terminals, and provides a frame delivery method and apparatus, an electronic device, and a readable storage medium. The frame delivery method comprises: acquiring rendering instructions for a first image frame and a second image frame; on the basis of the rendering instruction for the first image frame, rendering the first image frame in a first buffer and delivering the frame for display; on the basis of the rendering instruction for the second image frame, rendering the second image frame in the first buffer; and upon detecting that a rendering load characteristic meets a preset condition or upon detecting a clearing instruction for the first buffer, sending a frame delivery instruction for the second image frame. The first image frame and the second image frame are delivered separately, such that the frame delivery process better aligns with existing GPU frame delivery strategies. Hence, the GPU can promptly carry out frequency boosting, reduce the processing time of each frame delivery task, and accordingly reduce the buffer waiting time during delivery of the second image frame, thereby alleviating the problem of buffer blockage in the frame delivery process, enabling the frame rate after frame interpolation to reach the expected level, and improving user experience.

Description

Frame sending method, device, electronic device and readable storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on January 10, 2024, with application number 202410041697.0 and application name “Frame delivery method, device, electronic device and readable storage medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminals, and in particular to a frame sending method, device, electronic device and readable storage medium. Background Art

[0003] With the rapid development of the gaming industry, more and more users have higher requirements for the smoothness and high frame rate of game images.

[0004] In order to ensure that the user interface (UI) is not distorted during interpolation, there is an interpolation solution that can separate the main screen and the UI in the game screen and only interpolate the main screen.

[0005] However, there is currently a lack of a frame delivery mechanism that matches the interpolation solution, which results in cache blocking during the frame delivery process and the frame rate of the game screen cannot meet expectations. Summary of the Invention

[0006] This application provides a frame sending method, apparatus, electronic device, and readable storage medium. By sending a frame sending instruction for a second frame upon detecting that a rendering load characteristic meets a preset condition or a clear instruction for a first cache is detected, this method can alleviate the cache congestion problem during the post-insertion frame sending process, ensuring that the post-insertion frame rate meets the expected level.

[0007] To achieve the above objectives, this application adopts the following technical solutions:

[0008] In a first aspect, a frame sending method is provided, which is applied to an electronic device running a target program. The frame sending method includes: obtaining drawing instructions for a first frame image and a second frame image, wherein one of the first frame image and the second frame image is a real frame of the target program, and the other is a predicted frame, the predicted frame being predicted based on the real frame and a frame immediately before the real frame. Based on the drawing instructions for the first frame image, the first frame image is drawn in a first cache and sent for display. Based on the drawing instructions for the second frame image, the second frame image is drawn in the first cache. When it is detected that a drawing load characteristic meets a preset condition or a clear instruction for the first cache is detected, a frame sending instruction for the second frame image is sent.

[0009] In the embodiments of the present application, the frame sending method can be applied to electronic devices, including mobile phones, tablet computers, handheld game consoles, wearable devices, augmented reality / virtual reality devices, laptops, ultra-mobile personal computers, netbooks, personal digital assistants, etc.

[0010] In the first aspect, the timing for sending the second frame is determined by detecting whether the drawing load characteristics meet preset conditions or detecting a clear instruction for the first cache. This reduces the cache waiting time when sending the second frame, improves the cache blocking problem during the frame sending process, ensures that the frame rate after the frame insertion reaches the expected level, and improves the user experience.

[0011] In some possible implementations, the drawing load characteristic is the number of executed rendering instructions DrawCall when drawing the second frame of image in the first buffer.

[0012] Detecting that the drawing load characteristic meets a preset range includes: when the number of executed DrawCalls is greater than or equal to a first preset threshold, determining that the drawing load characteristic meets a preset condition.

[0013] In some possible implementations, the total number of DrawCalls for drawing one frame of the target program is N, where N is a positive integer. The first preset threshold is any integer between N / 3 and N / 2.

[0014] In some possible implementations, when a clear instruction for the first cache is detected, the frame delivery method further includes: obtaining a frame delivery time, where the frame delivery time is the duration between when the first frame image is delivered and displayed and when the clear instruction for the first cache is detected. When the frame delivery time is less than a first preset duration, a frequency increase instruction is sent to the graphics processor, where the frequency increase instruction is used to instruct the graphics processor to increase its operating frequency from an original frequency to a target frequency.

[0015] In some possible implementations, the frame sending method further includes: after a second preset time period, sending a recovery instruction to the graphics processor, where the recovery instruction is used to instruct the graphics processor to restore the operating frequency from the target frequency to the original frequency.

[0016] In some possible implementations, the target frame rate of the target program is M, where M is a positive number. The second preset duration is 2Mths of a second.

[0017] In some possible implementations, the first preset duration is one third of the second preset duration.

[0018] In a second aspect, a frame sending device is provided, which is applied to an electronic device that runs a target program. The device includes:

[0019] The acquisition module is configured to obtain drawing instructions for a first image frame and a second image frame, wherein one of the first and second image frames is a real frame of the target program, and the other is a predicted frame, which is predicted based on the real frame and the frame immediately preceding the real frame. The frame delivery module is configured to draw the first image frame in a first buffer according to the drawing instructions for the first image frame and deliver the frame for display. The frame delivery module is further configured to draw the second image frame in the first buffer according to the drawing instructions for the second image frame; when it is detected that the drawing load characteristics meet a preset range or a clear instruction for the first buffer is detected, the second image frame is delivered for display.

[0020] In some possible implementations, the drawing load characteristic is the number of executed rendering instructions DrawCall when drawing the second frame of image in the first buffer.

[0021] The frame sending module is further configured to determine that the drawing load characteristic meets a preset condition when the number of executed DrawCalls is greater than or equal to a first preset threshold.

[0022] In some possible implementations, the total number of DrawCalls for drawing one frame of the target program is N, where N is a positive integer. The first preset threshold is any integer between N / 3 and N / 2.

[0023] In some possible implementations, the frame feeding module is further configured to obtain a frame feeding time, where the frame feeding time is the duration between when the first frame of the image is fed and displayed and when a clear instruction of the first cache is detected. When the frame feeding time is less than a first preset duration, a frequency increase instruction is sent to the graphics processor, the frequency increase instruction being configured to instruct the graphics processor to increase its operating frequency from an original frequency to a target frequency.

[0024] In some possible implementations, the frame sending module is further configured to send a recovery instruction to the graphics processor after a second preset time period, where the recovery instruction is configured to instruct the graphics processor to recover the operating frequency from the target frequency to the original frequency.

[0025] In some possible implementations, the target frame rate of the target program is M, where M is a positive number. The second preset duration is 2Mths of a second.

[0026] In some possible implementations, the first preset duration is one third of the second preset duration.

[0027] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method provided in the first aspect when executing the computer program.

[0028] In a fourth aspect, a chip is provided, comprising: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the method provided in the first aspect.

[0029] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method provided in the first aspect.

[0030] In a sixth aspect, a computer program product is provided, the computer program product comprising: a computer program code, which, when executed by an electronic device, enables the electronic device to execute the method provided in the first aspect.

[0031] Among them, the beneficial effects of the second to sixth aspects can refer to the first aspect and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a schematic diagram of an application scenario of a frame sending method provided in an embodiment of the present application;

[0033] FIG2 is a hardware structure block diagram of an electronic device provided in an embodiment of the present application;

[0034] FIG3 is a system structure block diagram of an electronic device provided in an embodiment of the present application;

[0035] FIG4 is a block diagram of the software structure of the electronic device provided in an embodiment of the present application;

[0036] FIG5 is a flow chart of a frame sending method provided in an embodiment of the present application;

[0037] FIG6 is a flow chart of an implementation of step S506 in the frame sending method provided in an embodiment of the present application;

[0038] FIG7 is a flow chart of another frame sending method provided in an embodiment of the present application;

[0039] FIG8 is a structural block diagram of a frame sending device provided in an embodiment of the present application;

[0040] FIG9 is a schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The technical solution in this application will be described below with reference to the accompanying drawings.

[0042] In the description of the embodiments of this application, unless otherwise specified, " / " represents or. For example, A / B can represent A or B. "And / or" in this article is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" means two or more than two.

[0043] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0044] With the development of technology, mobile electronic devices such as mobile phones and tablets can often support refresh rates of 90Hz, 120Hz or even 140Hz. This means that when running game applications on these electronic devices, there are higher requirements for the smoothness and frame rate of the game screen.

[0045] However, due to the performance limitations of the user's device, when running a game application, the frame rate of the output game screen may not meet the screen refresh rate, resulting in frame drops and lags. In this case, using frame insertion technology to insert frames into the game screen can effectively improve the frame rate and smoothness of the game.

[0046] Currently, there is a universal interpolation solution that can separate the main scene and the user interface (UI) in the game screen, and then interpolate the main scene separately to improve the traditional universal interpolation method that predicts the UI together, resulting in a distorted UI.

[0047] However, after interpolation, this solution sends both the real and predicted frames together. Some graphics processing units (GPUs) lack the appropriate frame delivery strategy, preventing them from increasing their frequency in time when processing frame delivery tasks. This results in low GPU frequency and buffer congestion during the frame delivery process.

[0048] In view of this, the present application provides a frame sending method, which is applied to an electronic device that runs a target program. The frame sending method includes: obtaining drawing instructions for a first frame image and a second frame image, wherein one of the first frame image and the second frame image is a real frame of the target program, and the other frame is a predicted frame, and the predicted frame is predicted based on the real frame and the previous frame of the real frame. According to the drawing instruction of the first frame image, the first frame image is drawn in a first cache and sent for display. According to the drawing instruction of the second frame image, the second frame image is drawn in the first cache. When it is detected that the drawing load characteristics meet the preset conditions or the clearing instruction of the first cache is detected, the frame sending instruction of the second frame image is sent.

[0049] In this application, the first frame of the image is sent for display immediately after drawing is completed, and then the drawing load characteristics are detected to meet the preset conditions or the clearing instruction of the first cache is detected to determine the timing of sending the second frame of the image. The first frame of the image and the second frame of the image are sent separately, so that the frame sending process is more in line with the existing frame sending strategy of the GPU. This enables the GPU to increase the frequency in time, reduce the processing time of each frame sending task, and thus reduce the waiting time of the cache when the second frame of the image is sent, improve the problem of cache blocking during the frame sending process, so that the frame rate after the insertion can meet the expectation, and improve the user experience.

[0050] FIG1 is a schematic diagram of an application scenario of a frame sending method provided in an embodiment of the present application.

[0051] With reference to FIG1 , the application scenario of the embodiment of the present application is first briefly described.

[0052] FIG1 shows an electronic device 100 . When the electronic device 100 receives an operation instructing to start a game application, the electronic device 100 starts the game application in response to the operation.

[0053] When launching a game application, the electronic device 100 can enable the interpolation function based on the pre-enabled game acceleration function. The interpolation function can be automatically enabled or disabled based on factors such as the performance of the electronic device 100, the screen refresh rate, and the rendering difficulty of the game application, or it can be enabled or disabled in response to user operations. In actual applications, the interpolation function can be referred to as image enhancement, intelligent acceleration, intelligent optimization, etc.

[0054] When the interpolation function is turned on, a prompt message can be displayed in the floating window 21 in the interface of the game application, prompting that smart acceleration has been turned on. Alternatively, the switch of the interpolation function can be displayed in the floating window 21.

[0055] In this case, the interpolation function will first intercept the main interface rendering instructions and UI rendering instructions of the game application after detecting the start of UI rendering, and then render the main interface and UI of the game application in different frame buffers respectively.

[0056] Then, interpolation is performed based on two consecutive frames of the real frames of the main interface of the game application to obtain a predicted frame of the main interface.

[0057] Finally, the predicted frame of the main interface is mixed with the UI to obtain the predicted frame of the game interface, and the predicted frame and the real frame are sent for display through the frame sending method.

[0058] FIG2 is a block diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0059] As examples, electronic devices may include mobile phones, tablet computers, handheld game consoles, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0060] 2 , the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0061] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0062] For example, when the electronic device 100 is a mobile phone or a tablet computer, it may include all the components shown in the figure, or may include only some of the components shown in the figure.

[0063] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0064] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0065] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0066] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0067] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0068] The charging management module 140 is configured to receive charging input from a charger.

[0069] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160.

[0070] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0071] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0072] The mobile communication module 150 can provide wireless communication solutions including 2G / 3G / 4G / 5G applied on the electronic device 100.

[0073] The wireless communication module 160 can provide wireless communication solutions for application on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0074] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0075] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0076] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED).

[0077] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display 194, and an application processor. The ISP is used to process data fed back by the camera 193. In some embodiments, the ISP can be set in the camera 193. The camera 193 is used to capture still images or videos.

[0078] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

[0079] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 can include a program storage area and a data storage area.

[0080] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0081] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals.

[0082] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0083] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0084] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0085] The headphone jack 170D is used to connect a wired headphone.

[0086] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0087] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. The gyro sensor 180B can also be used for navigation and somatosensory gaming scenes.

[0088] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0089] The magnetic sensor 180D includes a Hall sensor, and the electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case.

[0090] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0091] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance by infrared or laser.

[0092] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100.

[0093] The ambient light sensor 180L is used to sense the brightness of the ambient light.

[0094] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0095] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy.

[0096] The touch sensor 180K, also known as a "touch device," can be mounted on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen." The touch sensor 180K is used to detect touch operations applied to or near the touch sensor.

[0097] The buttons 190 include a power button, a volume button, etc. The buttons 190 can be mechanical buttons or touch buttons.

[0098] Motor 191 can generate vibration prompts.

[0099] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0100] The SIM card interface 195 is used to connect a SIM card.

[0101] For the scenario in the above example, the operating system of the electronic device 100 may include but is not limited to Symbian, Android, Windows, MacOS, iOS, Blackberry, HarmonyOS, Linux or Unix operating systems.

[0102] FIG3 is a system structure block diagram of an electronic device provided in an embodiment of the present application.

[0103] As an example, when the frame sending method provided in the present application is run on the electronic device 100, the operating system of the electronic device 100 may be Android, and its system structure may refer to FIG. 3 .

[0104] The layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0105] The application layer can include a series of application packages.

[0106] As shown in FIG3 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0107] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0108] As shown in FIG3 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0109] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0110] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0111] The view system includes visual controls, such as controls for displaying text and images. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text message notification icon might include a view for displaying text and a view for displaying images.

[0112] The phone manager is used to provide communication functions for electronic devices, such as call status management (including answering, hanging up, etc.).

[0113] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0114] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0115] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0116] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0117] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0118] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0119] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0120] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0121] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing and layer processing.

[0122] A 2D graphics engine is a drawing engine for 2D drawings.

[0123] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0124] FIG4 is a block diagram of the software structure of the electronic device provided in an embodiment of the present application.

[0125] 4 , the software structure of the electronic device includes modules such as instruction stream interception, frame insertion, synthesis, intelligent frame sending, and display sending.

[0126] 3 , the instruction stream interception module may be deployed in the system library to intercept rendering instructions sent by the target program to the GPU.

[0127] Modules such as frame insertion, synthesis, and intelligent frame delivery can be deployed in the GPU. The frame insertion module is used to insert frames into the main screen, the synthesis module is used to blend the inserted main screen with the UI, and the intelligent frame delivery module is used to execute the frame delivery method provided in this application and transmit the blended screen to the display module at the appropriate time.

[0128] The display module can be deployed in the kernel layer to send the received images to the screen for display.

[0129] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0130] The functional units and modules in the embodiments may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implements the above-mentioned functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, a merged logic circuit, and / or other suitable components that support the described functions.

[0131] Therefore, the modules of each example described in the embodiments of this application can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] In addition, the specific names of the functional units and modules are only for the purpose of distinguishing each other and are not intended to limit the scope of protection of this application. The specific working processes of the units and modules in the above system can be referred to the corresponding processes in the following method embodiments and will not be repeated here.

[0133] FIG5 is a flow chart of a frame sending method provided in an embodiment of the present application.

[0134] In some possible implementations, the real frames of the target program need to be interpolated through an interpolation algorithm before sending the frames. For example, two consecutive real frames of the target program can be obtained by intercepting the instruction stream, and after the two real frames are separated by UI, the two separated real main frames are input into the interpolation algorithm for interpolation processing to obtain the main frame of the predicted frame, and then the main frame of the predicted frame is synthesized with the separated UI to obtain the predicted frame.

[0135] Since interpolation is divided into interpolation frames and extrapolation frames, interpolation frames refer to inserting predicted frames between two real frames, and extrapolation frames refer to inserting predicted frames after two real frames. Therefore, interpolation algorithms are also divided into interpolation frame algorithms and extrapolation frame algorithms. Common interpolation frame algorithms include motion estimation and compensation (EMEC), optical flow algorithms, interpolation algorithms based on deep learning, and interpolation algorithms based on linear predictive coding (LPC). Common interpolation frame algorithms include aligned extrapolation frame algorithms based on recurrent neural networks and extrapolation frame algorithms based on long short-term memory networks.

[0136] Referring to FIG5 , the interpolation method takes an interpolation frame as an example, that is, the first frame image is a predicted frame, and the second frame image is a real frame of the target program. The frame sending method includes:

[0137] S501: Obtain a drawing instruction for a second frame image and a drawing instruction for a third frame image.

[0138] In some possible implementations, the third frame image (F t-2 ) is the second frame image (F t ), the previous frame, the second frame and the third frame are all real frames.

[0139] In this embodiment, the drawing instruction may include a primitive list to be rendered and multiple DrawCalls pointing to the primitive list. When the DrawCalls pointing to the primitive list are executed, a frame of image can be obtained.

[0140] As an example, to obtain the second and third frames, the DrawCall instructions in memory can be intercepted. When the intercepted DrawCall instructions meet preset characteristics, the UI separation process is initiated. When performing UI separation, two frame buffers (FBs) can be created: a second frame buffer and a third frame buffer. The drawing instructions for the main screen of each real frame are then sequentially written into the second frame buffer, and the UI drawing instructions for each real frame are sequentially written into the third frame buffer.

[0141] In some possible implementations, the drawing instructions for the second frame image include drawing instructions for the second frame image main screen in the second buffer, and drawing instructions for the second frame image UI in the third frame buffer. The drawing instructions for the third frame image include drawing instructions for the third frame image main screen in the second buffer, and drawing instructions for the third frame image UI in the third frame buffer.

[0142] S502 : Predicting and obtaining a drawing instruction for the first frame of image according to the drawing instruction for the second frame of image and the drawing instruction for the third frame of image.

[0143] In some possible implementations, in the case of interpolated frames, the first frame image (F t-1 ) is an intermediate frame between the second and third frames. The main frames of the second and third frames can be input into the EMEC algorithm for prediction to obtain a drawing instruction for the main frame of the first frame. The drawing instruction for the main frame of the first frame can be stored in the second frame buffer.

[0144] In some possible implementations, since the UI generally does not change during interpolation, the UI drawing instructions for the second frame can be used as the UI drawing instructions for the first frame. Therefore, the drawing instructions for the first frame include the drawing instructions for the main screen of the first frame in the second frame buffer and the drawing instructions for the UI of the second frame in the third frame buffer.

[0145] S503 : Draw the first frame image in the first buffer according to the drawing instruction of the first frame image.

[0146] S504: Send the first frame of image to frame display.

[0147] S505 : Draw the second frame image in the first buffer according to the drawing instruction of the second frame image.

[0148] In some possible implementations, S503, S504, and S505 may be executed continuously by a piece of code, and these three steps are described together here.

[0149] As an example, when sending frames, when the third frame image (F t-2 ) After the frame is sent, the first frame image (F t-1 ) and the second frame image (F t ) send frames in sequence.

[0150] Generally, before sending a frame, the image can be drawn in the first frame buffer (FB0), and the frame sending operation is performed after the drawing is completed.

[0151] However, in this application, after the frame insertion, the first frame image (F t-1 ) and the second frame image (F t ) continuously sends frames. If the first frame image (Ft-1 ) and the second frame image (F t )'s drawing instructions and frame delivery instructions are submitted to the GPU. Based on the GPU's existing frame delivery strategy, the GPU cannot accurately judge the load and cannot increase the frequency in time, resulting in slow rendering speed of each frame image and blocking the cache.

[0152] Therefore, it is necessary to convert the first frame image (F t-1 ) and the second frame image (F t ) to adjust the frame delivery time of the two frames, staggering the frame delivery time of the two frames to ensure that the GPU can accurately identify complex and timely increase the frequency.

[0153] In some possible implementations, the first frame image (F t-1 ) is synthesized in FB0 and sent to the frame, and then the second frame image (F t ) is synthesized in FB0. Synthesize the second frame image (F t ) will not execute the frame sending operation of FB0 temporarily.

[0154] As an example, this application takes the Open Graphics Library (OpenGL) code as an example to provide an example of drawing the first frame image in the first buffer (FB0). The example code is as follows: glBindFramebuffer(GL_DRAW_FRAMEBUFFER,0); glViewport(0,0,width,height); glScissor(0,0,width,height); DrawTexture(interpolationMainTexture,false); DrawTexture(uiTexture,true);

[0155] Among them, glBindFramebuffer(GL_DRAW_FRAMEBUFFER,0) is used to bind the current drawing frame buffer to the default frame buffer FB0.

[0156] glViewport(0,0,width,height) is used to set the viewport. The viewport can be a rectangular area, where width is the width of the viewport and height is the height of the viewport. OpenGL will map the rendering results to this area.

[0157] glScissor(0,0,width,height) is used to set the clipping window. OpenGL will render the pixels within the clipping window. In this embodiment, the size of the clipping window is consistent with the viewport, that is, the entire viewport is the clipping area, and all pixels will be rendered and displayed.

[0158] DrawTexture(interpolationMainTexture,false) is used to render the main screen of the first frame image into FB0 according to the drawing instructions of the main screen of the first frame image. False is used to indicate that the mixed mode is prohibited when rendering the main screen, that is, the rendered main screen is used to overwrite the original data in FB0.

[0159] DrawTexture(uiTexture, true) is used to draw the UI of the first frame (the same as the UI of the second frame). True is used to enable mixed mode when rendering the UI, that is, the UI is rendered on the main screen and composited with the main screen.

[0160] After executing the above code, the first frame image (F t-1 ).

[0161] After the drawing is completed in FB0, the first frame image in FB0 needs to be sent for frame display. As an example, this application takes OpenGL code as an example to provide an example of sending the first frame image for frame display. The sample code is as follows:

[0162] eglSwapBuffers(display,drawSurface);

[0163] Taking double-buffered rendering as an example, there are two buffers: a front buffer and a back buffer. The front buffer contains the image to be displayed, while the back buffer is used for rendering. In this embodiment, the default frame buffer (FB0) is the back buffer. This code is used to swap the image in the default frame buffer (FB0) to the front buffer. This image swap process is called the frame sending process.

[0164] Once the image in the default frame buffer (FB0) is swapped to the front buffer, it can be displayed on the screen. This completes the steps for sending the first frame to display. You can then draw the second frame on FB0 using the same code used to draw the first frame, which is not detailed here.

[0165] In some possible implementations, the rendering pipeline can be configured before drawing the first frame and restored after drawing the second frame. The rendering pipeline configuration can include shaders, frame buffers, rendering resources, pipeline state, etc. Restoring the rendering pipeline clears the corresponding frame buffer to prepare for the next rendering task.

[0166] S506: Whether the drawing load characteristic is detected to meet the preset condition or a clear instruction of the first cache is detected. If so, execute S507; otherwise, continue detecting.

[0167] After executing S505, the second frame image is stored in FB0, and waits for a suitable frame sending opportunity, and executes S507 to send the second frame image for display.

[0168] In some possible implementations, the appropriate frame sending timing may be when it is detected that the drawing load characteristics meet a preset condition.

[0169] As an example, the drawing load feature is used to indicate the load of the GPU when drawing the second frame of image, which may be the number of DrawCall calls, the clock frequency of the GPU, the usage rate of the GPU video memory (VRAM), the power consumption of the GPU, etc.

[0170] When the drawing load characteristics meet the preset conditions, it means that the GPU drawing load has increased to a certain level, that is, the GPU is running at a higher frequency. At this time, sending the frame sending instruction for the second frame image can ensure that the second frame image can be drawn quickly and that it can be sent for display in time after the second frame image is drawn.

[0171] In some possible implementations, the drawing load characteristic is a number of executed rendering instructions DrawCall when drawing the second frame of image in the first buffer. Detecting that the drawing load characteristic meets a preset range includes determining that the drawing load characteristic meets a preset condition when the number of executed DrawCall is greater than or equal to a first preset threshold.

[0172] For example, the drawing instructions for the second frame of image include multiple DrawCalls pointing to the primitive list. The second frame of image can be obtained by executing the DrawCalls in sequence. Therefore, the number of DrawCalls executed can be used to represent the drawing load of the GPU.

[0173] As an example, when counting DrawCalls, the execution times of specific DrawCalls can be counted. For example, the execution times of DrawCalls including "glDrawArrays", "glDrawElements", "glDrawElementsInstanced", etc., which are used to indicate rendering and drawing, can be counted.

[0174] In some possible implementations, obtaining the number of DrawCall executions can involve using GPU performance analysis tools to obtain GPU activity, including the number of DrawCalls. Alternatively, third-party analysis libraries, such as GLIntercept or gDEBugger, can be used to obtain the number of DrawCalls. Alternatively, monitoring tools such as counters can be embedded in the target program through code implantation or instrumentation to obtain the number of DrawCalls.

[0175] In some possible implementations, the total number of DrawCalls for drawing one frame of the target program is N, where N is a positive integer. The first preset threshold is any integer between N / 3 and N / 2. For example, when N is 600, the first preset threshold can be any integer between 200 and 300.

[0176] That is, when 1 / 3 to 1 / 2 of the total DrawCall in the drawing instructions have been executed, it can be determined that the drawing load characteristics meet the preset conditions.

[0177] The total number of DrawCalls required to draw a frame of the target program can be determined by analyzing the target program code in advance or obtained from the developer of the target program. Alternatively, the total number of DrawCalls required to draw a frame of the target program can be determined by empirically analyzing the target program code.

[0178] As an example, assume that the total number of DrawCalls in the target program is 600 and the first preset threshold is 250. After drawing the second frame in FB0, the number of executed DrawCalls such as "glDrawArrays", "glDrawElements", and "glDrawElementsInstanced" can be obtained through the GPU performance analysis tool. When the number of executed DrawCalls is greater than or equal to 250, the drawing load characteristics meet the preset conditions.

[0179] In some other possible implementations, the appropriate frame sending timing may also be when a clear instruction of the first buffer (FB0) is detected.

[0180] As an example, the clearing instructions of FB0 may include a drawing instruction that covers FB0, a clearing instruction that clears FB0, etc. For example, the clearing instructions may include "glBindFramebuffer", "glInvalidateFramebuffer", and "glClear" that point to FB0.

[0181] In some possible implementations, the game rendering instruction stream can be intercepted in memory. The game rendering instruction stream includes multiple DrawCalls. When any of the instructions "glBindFramebuffer", "glInvalidateFramebuffer", and "glClear" appear in the intercepted DrawCall, it is confirmed that the clear instruction of the first buffer (FB0) has been detected.

[0182] Since the second frame image is drawn in FB0, and FB0 is called by many instructions in actual applications, when the second frame image is drawn but not yet sent, if another instruction calls FB0, the image in FB0 will be cleared or overwritten.

[0183] Therefore, when an instruction to clear FB0 is detected, a frame sending instruction for the second frame image needs to be issued to ensure that the second frame image is not covered, contaminated or cleared.

[0184] FIG6 is a flow chart of an implementation of step S506 in the frame sending method provided in an embodiment of the present application.

[0185] In some possible implementations, referring to FIG5 and FIG6 , when a clear instruction of the first cache is detected, the frame sending method further includes:

[0186] S5061. Obtain frame sending time.

[0187] In some possible implementations, the frame delivery time is the duration between the time when the first frame of the image is delivered for display and the time when the clear instruction of the first cache is detected.

[0188] For example, to obtain the frame delivery time, the timestamp of each instruction execution can be recorded. When a clear instruction for the first cache is detected, the timestamp of when the first frame of image is delivered for display is obtained. The time difference between the timestamp of the clear instruction for the first cache and the timestamp of when the first frame of image is delivered for display is the frame delivery time.

[0189] S5062: When the frame sending time is less than the first preset time length, execute S5063; otherwise, execute S507.

[0190] When the frame sending time is less than the first preset time length, it is confirmed that the time of detecting the clearing instruction of the first buffer (FB0) is much less than the second preset time length.

[0191] If the time it takes to clear the first buffer (FB0) is significantly less than the second preset duration, forcing the second frame to be delivered will still prevent the GPU from increasing its frequency in time, causing cache congestion. The second preset duration is the estimated time it takes for the GPU to render the second frame in FB0 after the frequency has been increased in time.

[0192] As an example, in some possible implementations, the second preset duration can be determined based on a target frame rate M when the target program is displayed on the screen, where M is a positive integer. For example, the second preset duration T (in seconds) can be calculated using the following formula:

[0193] As an example, the target frame rate M may be 90 frames per second (FPS), and the second preset time length T may be 1 / 180 second, approximately 5.6 milliseconds.

[0194] In some possible implementations, the first preset duration may be one third of the second preset duration. For example, when the second preset duration is 1 / 180 second, the first preset duration may be 1 / 540 second, approximately 1.85 milliseconds.

[0195] S5063. Send a frequency increase instruction to the graphics processor.

[0196] In some possible implementations, the frequency increase instruction is used to instruct the graphics processor to increase the operating frequency from an original frequency to a target frequency.

[0197] To address the issue of the GPU being unable to increase its frequency in time when forcing the second frame to be delivered is less than the first preset duration, causing cache congestion, you can proactively send a frequency increase command to the GPU, raising the GPU's operating frequency from its original frequency to the target frequency. This reduces the time it takes the GPU to render the second frame and alleviates the cache congestion issue.

[0198] As an example, in order to ensure battery life and reduce the power consumption of the GPU when it is idle, the original frequency of the GPU can be 220MHz. When the frame delivery time is less than the first preset duration, the GPU cannot respond to the frequency increase in time and will still run at a frequency of 220MHz. At this time, the intelligent frame delivery module can send a frequency increase instruction to the GPU through cross-thread communication, instructing the GPU to increase the frequency to 1000MHz (target frequency). In this case, the GPU can render the second frame of the image at a faster speed, reducing the time it takes for the GPU to render the second frame of the image. And immediately send the second frame of the image for display after the second frame of the image is rendered.

[0199] S5064: After the second preset time period, send a recovery instruction to the graphics processor.

[0200] In some possible implementations, the restoration instruction is used to instruct the graphics processor to restore the operating frequency from the target frequency to the original frequency.

[0201] As an example, after the GPU receives the frequency increase instruction, it will continue to work according to the target frequency indicated by the frequency increase instruction. This will increase the GPU power consumption and affect the battery life. Therefore, after the second preset time, the intelligent frame sending module can send a recovery instruction to the GPU through cross-thread communication, instructing the GPU to increase the frequency according to the scheduling strategy, that is, to restore the frequency from 1000MHz (target frequency) to the original frequency (220MHz), and then automatically increase or decrease the frequency according to the load level.

[0202] S507: Send the second frame of image to the display frame.

[0203] In some possible implementations, the method of sending the second frame of image for display is the same as the method of sending the first frame of image for display, which will not be described in detail here.

[0204] FIG7 is a flow chart of another frame sending method provided in an embodiment of the present application.

[0205] In some possible implementations, referring to FIG7 , the interpolation method takes an interpolated frame as an example, the first frame image is a real frame of the target program, and the second frame image is a predicted frame. The frame sending method includes:

[0206] S701: Obtain a drawing instruction for a first frame image and a drawing instruction for a third frame image.

[0207] In some possible implementations, the implementation of S701 is the same as that of S501 and is not described in detail here.

[0208] S702 : Predicting and obtaining a drawing instruction for the second frame of image based on the drawing instruction for the first frame of image and the drawing instruction for the third frame of image.

[0209] In some possible implementations, in the case of interpolated frames, the second frame is an extrapolated frame of the first and third frames, i.e., a predicted frame following the first frame. The main frames of the first and third frames can be input into a pre-trained long short-term memory network-based interpolation frame algorithm for prediction, thereby obtaining drawing instructions for the main frame of the second frame. The drawing instructions for the main frame of the second frame can be stored in the second frame buffer.

[0210] In some possible implementations, since the UI generally does not change during interpolation, the UI drawing instructions for the first frame can be used as the UI drawing instructions for the second frame. Therefore, the drawing instructions for the second frame include the drawing instructions for the second frame main screen in the second frame buffer and the drawing instructions for the UI of the first frame in the third frame buffer.

[0211] S703 : Draw the first frame image in the first buffer according to the drawing instruction of the first frame image.

[0212] S704: Send the first frame of image to frame display.

[0213] S705 : Draw the second frame image in the first buffer according to the drawing instruction of the second frame image.

[0214] S706: Whether it is detected that the drawing load characteristic meets the preset condition or a clear instruction of the first cache is detected. If it is detected, execute S507; otherwise, continue to detect.

[0215] S707: Send the second frame of image to the display frame.

[0216] In some possible implementations, the implementation of S703 to S707 is the same as that of S503 to S507, and will not be described in detail here.

[0217] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated.

[0218] It is obvious that those skilled in the art can make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0219] Corresponding to the frame delivery method provided in the above embodiment, FIG8 is a structural block diagram of a frame delivery device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0220] 8, a frame sending device is applied to an electronic device that runs a target program. The device includes:

[0221] The acquisition module 81 is used to obtain drawing instructions for the first frame image and the second frame image, where one of the first frame image and the second frame image is a real frame of the target program, and the other is a predicted frame, which is predicted based on the real frame and the previous frame of the real frame.

[0222] The frame sending module 82 is used to draw the first frame image in the first buffer and send the frame for display according to the drawing instruction of the first frame image.

[0223] The frame sending module 82 is further configured to draw the second frame image in the first cache according to the drawing instruction of the second frame image; and send the second frame image for display when it is detected that the drawing load characteristic meets the preset range or a clear instruction of the first cache is detected.

[0224] In some possible implementations, the drawing load characteristic is the number of executed rendering instructions DrawCall when drawing the second frame of image in the first buffer.

[0225] The frame sending module 82 is further configured to determine that the drawing load characteristic meets a preset condition when the number of executed DrawCalls is greater than or equal to a first preset threshold.

[0226] In some possible implementations, the total number of DrawCalls for drawing one frame of the target program is N, where N is a positive integer. The first preset threshold is any integer between N / 3 and N / 2.

[0227] In some possible implementations, the frame delivery module 82 is further configured to obtain a frame delivery time, where the frame delivery time is the duration between when the first frame of the image is delivered for display and when the clear instruction of the first cache is detected. When the frame delivery time is less than a first preset duration, a frequency increase instruction is sent to the graphics processor, instructing the graphics processor to increase its operating frequency from an original frequency to a target frequency.

[0228] In some possible implementations, the frame sending module 82 is further configured to send a recovery instruction to the graphics processor after a second preset time period, where the recovery instruction is configured to instruct the graphics processor to recover the operating frequency from the target frequency to the original frequency.

[0229] In some possible implementations, the target frame rate of the target program is M, where M is a positive number. The second preset duration is 2Mths of a second.

[0230] In some possible implementations, the first preset duration is one third of the second preset duration.

[0231] It should be understood that the hardware system and chip in the embodiments of the present application can execute the various frame sending methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0232] An embodiment of the present application also provides another electronic device, including a processor and a memory.

[0233] Memory is used to store computer programs that can be executed on the processor.

[0234] The processor is used to execute the processing steps in the frame sending method as described above.

[0235] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions; when the computer-readable storage medium is run on an electronic device, the electronic device executes the method as shown above.

[0236] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0237] The computer-readable storage medium may be any available medium that can be accessed by a computer or may include one or more servers, data centers, and other data storage devices that can be integrated with the medium.

[0238] The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium (eg, a solid state disk (SSD)).

[0239] An embodiment of the present application also provides a computer program product containing computer instructions, which, when executed on an electronic device, enables the electronic device to execute the technical solution shown above.

[0240] FIG9 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip shown in FIG9 can be a general-purpose processor or a dedicated processor. The chip includes a processor 901. The processor 901 is used to support the electronic device in executing the technical solution shown above.

[0241] Optionally, the chip further includes a transceiver 902, which is configured to accept control of the processor 901 and to support the communication device in executing the aforementioned technical solution.

[0242] Optionally, the chip shown in FIG9 may further include: a storage medium 903 .

[0243] It should be noted that the chip shown in Figure 9 can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.

[0244] The electronic device, computer storage medium, computer program product, and chip provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0245] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples.

[0246] For example, some steps in the various embodiments of the above method may be unnecessary, or some new steps may be added, etc. Or any two or more of the above embodiments may be combined. Such modifications, variations, or combinations also fall within the scope of the embodiments of the present application.

[0247] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced with each other. For the sake of brevity, they will not be repeated here.

[0248] It should also be understood that the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0249] It should also be understood that in the embodiments of the present application, "pre-setting" and "pre-definition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device (for example, including an electronic device), and the present application does not limit its specific implementation method.

[0250] It should also be understood that the division of the modes, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various modes, categories, situations and embodiments can be combined without contradiction.

[0251] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0252] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A frame sending method, characterized in that, Applied to an electronic device that runs a target program, the method includes: Obtain the drawing instructions for the first frame image and the second frame image, where one of the first frame image and the second frame image is the real frame of the target program, and the other is a predicted frame, and the predicted frame is predicted based on the real frame and the previous frame of the real frame; According to the drawing instructions of the first frame image, draw the first frame image in the first buffer and send the frame for display; Draw the second frame image in the first buffer according to the drawing instructions of the second frame image; When it is detected that the drawing load feature meets the preset condition or a clear instruction for the first buffer is detected, send a frame sending instruction for the second frame image.

2. The method according to claim 1, wherein The drawing load feature is the number of executed rendering instructions DrawCall when drawing the second frame image in the first buffer; The detection that the drawing load feature meets the preset range includes: When the number of executed DrawCall is greater than or equal to a first preset threshold, it is determined that the drawing load feature meets the preset condition.

3. The method according to claim 2, characterized in that, The total number of DrawCall for drawing one frame image of the target program is N, and N is a positive integer; The first preset threshold is any integer between N / 3 and N / 2.

4. The method according to claim 1, wherein When a clear instruction for the first buffer is detected, the method further includes: Obtain the frame sending time, which is the duration between the time when the first frame image is sent for display and the detection of the clear instruction for the first buffer; When the frame sending time is less than a first preset duration, send a frequency increase instruction to the graphics processor, and the frequency increase instruction is used to instruct the graphics processor to increase the operating frequency from the original frequency to the target frequency.

5. The method according to claim 4, characterized in that The method further includes: After a second preset duration, send a recovery instruction to the graphics processor, and the recovery instruction is used to instruct the graphics processor to restore the operating frequency from the target frequency to the original frequency.

6. The method according to claim 5, wherein The target frame rate of the target program is M, and M is a positive number; The second preset duration is 1 / (2M) seconds.

7. The method according to claim 5 or 6, characterized in that, The first preset duration is one-third of the second preset duration.

8. A frame sending device, characterized in that, Applied to an electronic device that runs a target program, the device includes: An acquisition module, configured to obtain the drawing instructions for the first frame image and the second frame image, where one of the first frame image and the second frame image is the real frame of the target program, and the other is a predicted frame, and the predicted frame is predicted based on the real frame and the previous frame of the real frame; A frame sending module, configured to draw the first frame image in the first buffer and send the frame for display according to the drawing instructions of the first frame image; The frame sending module is further configured to draw the second frame image in the first buffer according to the drawing instructions of the second frame image; when it is detected that the drawing load feature meets the preset range or a clear instruction for the first buffer is detected, send the second frame image for frame display.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 7.

Images