A frame rate improvement method, electronic device and storage medium

Abstract

This application discloses a frame rate enhancement method, electronic device, and storage medium, relating to the field of display processing technology. The method includes: when it is determined that the frame rate of a game screen needs to be increased, creating an interpolation rendering thread; generating an interpolated frame image in the interpolation rendering thread based on frame rendering data; and inserting the interpolated frame image before or after the original image generated by the main rendering thread based on the frame rendering data. Thus, when the frame rate of a game screen needs to be increased, the electronic device creates an interpolation rendering thread. This interpolation rendering thread can share some of the rendering tasks of the main rendering thread, allowing the main rendering thread more time to handle complex scenes and real-time interactions, reducing stuttering and latency caused by overload of the main rendering thread. This enables the game screen rendering to maintain a stable target frame rate, improves the rendering effect of the game screen, and enhances the user's gaming experience.

Description

Frame rate improvement method, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on February 1, 2024, with application number 202410144890.7 and invention name “A frame rate enhancement method, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of display processing technology, and in particular to a frame rate improvement method, electronic device, and storage medium. Background Art

[0003] With the advancement of science and technology, the screen resolution of electronic devices such as mobile phones and computers has continued to increase, and gaming scenarios have become increasingly diverse, in order to enhance the user's gaming experience. However, high-resolution screens and complex gaming scenarios require electronic devices to have stronger computing and image processing capabilities, which leads to increased power consumption. Furthermore, since the hardware resources of electronic devices are relatively limited, if the hardware resources of electronic devices cannot provide the corresponding computing and image processing capabilities, the electronic devices will experience heating, shortened battery life, and game frame drops, which will in turn reduce the user's gaming experience.

[0004] In related technologies, interpolation technology has been developed to address the aforementioned frame drop problem in games. Interpolation technology increases the frame rate by inserting new frames between existing frames, thereby reducing game power consumption and improving game smoothness.

[0005] However, even if interpolation technology is applied, the game load may be too heavy due to the rich game scenes, or the hardware resources of the electronic device may be too limited, resulting in poor rendering effects of the game scenes, thereby reducing the user's gaming experience. Summary of the Invention

[0006] The purpose of this application is to provide a frame rate improvement method, electronic device and storage medium, which can improve the user's gaming experience by rendering the game scene to a stable target frame rate.

[0007] In the first aspect, an embodiment of the present application provides a frame rate improvement method, which is applied to electronic devices such as mobile phones and computers. The method includes: when it is determined that the frame rate of the game screen needs to be improved, creating an interpolation rendering thread; generating an interpolation image in the interpolation rendering thread according to the frame rendering data; inserting the interpolation image before or after the original image generated by the main rendering thread according to the frame rendering data. Thus, when it is necessary to improve the frame rate of the game screen, the electronic device will create an interpolation rendering thread, which can share part of the rendering tasks of the main rendering thread, so that the main rendering thread can have more time to process complex scenes and real-time interactions, etc., reducing the jamming and delay caused by the overload of the main rendering thread, so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0008] In some specific implementations, when it is determined that the frame rate of the game screen needs to be improved, an interpolation rendering thread is created, including: when the electronic device is in a game scene and the electronic device meets the interpolation conditions, an interpolation rendering thread is created, and the interpolation conditions include one or more of the following: the image processor load value of the electronic device is higher than the first load threshold, the central processing unit load value of the electronic device is higher than the second load threshold, and the real-time game frame rate value of the electronic device is lower than the frame rate threshold. Therefore, when the frame rate of the game screen needs to be improved, the electronic device will create an interpolation rendering thread, which can share part of the rendering tasks of the main rendering thread, so that the main rendering thread can have more time to process complex scenes and real-time interactions, etc., reducing the freezes and delays caused by the overload of the main rendering thread, so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0009] In some specific implementations, the frame rendering data in the interpolation rendering thread is transferred from the main rendering thread to the interpolation rendering thread, and the frame rendering data includes one or more of texture data and camera data.

[0010] In some specific implementations, the frame rendering data in the interpolation frame rendering thread is created by the interpolation frame rendering thread, and the frame rendering data includes context data.

[0011] In some specific implementations, the method for the interpolation rendering thread to receive frame rendering data includes: after the frame rendering data in the main rendering thread is passed to the circular queue of the interpolation rendering thread, the interpolation rendering thread receives the frame rendering data according to the order of the frame rendering data in the circular queue. Therefore, when it is necessary to increase the frame rate of the game screen, the electronic device will create an interpolation rendering thread, which can share part of the rendering tasks of the main rendering thread, so that the main rendering thread can have more time to process complex scenes and real-time interactions, etc., reducing the jamming and delay caused by the overload of the main rendering thread, etc., so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0012] In some specific implementations, the interpolated rendering thread receives frame rendering data by copying the sequence number of the frame rendering data in the main rendering thread to a shared data structure of the main rendering thread, and then receiving the frame rendering data by matching the sequence number of the frame rendering data in the shared data structure. This allows the interpolated rendering thread to easily and conveniently determine which texture data to use for rendering, thereby improving subsequent rendering efficiency.

[0013] In some specific implementations, after inserting the interpolated image generated by the interpolated rendering thread based on the frame rendering data before or after the original image generated by the main rendering thread based on the frame rendering data, the method also includes: when it is determined that there is no need to increase the frame rate of the game screen, generating the original image based on the frame rendering data in the main rendering thread. This is because an independent interpolated rendering thread can significantly improve the frame rate under heavy game load, but not all games are suitable for creating an independent interpolated rendering thread. Especially for cases where the game itself is lightly loaded, using an independent interpolated rendering thread may result in increased power consumption compared to a non-independent interpolated rendering thread. Therefore, switching between different solutions should be performed depending on the specific situation.

[0014] In some specific implementations, when it is determined that there is no need to increase the frame rate of the game screen, the original image is generated according to the frame rendering data in the main rendering thread, including: when the electronic device meets the conditions for stopping interpolation, the original image is generated according to the frame rendering data in the main rendering thread, and the conditions for stopping interpolation include one or more of the real-time game frame rate value of the electronic device being higher than the frame rate threshold, the execution time of the main rendering thread being lower than the first time threshold, and the execution time of the interpolation rendering thread being lower than the second time threshold. This is because an independent interpolation rendering thread can significantly improve the frame rate under heavy game load, but not all games are suitable for creating an independent interpolation rendering thread. Especially when the game itself is lightly loaded, using an independent interpolation rendering thread may result in increased power consumption compared to a non-independent interpolation rendering thread. Therefore, switching between different solutions should be performed depending on the specific situation.

[0015] In some specific implementations, after generating the original image according to the frame rendering data in the main rendering thread, the method further includes: deleting the interpolated frame rendering thread when a thread deletion condition is met.

[0016] In some specific implementations, when the thread deletion conditions are met, the interpolation rendering thread is deleted, including: obtaining the historical switching frequency, which is the ratio of the historical switching count to the target time, and the historical switching count is the number of times the game screen was generated from the interpolation rendering thread and the main rendering thread to the main rendering thread; if the historical switching frequency is lower than the frequency threshold, the interpolation rendering thread is deleted. This is because when the interpolation thread switches from independent thread interpolation to non-independent thread interpolation, the interpolation rendering thread can be deleted or retained. The specific decision of whether to delete or retain can be made based on the actual situation of the game.

[0017] In some specific implementations, when the thread deletion conditions are met, the interpolation rendering thread is deleted, including: obtaining the historical switching time, which is the time when the game screen was last generated based on the interpolation rendering thread and the main rendering thread; if the time difference between the historical switching time and the current time is greater than a third duration threshold, then deleting the interpolation rendering thread. This is because when the interpolation thread switches from an independent thread to a non-independent thread, the interpolation rendering thread can be deleted or retained. The specific decision of whether to delete or retain can be made based on the actual situation of the game.

[0018] In a second aspect, the present application discloses an electronic device comprising a processor and a memory; the memory is used to store computer execution commands; the processor is used to execute the computer execution commands stored in the memory, so that the processor executes the frame rate improvement method as in the first aspect.

[0019] In a third aspect, the present application discloses a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed, the frame rate improvement method in the first aspect is implemented.

[0020] In a fourth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the frame rate improvement method in the first aspect.

[0021] In the fifth aspect, the present application discloses a frame rate improvement device, which is applied to electronic devices such as mobile phones and computers. The device includes: a creation module, a generation module and an interpolation module; the creation module is used to create an interpolation rendering thread when it is determined that the frame rate of the game screen needs to be improved; the generation module is used to generate an interpolation image in the interpolation rendering thread according to the frame rendering data; the interpolation module is used to insert the interpolation image before or after the original image generated by the main rendering thread according to the frame rendering data. Thus, when the frame rate of the game screen needs to be improved, the electronic device will create an interpolation rendering thread, which can share part of the rendering tasks of the main rendering thread, so that the main rendering thread can have more time to handle complex scenes and real-time interactions, etc., reducing the jamming and delay caused by the overload of the main rendering thread, so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0022] Compared with the existing technology, this application has the following beneficial effects:

[0023] This application discloses a frame rate improvement method, electronic device, and storage medium. When the electronic device needs to increase the frame rate of a game screen, it can create an independent interpolation rendering thread. Subsequently, the interpolation image generated by the interpolation rendering thread based on the frame rendering data is inserted before or after the original image generated by the main rendering thread based on the frame rendering data, thereby forming a game screen with an increased frame rate. In this way, the created interpolation rendering thread can share some of the rendering tasks of the main rendering thread, allowing the main rendering thread to have more time to handle complex scenes and real-time interactions, etc., reducing the lag and delay caused by the main rendering thread overload. Furthermore, when the main thread is blocked, if the interpolation rendering thread is not blocked, the interpolation rendering thread can take over more rendering tasks. When the main thread returns to normal, the interpolation rendering thread has completed some or all of the rendering work. The main rendering thread can then quickly catch up with the rendering progress and merge the rendered original image into the interpolation image rendered by the interpolation rendering thread, thereby maintaining the consistency and smoothness of the overall rendering, allowing the rendering of the game screen to maintain a stable target frame rate, improving the rendering effect of the game screen, and thus enhancing the user's gaming experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic diagram of a heavy-load game scenario;

[0025] FIG2 is a schematic diagram of a multi-threaded rendering;

[0026] FIG3 is a schematic diagram of interpolating frames in a game rendering thread;

[0027] FIG4 is a flow chart of a frame rate improvement method provided in an embodiment of the present application;

[0028] FIG5 is a schematic diagram of an interpolation rendering thread provided in an embodiment of the present application;

[0029] FIG6A is a schematic diagram of a dual-queue method provided in an embodiment of the present application;

[0030] FIG6B is a schematic diagram of a circular queue provided in an embodiment of the present application;

[0031] FIG7 is a schematic diagram of single-frame interpolation through an interpolation rendering thread provided by an embodiment of the present application;

[0032] FIG8 is a schematic diagram of overcoming multiple frame insertion blocking by using a frame insertion rendering thread according to an embodiment of the present application;

[0033] FIG9 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0034] FIG10 is a schematic diagram of a frame rate improvement device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] First, let’s explain the technical terms involved in this application:

[0036] Interpolation increases the frame rate by inserting new frames between existing ones, thereby reducing power consumption and improving game smoothness. Interpolation can be categorized into single-frame interpolation and multi-frame interpolation.

[0037] Single interpolation is a simple interpolation technique that usually inserts an extra frame between two adjacent original frames to make the action smoother between the two frames. The main goal of this method is to increase the frame rate of the video or game, thereby improving the visual effect, but the generated interpolated frames are relatively few.

[0038] Multiple interpolation technology inserts multiple additional frames between two adjacent original frames to capture movements in more detail and improve the visual effects of videos or games. This method may be more effective in processing fast movements or when higher frame rates are required, but it also increases the computational burden.

[0039] The terms "first", "second" and "third" in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0040] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0041] As mentioned earlier, high-resolution screens and complex gaming scenarios require electronic devices to provide more computing and image processing capabilities, which leads to increased power consumption. Furthermore, because electronic devices have limited hardware resources, if these resources cannot provide the corresponding computing and image processing capabilities, the devices will overheat, battery life will be shortened, and game frames will drop, which will seriously reduce the user's gaming experience.

[0042] See Figure 1, which is a schematic diagram of a game heavy load scenario.

[0043] A heavy-load game scenario refers to a scenario where the game is overloaded due to overly complex game graphics, physical calculations, or artificial intelligence logic processing. If an electronic device renders a heavy-load scenario like Figure 1 and lacks the corresponding computing and image processing capabilities, it can cause overheating, reduced battery life, and frame drops.

[0044] In related technologies, interpolation technology has been developed to address the aforementioned game frame drop issues. Interpolation refers to the process of adding new frames between two frames to increase the frame rate. This method can effectively reduce game power consumption and improve game smoothness. See Figure 2, which shows a schematic diagram of multi-threaded rendering. After the main thread creates two child threads (i.e., a logic thread and a rendering thread), the created logic thread performs logic initialization, and the rendering thread performs rendering context initialization. After the two child threads have completed initialization and all three threads reach the barrier point, the logic thread updates game resources and generates a queue of rendering instructions through steps such as the Mono::init() callback, swapping rendering queues, and the Mono::Update() callback. The rendering thread then consumes the rendering instructions to render the corresponding game screen. Furthermore, after the main thread updates the barrier point, the main thread, logic thread, and rendering thread perform barrier synchronization again. Subsequently, the subsequent game scene rendering steps proceed.

[0045] However, referring to Figure 3, which illustrates interpolation within a game rendering thread, single or multiple interpolation causes the rendering thread to simultaneously render both real and interpolated frames, resulting in excessively long rendering times for the game scene, delaying the normal operation of the logic thread and causing scene rendering bottlenecks.

[0046] In view of this, the present application discloses a frame rate improvement method, electronic device, and storage medium. When the electronic device needs to increase the frame rate of the game screen, it can create an independent interpolation rendering thread. Subsequently, the interpolation image generated by the interpolation rendering thread based on the frame rendering data is inserted before or after the original image generated by the main rendering thread based on the frame rendering data, thereby forming a game screen with an increased frame rate. In this way, the created interpolation rendering thread can share part of the rendering tasks of the main rendering thread, allowing the main rendering thread to have more time to process complex scenes and real-time interactions, etc., reducing the lag and delay caused by the overload of the main rendering thread. Furthermore, when the main thread is blocked, if the interpolation rendering thread is not blocked, the interpolation rendering thread can take over more rendering tasks. When the main thread returns to normal, the interpolation rendering thread has completed part or even all of the rendering work. Then, the main rendering thread can quickly catch up with the rendering progress and merge the rendered original image into the interpolation image rendered by the interpolation rendering thread, thereby maintaining the consistency and smoothness of the overall rendering, so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0047] See Figure 4, which is a flow chart of a frame rate improvement method provided in an embodiment of the present application. The method is applied to electronic devices such as mobile phones and computers, and includes:

[0048] S401: When the electronic device is in the target game scene, detect whether the electronic device meets the frame insertion condition. If so, execute S402.

[0049] Target games are those with high frame rate requirements, such as action games, shooting games, racing games, role-playing games, and strategy games. These target games need to ensure smooth game display, allowing players to make decisions and operate more quickly. It should be noted that this application does not limit the specific target games.

[0050] In some specific implementations, when the electronic device is in a target game scene and a load value of a graphics processing unit (GPU) of the electronic device is higher than a first load threshold, step S402 may be executed.

[0051] The GPU load value indicates the amount of tasks the GPU is performing or the amount of data it is processing. A higher GPU load value indicates that the GPU is performing more complex tasks or processing larger amounts of data, requiring more computing resources and image processing power to ensure smooth game display. Furthermore, the GPU load value typically varies with the complexity of the game screen. For example, the GPU load value may increase when the game screen contains more characters or objects, more complex lighting effects, or requires a higher resolution.

[0052] In some examples, the first load threshold may be 90%, 95%, or the like. That is, if the GPU load value is higher than 90% or 95%, the electronic device meets the interpolation condition and can proceed to step S402. It should be noted that this application does not limit the specific first load threshold.

[0053] It should be noted that the GPU load value can be monitored and measured using various tools and software. In some examples, the GPU load value can be monitored and measured using the task manager included in the electronic device's operating system. In other examples, the GPU load value can also be monitored and measured using third-party GPU monitoring software connected to the electronic device. This application does not limit the specific monitoring and measurement methods.

[0054] In some other specific implementations, when the electronic device is in the target game scene and the load value of the central processing unit (CPU) of the electronic device is higher than the second load threshold, step S402 can be executed.

[0055] The CPU load value refers to the sum of the number of processes the CPU is processing and waiting to be processed over a period of time. It can also be understood as the CPU's workload or level of busyness. The CPU load value depends on a number of factors, including the number of applications running on the electronic device, the number of open programs, and the hardware configuration of the electronic device.

[0056] In some examples, the second load threshold may be 80%, 85%, or the like. That is, if the CPU load value is higher than 80% or 85%, the electronic device meets the interpolation condition and can execute step S402. It should be noted that this application does not limit the specific second load threshold.

[0057] In some other specific implementations, when the electronic device is in the target game scene and the real-time game frame rate value (Frames Per Second, FPS) of the electronic device is lower than the frame rate threshold, step S402 can be executed.

[0058] In some examples, the frame rate threshold may be 40 FPS (frames per second), 60 FPS, etc. In other words, if the real-time game frame rate of the electronic device is lower than 40 FPS or 60 FPS, it means that the electronic device meets the interpolation condition and step S402 can be executed. It should be noted that this application does not limit the specific frame rate threshold.

[0059] It should be understood that the above description uses one of the three conditions for interpolation: the GPU load value is greater than the first load threshold, the CPU load value is greater than the second load threshold, and the real-time game frame rate value is less than the frame rate threshold as an example. In actual applications, the interpolation condition can also be a variety of the above three conditions, and this application does not limit this.

[0060] S402: Identify a main rendering thread and main context data among multiple threads of a target game.

[0061] To improve the performance and responsiveness of the target game, multiple threads are used to handle different tasks. For example, the logic thread is primarily responsible for handling the core game logic, including player input, game state updates, and artificial intelligence calculations. The logic thread typically works closely with the main thread to ensure the game's state and behavior meet expectations. The audio thread handles audio playback and sound effect processing. The main rendering thread is primarily responsible for rendering the game scene, such as drawing the game scene, characters, and special effects.

[0062] In games, the main rendering thread is responsible for converting the 3D model of the game scene into a 2D image and displaying it on the electronic device's display. In some examples, the main rendering thread typically uses a graphics library or engine to render the game scene, such as OpenGL, DirectX, Unity, Unreal Engine, Vulkan, Metal, and others. These libraries and engines provide various tools and functions that enable the main rendering thread to efficiently complete rendering tasks. By using a separate main rendering thread, computationally intensive rendering tasks can be separated from the main thread, reducing the burden on the main thread and improving the game's rendering performance.

[0063] The game's main context data (Context) refers to the data used by the OpenGL (Open Graphics Library) state machine to record all the information and status required for rendering during rendering. For example, it records the color data currently used for drawing in the game, whether there is data for lighting calculations, and data on enabled light sources. It should be noted that the game's context data is a type of frame rendering data. For example, the context data can be one or more of the following: Frame Buffer Object (FBO), Vertex Array Object (VAO), Vertex Buffer Object (VBO), and Element Buffer Object (EBO).

[0064] In a game, there may be multiple threads and multiple contexts. Therefore, the main rendering thread and main context can be identified among these multiple threads and multiple contexts by identifying the number of image drawing instructions. This is because the main rendering thread and main context usually call the most image drawing instructions.

[0065] For ease of understanding, the following embodiments will be described using OpenGL as a rendering engine. OpenGL's graphics drawing instructions include glDrawArrays (used to extract data from array data to render basic primitives), glDrawElements (used to draw three-dimensional models), and glDrawElementsInstanced (used to use indexes to specify vertex data and render multiple objects in one call). Therefore, by intercepting all graphics drawing instructions called in the game and counting the number of graphics drawing instructions called by each thread of the game, the thread with the largest number of graphics drawing instructions is the main rendering thread, and the corresponding context data is the main context data.

[0066] By identifying the main rendering thread and main context data in multiple threads of the target game, developers can better understand the usage of threads and contexts in the game process and optimize the game's performance and responsiveness. For example, if the main rendering thread and main context data in multiple threads of the target game are found to be too busy in certain situations, frame insertion processing can be performed in the subsequent step S403.

[0067] S403: Create an interpolation rendering thread.

[0068] By using the thread library or framework provided by the operating system of the electronic device, an interpolation rendering thread can be created. For example, an interpolation rendering thread can be created using Pthreads (POSIX threads) or the thread class in the C++11 standard library. In some specific implementations, an interpolation rendering thread can be created using the following code:

[0069] #include<phread.h>

[0070] intpthread_create(pthread_t*id,pthread_attr_t*attr,void(*fun)(void*),void*arg);

[0071] It should be noted that this application does not limit the specific method of creating an interpolation rendering thread.

[0072] S404: Creating interpolation context data in the interpolation rendering thread.

[0073] In some specific implementations, taking the embedded graphics library EGL as an example, the following code can be used to create interpolation context data in the interpolation rendering thread:

[0074] EGLContext eglCreateContext(EGLDisplaydisplay,

[0075] EGLConfig config,

[0076] EGLContext share_context,

[0077] EGLint const*attrib_list);

[0078] It's important to note that when creating interpolated context data, you must specify that it can share resources with the main context data. That is, the interpolated context data must correspond to the game's main context data. Specifically, you can achieve resource sharing between the main and interpolated context data by specifying share_context as the ID of the main rendering context data in the interpolated context data's corresponding function.

[0079] The purpose of sharing main context data and interpolated context data resources is to improve the performance and efficiency of game rendering. Sharing resources can reduce the overhead of resource copying and repeated loading, thereby improving the responsiveness and smoothness of the game.

[0080] S405: Transfer one or more of the texture data, camera data, and buffer data in the main rendering thread to the interpolation rendering thread.

[0081] It should be noted that texture data, camera data (shader) and buffer data (buffer) belong to frame rendering data. Among them, texture data generally refers to texture resources related to graphics rendering, which are used to provide surface details and appearance texture. Exemplarily, texture data may include the main scene, user interface (UI), depth map, etc. Camera data refers to data that describes the lighting conditions of the object surface during the graphics rendering process, which is used to calculate the lighting model to determine the visual effects such as color, brightness and shadow of the object surface. Buffer data refers to a memory area used to store and process data during the graphics rendering process. In game development and graphics rendering, buffer data is used to store texture data, vertex data, pixel data, etc., so as to facilitate reading and writing during the rendering process.

[0082] 5 , which is a schematic diagram of an interpolation rendering thread provided by an embodiment of the present application, after setting the main context data and interpolation context data shared resources in step S404 , texture data, camera data, and buffer data can be shared.

[0083] In some specific implementations, in order to transfer the game's texture data from the main rendering thread to the interpolation rendering thread so that the interpolation rendering thread can perform interpolation, the following steps need to be performed:

[0084] The first step is to create a shared data structure in the main rendering thread. This shared data structure will be used to store texture data in subsequent steps. In this step, the electronic device uses OpenGL or another graphics library to create the shared data structure. These shared data structures allocate space in the electronic device's memory to store the texture data.

[0085] The second step is to identify texture data. When the game is running, the electronic device needs to be able to identify which resources are the target resources that need to be used as texture data, that is, which texture data needs to be rendered.

[0086] The third step is to copy the texture data into the shared data structure. Once the texture data is recognized, the electronic device will copy the texture data into the previously created shared data structure.

[0087] The fourth step is to pass the shared data structure containing the texture data to the interpolated rendering thread. The populated shared data structure containing the texture data is passed from the main rendering thread to the interpolated rendering thread through a global variable, queue, semaphore, or other inter-thread communication mechanism.

[0088] It is understandable that after the texture data is identified in the second step, the texture data can also be marked with a unique texture ID (i.e., the serial number of the frame rendering data), so that the texture ID can be directly passed to the interpolation rendering thread through global variables, queues, semaphores or other inter-thread communication mechanisms, so that the interpolation rendering thread can easily and conveniently know which texture data to use for rendering, thereby improving subsequent rendering efficiency.

[0089] It should be noted that the steps of passing the game's camera data and buffer data from the main rendering thread to the interpolation rendering thread are similar to the steps of passing the game's texture data from the main rendering thread to the interpolation rendering thread, and will not be repeated here.

[0090] S406: Inserting the interpolated image generated by the interpolated rendering thread according to the frame rendering data before or after the original image generated by the main rendering thread according to the frame rendering data to obtain a game screen with an improved frame rate.

[0091] When the main rendering thread and the interpolated rendering thread simultaneously attempt to access or render the same texture data, camera data, or buffer data, data conflicts, data deadlocks, or other unpredictable contention issues may occur. To avoid contention issues caused by the main rendering thread and the interpolated rendering thread accessing or rendering the same texture data, camera data, or buffer data at the same time, an appropriate synchronization mechanism is required to coordinate operations between the two threads. The data copy method, the dual queue method, and the circular queue method can be used to fundamentally avoid contention issues:

[0092] The data copy method refers to a method in which all texture data, camera data, or buffer data in the main rendering thread is copied to the interpolated rendering thread. The main rendering thread uses the texture data, camera data, or buffer data in the main rendering thread for rendering, and the interpolated rendering thread uses the texture data, camera data, or buffer data in the interpolated rendering thread for rendering, thereby avoiding two threads trying to access or render the same texture data, camera data, or buffer data at the same time.

[0093] The benefit of copying data is that it avoids resource contention between the main rendering thread and the interpolation rendering thread. However, this method requires copying large amounts of texture data, camera data, or buffer data from the main rendering thread to the interpolation rendering thread, resulting in significant memory bandwidth overhead, reduced rendering efficiency and performance, and increased rendering power consumption.

[0094] See Figure 6A, which is a schematic diagram of a dual-queue method provided by an embodiment of the present application. The dual-queue method means that before the game scene starts rendering, the main rendering thread and the interpolation rendering thread create an update queue and a rendering queue respectively. The main rendering thread first renders the current frame (the second frame) and puts all rendering instructions of the current frame (the second frame) into the update queue. At the same time, the interpolation rendering thread is executing all rendering instructions of the previous frame (the first frame).

[0095] After the main rendering thread places all rendering instructions for the current frame (the second frame) into the update queue, it executes the submit command. After the interpolated rendering thread completes all rendering instructions for the previous frame (the first frame), it responds to the submit command and swaps all rendering instructions between the update queue and the render queue.

[0096] After exchanging all rendering instructions, the interpolation rendering thread begins to perform interpolation rendering based on all new rendering instructions. After the interpolation rendering thread completes the interpolation work of the current frame (the second frame), it suspends and waits for the next frame (the third frame). At the same time, the main rendering thread waits for the logic thread to wake up. After the logic thread wakes up, the main rendering thread continues to render the next frame (the third frame) and puts all rendering instructions for the next frame (the third frame) into the update queue. And so on, the above process is repeated to continue multi-threaded rendering.

[0097] The advantage of the dual-queue approach is that the main rendering thread and the interpolated rendering thread can operate their own queues separately, without having to consider resource competition caused by the two threads accessing the same queue data at the same time. However, since it involves the management and synchronization of two queues (update queue and rendering queue), this increases the complexity of resource management, and it is necessary to ensure the correct exchange and synchronization of queues to avoid data inconsistencies or conflicts between threads. In addition, synchronization operations between threads (such as waiting and waking up) may cause additional performance overhead. These synchronization operations need to be handled with caution to avoid unnecessary waiting and performance bottlenecks. Furthermore, since the queues are exchanged only at the end of a frame rendering, there is also the possibility of rendering blocking.

[0098] See Figure 6B, which is a schematic diagram of a circular queue method provided in an embodiment of the present application. A circular queue is a linear data structure. The circular queue method refers to using a fixed-size array and two pointers (Head pointer and Tail pointer) to indicate the start and end positions of the queue in the interpolation rendering thread. Specifically, the Head pointer indicates the starting position of the queue. Whenever a texture data, camera data, or buffer data is added to the queue, the Head pointer moves forward one position; whenever a texture data, camera data, or buffer data is removed from the queue, the Head pointer moves backward one position. The Tail pointer indicates the end position of the queue, that is, the position of the tail of the queue. Whenever a texture data, camera data, or buffer data is added to the queue, the Tail pointer moves backward one position; whenever a texture data, camera data, or buffer data is removed from the queue, the Tail pointer moves forward one position.

[0099] In a circular queue, when the queue is empty, the head pointer and tail pointer both point to the first position in the queue; when the queue is full, the head pointer and tail pointer both point to the last position in the queue. The rendering thread continuously adds texture data, camera data, or buffer data to the circular queue. If the circular queue is full, the rendering thread is blocked. The insertion thread continuously extracts texture data, camera data, or buffer data from the circular queue to perform insertion. If the circular queue is empty, the insertion thread is blocked.

[0100] The main feature of a circular queue is that when the head or tail pointer reaches the end of the array, it automatically returns to the first position, making the circular queue more efficient. Furthermore, unlike a dual queue, a circular queue doesn't need to synchronize the rendering thread with the insertion thread for every frame, resulting in shorter latency.

[0101] In some specific implementations, the interpolation rendering thread can also freely control the frame delivery time (i.e., rendering time). Specifically, to achieve a more stable frame rate, the frame delivery time difference between the real frame rendered by the main rendering thread and the interpolation rendering thread can be first recorded, and then the frame delivery time of the interpolation rendering thread can be adjusted based on this frame delivery time difference.

[0102] It's important to note that while independent interpolation threads can significantly improve frame rates under heavy game loads, not all games are suitable for them. Especially for games with light loads, using independent interpolation threads may result in increased power consumption compared to non-independent interpolation threads. Therefore, switching between different solutions should be considered based on the specific situation.

[0103] In some specific implementations, if the game scene is currently being rendered based on the frame rendering data from the main rendering thread and the interpolation rendering thread, then one or more of the following information can be counted during the interpolation process: the actual game frame rate, the rendering thread execution time, and the interpolation thread execution time. If one or more of the following conditions are met during the game: the real-time game frame rate is higher than a frame rate threshold, the main rendering thread execution time is lower than a first duration threshold, and the interpolation rendering thread execution time is lower than a second duration threshold, then the independent thread interpolation is switched to the non-independent thread interpolation, that is, the game scene is rendered based solely on the frame rendering data from the main rendering thread.

[0104] It should be noted that when independent thread interpolation is switched to non-independent thread interpolation, the interpolation rendering thread can be deleted or retained. The specific decision on whether to delete or retain it can be made based on the actual situation of the game.

[0105] In some specific implementations, some game scenes change significantly, so the interpolation rendering thread cannot be deleted and needs to be retained.

[0106] In other specific implementations, the historical switching frequency or the historical switching time may be recorded online.

[0107] Specifically, if the historical switching frequency is higher than or equal to the frequency threshold, the interpolated rendering thread cannot be deleted and must be retained. The historical switching frequency is the ratio of the number of historical switching times to the target time. The historical switching times are the number of times the game screen was generated from the interpolated rendering thread and the main rendering thread to the main rendering thread. Conversely, if the historical switching frequency is lower than the frequency threshold, the interpolated rendering thread is deleted.

[0108] Specifically, if the difference between the historical switching time and the current time is less than or equal to the third duration threshold, the interpolation rendering thread cannot be deleted and must be retained. The historical switching time is the last time the game screen was generated by the interpolation rendering thread and the main rendering thread. Conversely, if the difference between the historical switching time and the current time is greater than the third duration threshold, the interpolation rendering thread is deleted.

[0109] See Figure 7, which is a schematic diagram of single-frame interpolation through an interpolation rendering thread provided in an embodiment of the present application. See Figure 8, which is a schematic diagram of overcoming multiple-frame interpolation blockage through an interpolation rendering thread provided in an embodiment of the present application. In the prior art, if multi-threaded rendering technology is used for single-frame interpolation or multiple-frame interpolation, the rendering thread simultaneously performs the rendering of the real frame and the rendering of the interpolation frame, resulting in too long rendering time for the game scene, thereby delaying the normal operation of the logic thread and causing the problem of blockage in the rendering of the game scene. In the present application, by creating an independent interpolation rendering thread, the interpolation rendering task originally performed in the main rendering thread can be separated out, reducing the burden on the main rendering thread and significantly improving the rendering speed and efficiency, thereby solving the problem of insufficient frame rate due to heavy game load, poor mobile phone performance, etc., and providing a smoother gaming experience.

[0110] In summary, the present application discloses a frame rate improvement method. When an electronic device needs to improve the frame rate of a game screen, an independent interpolation rendering thread can be created. Subsequently, the interpolation image generated by the interpolation rendering thread based on the frame rendering data is inserted before or after the original image generated by the main rendering thread based on the frame rendering data, thereby forming a game screen with an increased frame rate. In this way, the created interpolation rendering thread can share part of the rendering tasks of the main rendering thread, so that the main rendering thread can have more time to process complex scenes and real-time interactions, etc., reducing the jamming and delay caused by the overload of the main rendering thread. Furthermore, when the main thread is blocked, if the interpolation rendering thread is not blocked, then the interpolation rendering thread can take over more rendering tasks. When the main thread returns to normal, the interpolation rendering thread has completed part or even all of the rendering work, then the main rendering thread can quickly catch up with the rendering progress and merge the rendered original image into the interpolation image rendered by the interpolation rendering thread, thereby maintaining the consistency and smoothness of the overall rendering, so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0111] Referring to FIG9 , which is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device can perform the above-mentioned frame rate improvement method. The electronic device 100 can be a mobile phone powered by multiple batteries, a laptop computer, a wearable terminal device (such as a smart watch), a tablet computer, an augmented reality (AR) device, a virtual reality (VR) device, and an in-vehicle device.

[0112] The electronic device 100 may include a processor 121, an antenna 1, an antenna 2, a mobile communication module 122, a wireless communication module 123, and the like. It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than illustrated, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0113] The processor 121 may include one or more processing units, for example: the processor 121 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. For example, in the present application, when the processor 121 determines that the frame rate of the game screen needs to be improved, it creates an interpolation rendering thread; generates an interpolation image in the interpolation rendering thread according to the frame rendering data; and inserts the interpolation image before or after the original image generated by the main rendering thread according to the frame rendering data. Therefore, when the frame rate of the game screen needs to be improved, the electronic device will create an interpolated rendering thread, which can share part of the rendering tasks of the main rendering thread, so that the main rendering thread can have more time to process complex scenes and real-time interactions, etc., reducing the jamming and delay caused by overload of the main rendering thread, so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0114] Processor 121 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 121 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 121. If processor 121 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 121's latency, and thus improves system efficiency.

[0115] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 122, the wireless communication module 123, the modem processor and the baseband processor.

[0116] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0117] The mobile communication module 122 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 122 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 122 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 122 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 122 can be set in the processor 121. In some embodiments, at least some of the functional modules of the mobile communication module 122 can be set in the same device as at least some of the modules of the processor 121.

[0118] The wireless communication module 123 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 123 can be one or more devices that integrate at least one communication processing module. The wireless communication module 123 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 121. The wireless communication module 123 can also receive the signal to be sent from the processor 121, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0119] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed, the various functions or steps performed by the electronic device 100 in the above method embodiment are implemented.

[0120] Another embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer or processor, causes the computer or processor to execute one or more steps in any of the above methods.

[0121] Referring to FIG10 , which is a schematic diagram of a frame rate improvement device provided in an embodiment of the present application, the frame rate improvement device 1000 includes: a creation module 1010 , a generation module 1020 , and a frame insertion module 1030 ;

[0122] A creation module 1010 is configured to create an interpolation rendering thread when it is determined that the frame rate of the game screen needs to be increased;

[0123] A generating module 1020 is configured to generate an interpolated frame image in an interpolated frame rendering thread according to the frame rendering data;

[0124] The frame insertion module 1030 is configured to insert an interpolated frame image before or after the original image generated by the main rendering thread according to the frame rendering data.

[0125] In summary, the present application discloses a frame rate improvement device, in which the created interpolation rendering thread can share part of the rendering tasks of the main rendering thread, so that the main rendering thread can have more time to process complex scenes and real-time interactions, etc., reducing the jamming and delay caused by the overload of the main rendering thread. Furthermore, when the main thread is blocked, if the interpolation rendering thread is not blocked, then the interpolation rendering thread can take over more rendering tasks. When the main thread returns to normal, the interpolation rendering thread has completed part or even all of the rendering work, then the main rendering thread can quickly catch up with the rendering progress, and merge the rendered original image into the interpolation image rendered by the interpolation rendering thread, so as to maintain the consistency and smoothness of the overall rendering, so that the rendering of the game screen can maintain a stable target frame rate, improve the rendering effect of the game screen, and thus enhance the user's gaming experience.

[0126] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] In the several embodiments provided in this embodiment, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0129] In addition, the functional units in each embodiment of the present invention may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment. The aforementioned storage medium includes: various media that can store program code, such as flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

[0131] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A frame rate improvement method, characterized in that: The method comprises: When it is determined that the frame rate of the game screen needs to be increased, an interpolation rendering thread is created; Generating an interpolated frame image in the interpolated frame rendering thread according to the frame rendering data; The interpolated frame image is inserted before or after the original image generated by the main rendering thread according to the frame rendering data.

2. The method according to claim 1, characterized in that When it is determined that the frame rate of the game screen needs to be increased, an interpolation rendering thread is created, including: When an electronic device is in a game scene and the electronic device meets the interpolation conditions, an interpolation rendering thread is created. The interpolation conditions include one or more of the following: the image processor load value of the electronic device is higher than a first load threshold, the central processing unit load value of the electronic device is higher than a second load threshold, and the real-time game frame rate value of the electronic device is lower than a frame rate threshold.

3. The method according to claim 1, characterized in that The frame rendering data in the interpolation frame rendering thread is transferred from the main rendering thread to the interpolation frame rendering thread, and the frame rendering data includes one or more of texture data and camera data.

4. The method according to claim 1, wherein The frame rendering data in the interpolation frame rendering thread is created by the interpolation frame rendering thread, and the frame rendering data includes context data.

5. The method according to claim 3, characterized in that The method for the interpolation frame rendering thread to receive the frame rendering data includes: After the frame rendering data in the main rendering thread is transferred to the circular queue of the interpolation frame rendering thread, the interpolation frame rendering thread receives the frame rendering data according to the order of the frame rendering data in the circular queue.

6. The method according to claim 3, characterized in that The method for the interpolation frame rendering thread to receive the frame rendering data includes: After the sequence number of the frame rendering data in the main rendering thread is copied to the shared data structure of the main rendering thread, the interpolated frame rendering thread receives the frame rendering data by matching the sequence number of the frame rendering data in the shared data structure.

7. The method according to claim 1, characterized in that After inserting the interpolated frame image generated by the interpolated frame rendering thread according to the frame rendering data before or after the original image generated by the main rendering thread according to the frame rendering data, the method further includes: When it is determined that the frame rate of the game screen does not need to be increased, an original image is generated according to the frame rendering data in the main rendering thread.

8. The method according to claim 7, characterized in that When it is determined that the frame rate of the game screen does not need to be increased, generating an original image according to the frame rendering data in the main rendering thread includes: When the electronic device meets the condition for stopping interpolation, an original image is generated according to the frame rendering data in the main rendering thread. The condition for stopping interpolation includes one or more of the following: a real-time game frame rate value of the electronic device is higher than a frame rate threshold, an execution duration of the main rendering thread is lower than a first duration threshold, and an execution duration of the interpolation rendering thread is lower than a second duration threshold.

9. The method according to claim 7, characterized in that After generating the original image according to the frame rendering data in the main rendering thread, the method further includes: When the thread deletion condition is met, the interpolation rendering thread is deleted.

10. The method according to claim 9, characterized in that When the thread deletion condition is met, deleting the interpolation rendering thread includes: Obtaining a historical switching frequency, where the historical switching frequency is a ratio of the number of historical switching times to a target time, where the historical switching times is the number of times the game screen is generated from the interpolation rendering thread and the main rendering thread to the number of times the game screen is generated from the main rendering thread; If the historical switching frequency is lower than the frequency threshold, the interpolation rendering thread is deleted.

11. The method according to claim 9, characterized in that When the thread deletion condition is met, deleting the interpolation rendering thread includes: Obtaining a historical switching time, where the historical switching time is the time when a game screen was last generated according to the interpolation rendering thread and the main rendering thread; If the time difference between the historical switching time and the current time is greater than a third duration threshold, the interpolation rendering thread is deleted.

12. An electronic device, characterized in that: including processor and memory; The memory is used to store computer-executable instructions; The processor is configured to execute the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program or instructions, and when the computer program or instructions are executed, the method according to any one of claims 1 to 11 is implemented.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images