Cloud game rendering optimization method applied to Android 12 device

Abstract

Embodiments of the present application provide a cloud game rendering optimization method, device, equipment and computer readable storage medium applied to an Android 12 device. The method comprises: cyclically calling an index of to-be-rendered data of an output buffer from a system cache, and putting the index into a cache queue; and reading a head element in the cache queue to perform rendering according to a refresh timing of the Android 12 device. In this way, the rendering frame number requirement of the Android 12 device in a weak network and non-weak network scenario can be met.

Description

Technical Field

[0001] Embodiments of this application relate to the field of rendering optimization, and more particularly to cloud gaming rendering optimization methods, apparatus, devices, and computer-readable storage media applied to Android 12 devices. Background Technology

[0002] In cloud gaming scenarios, when using Android 12 devices, the uneven intervals of video stream data delivery cause the system to actively drop frames after they are placed in the system rendering queue (the number of rendered frames fluctuates around 50+ frames), resulting in the number of rendered frames not meeting the current business requirements.

[0003] In summary, how to ensure that Android 12 devices meet the rendering frame requirements in cloud gaming scenarios is a problem that urgently needs to be solved. Summary of the Invention

[0004] According to an embodiment of this application, a cloud gaming rendering optimization scheme for Android 12 devices is provided.

[0005] In a first aspect of this application, a cloud gaming rendering optimization method for Android 12 devices is provided. The method includes:

[0006] Retrieve the index of the data to be rendered from the output buffer from the system cache in a loop, and put the index into the cache queue;

[0007] Based on the refresh timing of the Android 12 device, the first element in the cache queue is read and rendered.

[0008] Furthermore, the index for retrieving the output buffer data to be rendered from the system cache in a loop includes:

[0009] Call the cache queue read function to retrieve the index of the data to be rendered from the output buffer in a loop from the system cache.

[0010] Furthermore, placing the index into the cache queue includes:

[0011] If the length of the current cache queue exceeds the threshold, the data at the head of the current cache queue is released, and the index is placed into the cache queue.

[0012] Furthermore, the step of reading the head element of the cache queue and performing rendering based on the refresh timing of the Android 12 device includes:

[0013] The callback trigger time interval of the Android 12 device is determined based on the refresh rate of the Android 12 device;

[0014] Based on the callback trigger time interval, read the first element in the cache queue and call the rendering function to perform rendering.

[0015] Furthermore, the buffer queue read function includes MediaCodec.dequeueOutputBuffer().

[0016] Furthermore, the rendering function includes MediaCodec.releaseOutputBuffer().

[0017] In a second aspect of this application, a cloud gaming rendering optimization device for Android 12 devices is provided. The device includes:

[0018] The retrieval module is used to cyclically retrieve the index of the data to be rendered in the output buffer from the system cache and put the index into the cache queue;

[0019] The rendering module is used to read the first element of the cache queue and perform rendering based on the refresh timing of the Android 12 device.

[0020] In a third aspect of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0021] In a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to the first aspect of this application.

[0022] The cloud gaming rendering optimization method for Android 12 devices provided in this application embodiment retrieves the index of the data to be rendered from the output buffer in a loop from the system cache and puts the index into a cache queue; according to the refresh timing of the Android 12 device, the first element in the cache queue is read and rendering is performed, which enables the Android 12 device to meet the rendering frame rate requirements in both weak and strong network scenarios.

[0023] It should be understood that the description in the Summary Section is not intended to limit the key or essential features of the embodiments of this application, nor is it intended to restrict the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0024] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0025] Figure 1 A flowchart illustrating a cloud gaming rendering optimization method for Android 12 devices according to an embodiment of this application;

[0026] Figure 2 A flowchart for retrieving the index of data to be rendered according to an embodiment of this application;

[0027] Figure 3 This is a flowchart illustrating the process of placing an index into a cache queue according to an embodiment of this application;

[0028] Figure 4 This is a rendering flowchart according to an embodiment of this application;

[0029] Figure 5 A block diagram of a cloud gaming rendering optimization apparatus for an Android 12 device according to an embodiment of this application;

[0030] Figure 6 This is a schematic diagram of the structure of a terminal device or server suitable for implementing the embodiments of this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0032] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0033] Noun Analysis:

[0034] Choreographer: A mechanism added by Google's Project Butter in Android 4.1 to work with the system's VSYNC interrupt signal. Its main purpose is to receive the system's VSYNC signal and manage the timing of application input, animation, and drawing tasks.

[0035] Figure 1 A flowchart illustrating a cloud gaming rendering optimization method for Android 12 devices according to an embodiment of the present disclosure is shown. The method includes:

[0036] S110: Retrieve the index of the data to be rendered from the output buffer from the system cache in a loop, and put the index into the cache queue.

[0037] In some embodiments, such as Figure 2 As shown, the cache queue read function is called repeatedly to retrieve the index of the data to be rendered from the output buffer in the system cache.

[0038] Preferably, the buffer queue reading function is MediaCodec.dequeueOutputBuffer().

[0039] Specifically, MediaCodec.dequeueOutputBuffer() is called repeatedly to retrieve decoded video frames from the system cache and obtain the return value (int) index. When index is greater than or equal to 0, it indicates that the data (video frame) marked at the current index has been decoded and can be rendered.

[0040] The video frames and indexes are in one-to-one correspondence, that is, one video frame corresponds to one index.

[0041] In some embodiments, such as Figure 3 As shown, the length of the cache queue is determined. If the length of the cache queue is greater than or equal to the threshold, the first element in the cache queue is read, and the MediaCodec.releaseOutputBuffer(first element, false) method is executed to release the first element (delete it without rendering, a frame dropping strategy). The newly acquired index is then placed into the cache queue. If the length of the queue is less than the threshold, the newly acquired index is directly placed into the cache queue. The threshold is set according to the application scenario, for example, 5.

[0042] Wherein, the index placed in the cache queue is greater than or equal to 0.

[0043] S120: Based on the refresh timing of the Android 12 device, read the first element of the cache queue and perform rendering.

[0044] In some embodiments, rendering data is inserted based on the Choreographer's doFrame callback method, according to the device's refresh timing.

[0045] Specifically, based on the refresh rate of the Android 12 device, the callback triggering time interval of the Android 12 device is determined. If the time difference between two callbacks is greater than the current callback triggering time interval of the Android 12 device, the head element of the cache queue is read, and the rendering function is called for rendering. The rendering function is preferably MediaCodec.releaseOutputBuffer().

[0046] For example, Android 12 devices have a refresh rate of 120 frames per second. Figure 4 The doFrame method is triggered every 8.3 milliseconds. To ensure smooth rendering of 60 frames of data and reduce the generation of jank data, the following judgment is made: if the time interval between two doFrame method triggers is greater than 12 milliseconds, the first element in the buffer queue is read and the MediaCodec.releaseOutputBuffer(first element, true) method is executed, and the device performs rendering (equivalent to rendering once every two callbacks); if it is less than 12 milliseconds, the Choreographer's doFrame callback is executed.

[0047] According to the embodiments of this disclosure, the following technical effects are achieved:

[0048] The method disclosed herein allows for the placement of rendered video frames based on the device's refresh timing, preventing the placement of multiple video frames within a single refresh cycle and the rendering of only the last frame, which would result in insufficient frame rates. This resolves the stuttering issue on Android 12 devices in cloud gaming scenarios. In other words, it meets the rendering frame rate requirements of Android 12 devices in both weak and strong network environments.

[0049] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0050] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.

[0051] Figure 5 A block diagram of a cloud gaming rendering optimization apparatus 500 for an Android 12 device according to an embodiment of this application is shown below. Figure 5 As shown, the device 500 includes:

[0052] The retrieval module 510 is used to cyclically retrieve the index of the data to be rendered in the output buffer from the system cache and put the index into the cache queue;

[0053] The rendering module 520 is used to read the first element of the cache queue and perform rendering according to the refresh timing of the Android 12 device.

[0054] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0055] Figure 6 A schematic diagram of a terminal device or server suitable for implementing embodiments of this application is shown.

[0056] like Figure 6 As shown, the terminal device or server includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the terminal device or server. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0057] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0058] Specifically, according to embodiments of this application, the above method flow steps can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined in the system of this application.

[0059] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0060] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0061] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor. The names of these units or modules do not, in certain circumstances, constitute a limitation on the unit or module itself.

[0062] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the methods described in this application.

[0063] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions claimed in this application.

Claims

1. A cloud gaming rendering optimization method applied to Android 12 devices, characterized in that, include: Retrieve the index of the data to be rendered from the output buffer from the system cache in a loop, and put the index into the cache queue; The indexes for retrieving the output buffer data to be rendered from the system cache in a loop include: Call the cache queue read function to retrieve the index of the data to be rendered from the output buffer from the system cache in a loop; The step of putting the index into the cache queue includes: If the length of the current cache queue exceeds the threshold, release the data at the head of the current cache queue and put the index into the cache queue; Based on the refresh timing of the Android 12 device, read the first element in the cache queue and perform rendering; The step of reading the head element of the cache queue and performing rendering based on the refresh timing of the Android 12 device includes: The callback trigger time interval of the Android 12 device is determined based on the refresh rate of the Android 12 device; If the time difference between two callbacks is greater than the callback triggering time interval of the current Android 12 device, then the first element in the cache queue is read, and the rendering function is called to perform rendering; If it is less than, then the Choreographer's doFrame callback is executed.

2. The method according to claim 1, characterized in that, The buffer queue read function includes MediaCodec.dequeueOutputBuffer().

3. The method according to claim 2, characterized in that, The rendering function includes MediaCodec.releaseOutputBuffer().

4. A cloud gaming rendering optimization device for Android 12 devices, characterized in that, include: The retrieval module is used to cyclically retrieve the index of the data to be rendered in the output buffer from the system cache and put the index into the cache queue; The process of retrieving the index of the data to be rendered from the output buffer from the system cache includes: calling the cache queue read function to retrieve the index of the data to be rendered from the output buffer from the system cache in a loop; The step of putting the index into the cache queue includes: if the length of the current cache queue exceeds the threshold, then release the head data in the current cache queue and put the index into the cache queue; The rendering module is used to read the first element of the cache queue and perform rendering based on the refresh timing of the Android 12 device. The step of reading the head element of the cache queue and performing rendering based on the refresh timing of the Android 12 device includes: determining the callback triggering time interval of the Android 12 device based on the refresh rate of the Android 12 device; if the time difference between two callbacks is greater than the current callback triggering time interval of the Android 12 device, then reading the head element of the cache queue and calling the rendering function for rendering; if it is less than, then executing the doFrame callback of Choreographer.

5. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 3.

Images