Frame rate control method, device and equipment

By real-time cache of the graphical user interface on the terminal device and calculating the instruction sending cycle according to the frame rate, the problems of complexity and high cost of frame rate control in small terminal control scenarios are solved, and stable frame rate transmission and user service quality are guaranteed.

CN119071278AActive Publication Date: 2024-12-03CHINA UNITED NETWORK COMM GRP CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310632656.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03
Estimated Expiration
2043-05-30

Smart Images

  • Figure CN119071278A_ABST
    Figure CN119071278A_ABST
Patent Text Reader

Abstract

The invention provides a frame rate control method, device and equipment, and the method comprises the steps: responding to the operation of a user on an application program, caching a graphic user interface snapshot of the application program in real time, obtaining an operation instruction, determining the type of the operation instruction, and if the operation instruction is a plurality of continuous operations, controlling the frame rate. If yes, whether the operation duration of the operation instruction exceeds the instruction sending period or not is determined, and if not, after target time delay is carried out, the latest frame of snapshot in the local cache is taken out to serve as response information of the operation instruction to be sent to the server. Therefore, stable transmission of the frame rate is realized, front-end real-time calculation of the terminal equipment is realized, the pressure of the server is relieved, and the operation cost is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a frame rate control method, device and equipment. Background Art

[0002] With the continuous development of the Internet, people's lives are increasingly inseparable from smart electronic products. Smart devices such as mobile phones, televisions, tablets, etc. used in people's daily lives all respond to user operations and continuously send frames according to the frame rate agreed upon by the background server to achieve data transmission and control, and finally display the picture on the screen. As more and more people use smart devices, the background server freezes and freezes, making people's experience worse.

[0003] In the prior art, the frame rate control method is usually to reduce the frame rate of the frozen image through real-time negotiation between the front end and the back end, or through real-time calculation by the terminal or based on hardware cache.

[0004] However, most of the smart devices people use in life are small terminal devices. Under the premise of stable frame rate transmission, the existing technology has high maintenance costs, consumes manpower, and has complex engineering implementation, which is not applicable in small terminal control scenarios. Summary of the invention

[0005] The present application provides a frame rate control method, device and equipment to solve the problem that in small terminal control scenarios in daily life, the existing technology has high maintenance cost, labor consumption and complex engineering implementation under the premise of ensuring a stable frame rate.

[0006] In a first aspect, the present application provides a frame rate control method, applied to a terminal device, comprising:

[0007] In response to a user's operation on an application, caching a snapshot of a graphical user interface of the application in real time;

[0008] After the user operation is completed, an operation instruction is obtained, and a type of the operation instruction is determined to be a single operation or a plurality of operations;

[0009] If the type of the operation instruction is a plurality of continuous operation instructions, determining whether the operation duration of the operation instruction exceeds an instruction sending cycle, wherein the instruction sending cycle is calculated in advance according to a frame rate set by the application;

[0010] If the operation duration of the operation instruction does not exceed the instruction sending cycle, the latest frame snapshot in the local cache is taken out and sent to the server as the response information of the operation instruction after the target delay, wherein the target delay is the difference between the instruction sending cycle and the operation duration of the operation instruction.

[0011] In combination with the first aspect, in some embodiments, the method further includes:

[0012] If the operation duration of the operation instruction exceeds the instruction sending cycle, the latest frame snapshot in the local cache is taken out and sent to the server as the response information of the operation instruction.

[0013] In combination with the first aspect, in some embodiments, the method further includes:

[0014] If the type of the instruction is a single operation, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0015] In combination with the first aspect, in some embodiments, the method further includes:

[0016] If at least one of network fluctuation, network frame loss or network resource exhaustion is detected, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0017] In combination with the first aspect, in some embodiments, the method further includes:

[0018] After sending the response information of the operation instruction to the server, a timer lock with the instruction sending cycle as the timing duration is set and locked, and the timer lock is unlocked after the timing duration reaches the next one after the instruction sending cycle; during the locking period of the timer lock, no response information of the operation instruction is sent to the server.

[0019] In combination with the first aspect, in some embodiments, the method further includes:

[0020] The instruction sending period for the application to send instructions to the server is calculated according to the frame rate set by the application.

[0021] In a second aspect, the present application provides a frame rate control device, comprising:

[0022] A response cache module, used for caching a snapshot of the graphical user interface of the application in real time in response to a user's operation on the application;

[0023] An instruction determination module, used for obtaining an operation instruction after the user operation is completed, and determining whether the type of the operation instruction is a single operation or a series of operations;

[0024] a duration determination module, for determining, if the type of the operation instruction is a plurality of consecutive operation instructions, whether the operation duration of the operation instruction exceeds an instruction sending cycle, wherein the instruction sending cycle is calculated in advance according to a frame rate set by the application program;

[0025] The first sending module is used to take out the latest frame snapshot in the local cache and send it to the server as the response information of the operation instruction after the target delay if the operation duration of the operation instruction does not exceed the instruction sending cycle, wherein the target delay is the difference between the instruction sending cycle and the operation duration of the operation instruction.

[0026] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0027] The second sending module is used to take out the latest frame snapshot in the local cache and send it to the server as the response information of the operation instruction if the operation time of the operation instruction exceeds the instruction sending cycle.

[0028] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0029] The third sending module is used for taking out the latest frame snapshot in the local cache and sending it to the server as the response information of the operation instruction if the type of the instruction is a single operation.

[0030] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0031] The fourth sending module is used to take out the latest frame snapshot in the local cache and send it to the server as response information of the operation instruction if at least one of network fluctuation, network frame loss or network resource exhaustion is detected.

[0032] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0033] The timing setting module is used to set a timer lock with the instruction sending cycle as the timing duration and lock it after sending the response information of the operation instruction to the server, and unlock the timer lock after the timing duration reaches the next one after the instruction sending cycle; the response information of the operation instruction is not sent to the server during the locking period of the timer lock.

[0034] In conjunction with the second aspect, in some embodiments, the apparatus further includes:

[0035] The cycle calculation module is used to calculate the instruction sending cycle of the application sending instructions to the server according to the frame rate set by the application.

[0036] In a third aspect, the present application provides a terminal device, including: a memory, a processor,

[0037] The memory stores computer-executable instructions;

[0038] The processor executes the computer-executable instructions stored in the memory to implement the method described in the first aspect.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the frame rate control method described in the first aspect.

[0040] The frame rate control method, device and equipment provided in this application. Using the front-end engineering method, through the front-end real-time calculation and software caching, it is encapsulated into a method, which can stabilize the frame rate within a certain range, increase time calibration, ensure stable transmission of the frame rate through a timer lock, establish a waiting queue at the front end, and avoid losing key information by caching key frames. Therefore, on the basis of stable transmission of the frame rate, the user service quality is guaranteed early while reducing the operating cost, simplifying the implementation of the project, and reducing manpower and learning time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] Figure 1 An application scenario diagram of the frame rate control method provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of a first embodiment of a method for controlling a frame rate provided in an embodiment of the present application;

[0044] Figure 3 A flow chart of a second embodiment of a method for controlling a frame rate provided in an embodiment of the present application;

[0045] Figure 4 A flowchart of a specific example of a frame rate control method provided in an embodiment of the present application;

[0046] Figure 5 A structural diagram of a first embodiment of a frame rate control device provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of a second embodiment of a frame rate control device provided in an embodiment of the present application;

[0048] Figure 7A schematic diagram of the structure of a terminal device provided in an embodiment of the present application.

[0049] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0050] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0051] In today's Internet era, in the fields of games, communications, control, etc., the transmission of network data is crucial, but different scenarios have different requirements for network data transmission. In small terminal control scenarios, such as game controllers, mobile phone screen touches, VR devices, etc., each touch operation may generate thousands of control messages. Due to the processing power of the server, the front end cannot send them to the back end without restraint, otherwise the server will have problems such as freezes and pseudo-death, affecting the user experience. In the video field, because the naked eye will think that static pictures with more than 24 frames per second are continuous dynamic videos, the transmission video is usually 24-30 frames per second, and the transmission control message has a lower frame rate requirement, usually 5-10 frames per second can meet the control requirements. In the prior art, the control of the frame rate is usually negotiated in real time between the front end and the back end, or through real-time calculation by the terminal or based on hardware cache, the frozen picture is reduced in frame rate. The above methods can all meet the stable transmission of the frame rate, but in the small terminal control scenario, the development and cost of the above methods are relatively high, and the hardware requirements are also high, which increases the cost while consuming manpower and learning time, and the engineering implementation is complex.

[0052] In response to the above problems, the present application provides a frame rate control method, device and equipment, which, on the basis of stable frame rate transmission, relieves the working pressure of the background server, and ensures the user's service quality while reducing operating costs. Specifically, the traditional frame rate control method is real-time negotiation between the front end and the back end, or real-time calculation by the terminal or based on hardware cache, to reduce the frame of the frozen picture. During the research process, the inventor found that the above traditional method can meet the stable transmission of the frame rate for small terminal control scenarios, but the development and cost are relatively high, and the hardware requirements are also high. While increasing the cost, it also consumes manpower and learning time, and the engineering implementation is complicated. Taking these problems into consideration, the inventor studied whether it is possible to realize real-time calculation through the front end of the terminal device and realize the stability of the frame rate through a timer lock, thereby reducing the pressure on the background server. Based on this, the technical solution of the present application is proposed.

[0053] Figure 1 This is an application scenario diagram of the frame rate control method provided in an embodiment of the present application. The frame rate control method provided in the present application is mainly used in small terminal control scenarios, such as game controllers, mobile phones, VR devices, etc. The small terminal control scenarios include at least users and terminal devices. The terminal devices are configured with controls for user operation, a background server and a front end. The controls for user operation may be a mobile phone screen, a game controller, and a joystick in a VR device. There is no specific limitation on the type of terminal device, the type of user, and the controls configured for the terminal device. This solution does not limit the specific form of the above devices.

[0054] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0055] Figure 2 The flowchart of the first embodiment of the frame rate control method provided in the present application is as follows: Figure 2 As shown, the frame rate control method provided in this embodiment is applied to a terminal device, and specifically includes the following steps:

[0056] S101: In response to a user's operation on an application, a snapshot of a graphical user interface of the application is cached in real time.

[0057] In this step, in order to avoid frame loss, when the user operates the application of the terminal device, the terminal device will cache the snapshot of the graphical user interface of the application and update it in real time according to the user operation. Specifically, when the user operates the application, the terminal device activates the control.

[0058] S102: After the user operation is completed, an operation instruction is obtained, and the type of the operation instruction is determined to be a single operation or a plurality of operations.

[0059] In this step, in order to update the cache in real time, after the user operation is completed, the operation instructions input by the user are obtained in real time and converted into control instructions. In order to better control the frame rate later, it is determined whether the type of operation instruction is a single operation or multiple operations read continuously.

[0060] Specifically, obtain user operation instructions, capture the user's touch action when the user starts the operation, capture the movement distance after the touch when the user moves the operation, and finally capture the user's end of the touch action when the user ends the operation. If the user only touches or touches the application control momentarily, it is determined as a single operation. If the user swipes or continuously touches the application control, it is determined as multiple consecutive operations.

[0061] S103: If the type of the operation instruction is a plurality of continuous operation instructions, determine whether the operation duration of the operation instruction exceeds the instruction sending cycle.

[0062] In this step, if it is determined according to the operations in the above steps that the user is swiping or continuously touching the application control, then it is determined that the user's operation instructions at this time are multiple continuous operation instructions. In order to ensure stable frame rate transmission, after determining that the operation instruction type is multiple continuous operation instructions, it is determined whether the operation duration of the operation instruction exceeds the instruction sending cycle.

[0063] Specifically, the instruction sending cycle is calculated in advance based on the frame rate set by the application. The specific calculation formula is:

[0064]

[0065] Here, 1000 is 1000 milliseconds, and the frame rate is a preset frame rate.

[0066] S104: If the operation duration of the operation instruction does not exceed the instruction sending cycle, the latest frame snapshot in the local cache is taken out after the target delay and sent to the server as response information of the operation instruction.

[0067] In this step, the instruction sending cycle is calculated according to the formula in the above step and compared with the operation duration of the operation instruction. If the operation duration of the operation instruction does not exceed the instruction sending cycle, in order to ensure stable service quality for users, a waiting queue is established at this time and a target delay is set to send the response information of the operation instruction.

[0068] Specifically, if the operation duration of the operation instruction does not exceed the instruction sending cycle, the difference between the instruction sending cycle and the operation duration of the operation instruction is used as the target delay for waiting to be sent, and after the target delay, the latest frame snapshot in the local cache is taken out and sent to the server as the response information of the operation instruction. During the target delay process, the local cache will be updated in real time to avoid frame loss, and the historical cache will be discarded after the target delay, and the latest frame snapshot in the local cache will be taken out.

[0069] The frame rate control method provided in this embodiment caches the snapshot of the graphical user interface of the application in real time in response to the user's operation on the application, obtains the operation instruction and converts it into a control instruction, and then determines whether the type of the operation instruction is a single operation or multiple operations in succession. If the type of the operation instruction is multiple operations in succession, it determines whether the operation duration of the operation instruction exceeds the instruction sending cycle. If it does not exceed the instruction sending cycle, the target delay is performed, and a waiting queue is established to send the response information of the operation instruction. Through the above operations, the front-end calculation of the terminal device is realized, and the waiting queue is established through time calibration to ensure the stability of the frame rate, thereby avoiding frame loss by updating the local cache in real time.

[0070] Figure 3 The flowchart of the second embodiment of the frame rate control method provided in the embodiment of the present application is as follows: Figure 3 As shown, based on the above embodiment, the frame rate control method provided in the embodiment of the present application further includes the following steps:

[0071] S105: Calculating a command sending period for the application to send commands to the server according to the frame rate set by the application.

[0072] In this step, in order to adapt to different scenarios, the frame rate can be preset according to the specific scenario requirements, and the instruction sending cycle for sending instructions to the server is calculated based on the preset frame rate.

[0073] Specifically, the calculation formula for the instruction sending cycle is:

[0074]

[0075] Here, 1000 is 1000 milliseconds, and the frame rate is a preset frame rate.

[0076] S106: If the instruction type is a single operation, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0077] In this step, if the instruction type is determined to be a single operation based on the captured user operation, there is no need to analyze the response information of the operation instruction for the instruction. The latest frame snapshot in the local cache is directly taken out as the response information of the operation instruction and sent to the server.

[0078] S107: If the operation duration of the operation instruction exceeds the instruction sending cycle, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0079] In this step, the type of instruction is determined to be multiple continuous operations based on the captured user operations, and the operation duration of the operation instruction is calculated to exceed the instruction sending cycle. In order to avoid frame loss, the latest frame snapshot in the local cache is taken out as the response information of the operation instruction and sent to the server.

[0080] S108: If at least one of network fluctuation, network frame loss or network resource exhaustion is detected, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0081] In this step, during the user operation process and frame rate transmission process, if any unexpected situation such as network fluctuation, network frame loss or network resource exhaustion is detected, in order to ensure user service quality and avoid frame loss in unexpected situations, the latest frame snapshot in the local cache is taken out and sent to the server as the response information of the operation instruction.

[0082] S109: After sending the response information of the operation instruction to the server, a timer lock with the instruction sending cycle as the timing duration is set and locked, and the timer lock is unlocked after the timing duration reaches the next one after the instruction sending cycle.

[0083] In this step, after the response information of the operation instruction is sent to the server according to the user operation instruction, in order to ensure the stable transmission of the frame rate in real time, a timer lock is added for locking.

[0084] Specifically, the timing duration of the timer lock is the instruction sending cycle. In the above steps, if the type of instruction is a single operation, an unexpected situation, or the type of instruction is multiple continuous operations and the operation duration of the operation instruction exceeds the instruction sending cycle, the timer lock is added after the response information of the operation instruction is sent, and is unlocked after the timing duration is reached. If the type of instruction is multiple continuous operations and the operation duration of the operation instruction does not exceed the instruction sending cycle, the timer lock is added after the target delay is sent. In this case, the timer lock duration is set to the duration of the target delay, and is unlocked after the timing duration is reached.

[0085] The frame rate control method provided in this embodiment sends response information of the operation instruction by determining and analyzing different operation instructions of the user. If the operation instruction is a single operation or encounters unexpected situations such as network frame loss, it is sent immediately. If the operation instruction is multiple operations in a row, a time calibration is set, and the response information of the operation instruction is sent according to the relationship between the duration of the operation instruction and the instruction sending cycle. After the response information is sent, a timer lock is added, thereby realizing real-time calculation at the front end, reducing the pressure on the server, and controlling the frame rate within a certain range. The time calibration and timer lock are added to realize stable transmission of the frame rate. On this basis, frame loss is avoided by establishing a waiting queue to update the local cache in real time.

[0086] The following uses the UE4 joystick controller as a specific example to describe in detail the frame rate control method provided by this application. Figure 4 A flowchart of a specific example of a frame rate control method provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the specific steps include:

[0087] S201: Capture user operation instructions, convert them into control instructions, and calculate the instruction sending cycle.

[0088] In this step, the user moves the joystick control of the UE4 joystick controller to activate the joystick control, obtains operation instructions by capturing the user's movements, and converts the user's operation instructions into software control instructions. In order to adapt to different scenarios, the frame rate required by the scene is pre-set according to the specific scene, and the instruction sending cycle is calculated according to the pre-set frame rate. A cache is established locally, and a snapshot of the joystick controller's graphical user interface is updated in real time in the local cache.

[0089] Specifically, the formula for calculating the instruction sending cycle according to the preset frame rate is:

[0090]

[0091] Here, 1000 is 1000 milliseconds, and the frame rate is a preset frame rate.

[0092] S202: Determine whether the type of the operation instruction is a start-up, shutdown operation, or a move operation.

[0093] In this step, the type of operation instruction is determined based on the user actions captured in the above steps, and the types mainly include single operation and continuous multiple operations.

[0094] Specifically, a single operation includes a start and a close operation, and continuous multiple operations include a move operation. When the user's touch action is captured, the start operation instruction is obtained. When the user's continuous touch and movement for a certain distance are captured, the move operation instruction is obtained. When the user ends the touch action, the close operation instruction is obtained.

[0095] S203: If the type of the instruction is a start or close operation, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0096] In this step, the type of operation instruction is determined according to the above steps. If the instruction type is a startup or shutdown operation, it is a single operation. Therefore, there is no need to analyze the response information of the operation instruction for the instruction. The latest frame snapshot in the local cache is directly taken out and sent to the server as the response information of the operation instruction.

[0097] S204: If the type of the operation instruction is a move operation, then determine whether the operation duration of the move operation instruction exceeds the instruction sending cycle.

[0098] In this step, the type of operation instruction is determined according to the above steps. If the type of operation instruction is a mobile operation, it is a series of operations. In order to reduce server pressure while stabilizing the frame rate, it is necessary to determine whether the operation duration of the mobile operation instruction exceeds the instruction sending cycle.

[0099] S205: If the operation duration of the mobile operation instruction does not exceed the instruction sending cycle, the latest frame snapshot in the local cache is taken out after the target delay and sent to the server as response information of the operation instruction.

[0100] In this step, the type of operation instruction is determined to be a mobile operation according to the above steps, and the operation duration of the mobile operation instruction is calculated and compared with the instruction sending cycle. If the operation duration of the mobile operation instruction does not exceed the instruction sending cycle, in order to ensure stable frame rate transmission, a waiting queue is established, and a target delay is set. After the target delay, the latest frame snapshot in the local cache is taken out as the response information of the operation instruction and sent to the server. The target delay is the difference between the instruction sending cycle and the operation duration of the operation instruction, and during the target delay process, the local cache will be updated in real time, thereby avoiding frame loss, and the historical cache will be discarded after the target delay, and the latest frame snapshot in the local cache will be taken out.

[0101] S206: If the operation duration of the mobile operation instruction exceeds the instruction sending cycle, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0102] In this step, if the operation duration of the mobile operation instruction exceeds the instruction sending cycle according to the above steps, in order to avoid frame loss, the latest frame snapshot in the local cache is taken out and sent to the server as the response information of the operation instruction.

[0103] S207: If an unexpected situation is detected, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

[0104] In this step, network transmission will inevitably encounter unexpected situations. When encountering unexpected situations, in order to ensure service quality and avoid frame loss, the latest frame snapshot in the local cache is taken out as the response information of the operation instruction and sent to the server. The unexpected situations include common network problems such as network fluctuations, network frame loss or network resource exhaustion.

[0105] S208: Set the timer lock.

[0106] In this step, in order to control the frame rate and make the frame rate transmission more stable, after sending the response information of the operation instruction to the server, a timer lock is set to lock it and prohibit sending information to the background. When the timer duration is reached, it is unlocked to send information.

[0107] Specifically, for a single operation of startup or shutdown operation, an unexpected situation, or multiple consecutive operations of a moving operation in the above steps, and the operation duration of the operation instruction exceeds the instruction sending cycle, a timer lock with an instruction sending cycle of the timing duration is added after the response information of the operation instruction is sent, and the lock is unlocked after the timing duration is reached. For multiple consecutive operations of a moving operation and the operation duration of the operation instruction does not exceed the instruction sending cycle, a timer lock is added after the target delay is sent. In this case, the timer lock duration is set to the target delay duration, and the lock is unlocked after the timing duration is reached.

[0108] The frame rate control method provided in this embodiment encapsulates the method into a universal function through front-end real-time calculation and software caching, thereby reducing the pressure on the server. At the same time, it realizes stable transmission of the frame rate through time calibration and timer lock, avoids frame loss, reduces complexity, and reduces operating costs.

[0109] Figure 5 This is a structural diagram of a first embodiment of a frame rate control device provided in an embodiment of the present application, such as Figure 5 As shown, the frame rate control device 300 includes:

[0110] The response cache module 301 is used to cache the snapshot of the graphical user interface of the application in real time in response to the user's operation on the application.

[0111] The instruction determination module 302 is used to obtain the operation instruction after the user operation is completed, and determine the type of the operation instruction is a single operation or a plurality of operations.

[0112] The duration determination module 303 is used to determine whether the operation duration of the operation instruction exceeds the instruction sending period if the type of the operation instruction is a plurality of consecutive operation instructions. The instruction sending period is calculated in advance according to the frame rate set by the application.

[0113] The first sending module 304 is used to take out the latest frame snapshot in the local cache and send it to the server as the response information of the operation instruction after the target delay if the operation duration of the operation instruction does not exceed the instruction sending cycle, wherein the target delay is the difference between the instruction sending cycle and the operation duration of the operation instruction.

[0114] Figure 6 This is a structural diagram of Embodiment 2 of the frame rate control device provided in the embodiment of the present application, such as Figure 6 As shown, the frame rate control device 300 also includes:

[0115] The second sending module 305 is used to take out the latest frame snapshot in the local cache and send it to the server as response information of the operation instruction if the operation duration of the operation instruction exceeds the instruction sending cycle.

[0116] The third sending module 306 is configured to fetch the latest frame snapshot in the local cache and send it to the server as response information of the operation instruction if the instruction type is a single operation.

[0117] The fourth sending module 307 is used to take out the latest frame snapshot in the local cache and send it to the server as response information of the operation instruction if at least one of network fluctuation, network frame loss or network resource exhaustion is detected.

[0118] The timing setting module 308 is used to set a timer lock with the instruction sending cycle as the timing duration and lock it after sending the response information of the operation instruction to the server, and unlock the timer lock after the timing duration reaches the next instruction sending cycle; no response information of the operation instruction is sent to the server during the timer lock period.

[0119] The cycle calculation module 309 is used to calculate the instruction sending cycle of the application program sending instructions to the server according to the frame rate set by the application program.

[0120] The frame rate control device provided in this embodiment is used to execute the frame rate control method in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be described in detail here.

[0121] The present application also provides a terminal device, Figure 7A schematic diagram of the structure of a terminal device provided in an embodiment of the present application, such as Figure 7 As shown, the terminal device 400 includes: a memory 401, a processor 402, and a control 403;

[0122] Memory 401 stores computer-executable instructions.

[0123] The processor 402 executes the computer-implemented instructions stored in the memory to implement the method in any one of the embodiments.

[0124] Control 403 is used for users to operate.

[0125] Optionally, the above-mentioned components of the terminal device 400 may be connected via a system bus.

[0126] The memory 401 may be a separate storage unit, or may be a storage unit integrated in the processor 402. The number of the processor 402 may be one or more.

[0127] Optionally, the terminal device 400 may further include a display, which may be used to display the processing results of the processor and interact with the human-computer. In some embodiments, the display may be a front panel of the terminal device; in other embodiments, the display may be a flexible display screen, which is disposed on a curved surface or a folding surface of the terminal device. The display may even be configured as a display screen of a non-rectangular irregular shape, i.e., a special-shaped screen. The display may be made of materials such as a liquid crystal display (LCD) and an organic light-emitting diode (OLED).

[0128] It should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0129] The system bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0130] All or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0131] The terminal device provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments, and its implementation principle and technical effects are similar and will not be repeated here.

[0132] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method in any one of the embodiments.

[0133] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0134] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0135] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution provided by any of the above method embodiments can be implemented.

[0136] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0137] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0138] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for controlling a frame rate, It is characterized in that Applied to terminal equipment, including: In response to a user's operation on an application, caching a snapshot of a graphical user interface of the application in real time; After the user operation is completed, an operation instruction is obtained, and a type of the operation instruction is determined to be a single operation or a plurality of operations; If the type of the operation instruction is a plurality of continuous operation instructions, determining whether the operation duration of the operation instruction exceeds an instruction sending cycle, wherein the instruction sending cycle is calculated in advance according to a frame rate set by the application; If the operation duration of the operation instruction does not exceed the instruction sending cycle, the latest frame snapshot in the local cache is taken out and sent to the server as the response information of the operation instruction after the target delay, wherein the target delay is the difference between the instruction sending cycle and the operation duration of the operation instruction.

2. The method according to claim 1, It is characterized in that The method further comprises: If the operation duration of the operation instruction exceeds the instruction sending cycle, the latest frame snapshot in the local cache is taken out and sent to the server as the response information of the operation instruction.

3. The method according to claim 2, It is characterized in that The method further comprises: If the type of the instruction is a single operation, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

4. The method according to claim 1, It is characterized in that The method further comprises: If at least one of network fluctuation, network frame loss or network resource exhaustion is detected, the latest frame snapshot in the local cache is taken out and sent to the server as response information of the operation instruction.

5. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: After sending the response information of the operation instruction to the server, a timer lock with the instruction sending cycle as the timing duration is set and locked, and the timer lock is unlocked after the timing duration reaches the next one after the instruction sending cycle; during the locking period of the timer lock, no response information of the operation instruction is sent to the server.

6. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: The instruction sending period for the application to send instructions to the server is calculated according to the frame rate set by the application.

7. A frame rate control device, It is characterized in that include: A response cache module, used for caching a snapshot of the graphical user interface of the application in real time in response to a user's operation on the application; An instruction determination module, used for obtaining an operation instruction after the user operation is completed, and determining whether the type of the operation instruction is a single operation or a series of operations; a duration determination module, for determining, if the type of the operation instruction is a plurality of consecutive operation instructions, whether the operation duration of the operation instruction exceeds an instruction sending cycle, wherein the instruction sending cycle is calculated in advance according to a frame rate set by the application program; The first sending module is used to take out the latest frame snapshot in the local cache and send it to the server as the response information of the operation instruction after the target delay if the operation duration of the operation instruction does not exceed the instruction sending cycle, wherein the target delay is the difference between the instruction sending cycle and the operation duration of the operation instruction.

8. The device according to claim 7, It is characterized in that The device also includes: The second sending module is used to take out the latest frame snapshot in the local cache and send it to the server as the response information of the operation instruction if the operation time of the operation instruction exceeds the instruction sending cycle.

9. The device according to claim 8, It is characterized in that The device also includes: The third sending module is used for taking out the latest frame snapshot in the local cache and sending it to the server as the response information of the operation instruction if the type of the instruction is a single operation.

10. The device according to claim 7, It is characterized in that The device also includes: The fourth sending module is used to take out the latest frame snapshot in the local cache and send it to the server as response information of the operation instruction if at least one of network fluctuation, network frame loss or network resource exhaustion is detected.

11. The device according to any one of claims 7 to 10, It is characterized in that The device also includes: The timing setting module is used to set a timer lock with the instruction sending cycle as the timing duration and lock it after sending the response information of the operation instruction to the server, and unlock the timer lock after the timing duration reaches the next one after the instruction sending cycle; the response information of the operation instruction is not sent to the server during the locking period of the timer lock.

12. The device according to any one of claims 7 to 10, It is characterized in that The device also includes: The cycle calculation module is used to calculate the instruction sending cycle of the application sending instructions to the server according to the frame rate set by the application.

13. A terminal device, include: Memory, processor, controls; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the frame rate control method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Webpage frame acquisition method, server and storage medium

    CN110457623A

  • Video generation method and device, server and storage medium

    CN110457624A

  • Cloud mobile phone implementation method and apparatus

    WO2023045392A1