Frame synchronization method, device, equipment, storage medium and program product

Abstract

The present application discloses a frame synchronization method, device, equipment, storage medium and program product, which belongs to the field of computer technology. The method includes: obtaining a predicted network delay time, the predicted network delay time is the predicted network delay time of a data frame; based on the predicted network delay time and the remaining display time, updating the display interval of the client, the remaining display time is the playback time corresponding to at least one cached display frame, and the display interval is used to indicate the time corresponding to playing the cached display frame in a single refresh; based on the display interval, the cached display frame is played to refresh the game screen. Through the above method, the problem of game screen freeze caused by network delay within a period of time can be effectively combated. In addition, the number of cached display frames in the client can be dynamically adjusted based on the predicted network delay time and the remaining display time corresponding to the cached display frame, thereby reducing the player's operation feedback delay.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a frame synchronization method, apparatus, device, storage medium and program product. Background Art

[0002] Frame synchronization (Lockstep) is a technology that sends data frames to clients through a server, and the data frames include the operation data of each client, thereby synchronizing the operation data of each client to all clients.

[0003] In the related technology, in order to avoid network delay or jitter, the client will cache M frames of data. After receiving the M+1 frame of data, the client will start to display the data of the first frame. That is, when there is network delay or jitter, the cached data frames can be used for display, thereby avoiding the phenomenon of freezing in the client screen. M is a positive integer.

[0004] However, the method of caching M frames of data frames in the related technology will delay the player's operation by M frames, resulting in the player having to wait M frames to receive his operation feedback on the client after operating the client. Therefore, how to avoid network delay or jitter while reducing the player's operation delay is an important issue that needs to be solved urgently. Summary of the invention

[0005] The present application provides a frame synchronization method, apparatus, device, storage medium and program product, and the technical solution is as follows:

[0006] According to one aspect of the present application, a frame synchronization method is provided, the method comprising:

[0007] Acquire a predicted network delay time, where the predicted network delay time is a predicted network delay time of a data frame, where the network delay time is the time required from when the server broadcasts the data frame until the client receives the data frame, and the data frame includes operation data of at least one client;

[0008] Based on the predicted network delay time and the remaining display time, updating the display interval of the client, the remaining display time is the playback duration corresponding to at least one cached display frame, the display interval is used to indicate the duration corresponding to playing the cached display frame in a single refresh, and the cached display frame is an image frame obtained after data processing of the received data frame;

[0009] The cached display frame is played based on the display interval to refresh the game screen.

[0010] According to one aspect of the present application, a frame synchronization device is provided, the device comprising:

[0011] An acquisition module, configured to acquire a predicted network delay time, wherein the predicted network delay time is a predicted network delay time of a data frame, wherein the network delay time is the time required from when the server broadcasts the data frame until the client receives the data frame, and the data frame includes operation data of at least one client;

[0012] an updating module, configured to update a display interval of the client based on the predicted network delay time and the remaining display time, wherein the remaining display time is the playing time corresponding to at least one cached display frame, and the display interval is used to indicate the playing time corresponding to the cached display frame in a single refresh, and the cached display frame is an image frame obtained after data processing of the received data frame;

[0013] A refresh module is used to play the cached display frame based on the display interval to refresh the game screen.

[0014] According to another aspect of the present application, a computer device is provided, which includes: a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the frame synchronization method described above.

[0015] According to another aspect of the present application, a computer storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the frame synchronization method described above.

[0016] According to another aspect of the present application, a computer program product is provided, which includes a computer program stored in a computer-readable storage medium; the computer program is read and executed from the computer-readable storage medium by a processor of a computer device, so that the computer device executes the frame synchronization method described above.

[0017] The beneficial effects of the technical solution provided by this application include at least:

[0018] By obtaining the predicted network delay time; based on the predicted network delay time and the remaining display time, updating the display interval of the client, the display interval is used to indicate the duration corresponding to playing the cached display frame in a single refresh; based on the display interval, playing the cached display frame to refresh the game screen. The cached display frames in this application can effectively combat the problem of game screen freeze caused by network delays within a period of time. In addition, based on the predicted network delay time and the remaining display time corresponding to the cached display frames, the number of cached display frames in the client can be dynamically adjusted, thereby reducing the player's operation feedback delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a schematic diagram of a frame synchronization process provided by an exemplary embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a frame synchronization method provided by an exemplary embodiment of the present application;

[0022] Figure 3 is a schematic diagram of the architecture of a computer system provided by an exemplary embodiment of the present application;

[0023] Figure 4 is a flow chart of a frame synchronization method provided by an exemplary embodiment of the present application;

[0024] Figure 5 is a flow chart of a frame synchronization method provided by an exemplary embodiment of the present application;

[0025] Figure 6 is a schematic diagram of a frame chasing state provided by an exemplary embodiment of the present application;

[0026] Figure 7 is a schematic diagram of a stable display state provided by an exemplary embodiment of the present application;

[0027] Figure 8 It is a schematic diagram of an implementation architecture of a frame synchronization method provided by an exemplary embodiment of the present application;

[0028] Fig. 9 is a flow chart of a frame synchronization method provided by an exemplary embodiment of the present application;

[0029] Fig.10 is a block diagram of a frame synchronization device provided by an exemplary embodiment of the present application;

[0030] Fig.11 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the present application clearer, the implementation mode of the present application will be further described in detail below in conjunction with the accompanying drawings. The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation modes described in the following exemplary embodiments do not represent all implementation modes consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0032] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0033] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.

[0034] First, the related technologies involved in the embodiments of this application are briefly introduced:

[0035] Server frame interval: refers to how often the server broadcasts a data frame; or, the server frame interval refers to the time it takes for the client to stably update the game screen after receiving the frame data.

[0036] Data frame: includes operation data of at least one client.

[0037] Cache display frame: an image frame obtained after data processing of the received data frame.

[0038] Delay: Network delay refers to the time from when the server broadcasts until the client receives the data frame.

[0039] Server frame rate: refers to the number of times the server broadcasts data frames in one second.

[0040] Client frame rate: refers to the number of times the client updates the game screen in one second.

[0041] Display time: refers to the number of display frames used by the client multiplied by the server frame interval. Since the client frame rate is not necessarily equal to the server frame rate, the number of display frames here is not necessarily an integer.

[0042] Remaining display time: the time that the client can display stably without receiving new data frames, or the number of data frames received by the client multiplied by the server frame interval, minus the client display time.

[0043] Basic display interval (also called client frame interval): the preset duration of the cached display frame played during a single refresh, or the preset time difference between the time the client updates the game screen and the time the game screen was last updated.

[0044] In online games, such as Fighting Technology Game (FTG), Action Game (ACT), Multiplayer Online Battle Arena (MOBA), Real-Time Strategy Game (RTS), Massive / Massively Multiplayer Online Game (MMOG), Shooting Game (STG), First Person Shooting Game (FPS), Third Person Shooting Game (TPS) and arcade games, it is usually necessary to use synchronization technology to make multiple clients display consistent picture effects so that players can have a better gaming experience.

[0045] Lockstep technology is a commonly used synchronization technology in online games. Its principle is that the server obtains the player operations sent to the server by all clients in the battlefield (or game, or field of view) within a unit of time and broadcasts them. In other words, frame synchronization is achieved by the server broadcasting the player's operation instructions obtained to all clients at regular intervals, synchronizing the operation instructions of all players on all clients, and then synchronizing the game status on all clients. Among them, player operations can also be understood as player operation instructions, or player instructions, or game actions.

[0046] Specifically, the frame synchronization process is as follows: Figure 1As shown, the timer is set to have a clock interrupt (i.e., a "tick") every 100ms, and the unit time is 100ms. The time between two adjacent clock interrupts is called a time slice (time slice or tick period). For ease of explanation, the data (or message) sent by the client to the server is called uplink data (or message), and the data (or message) sent by the server to the client is called downlink data (or message). In the time slice between the Nth clock interrupt and the N+1th clock interrupt, the client 1 and the client 2 respectively send the uplink Nth frame message (also understood as the Nth frame data, or the Nth frame player operation instruction) to the server (also understood as the host), as shown in steps 101 and 102. At the N+1th clock interrupt, the server broadcasts the received uplink Nth frame message to the client 1 and the client 2 through the downlink data packet, as shown in steps 103 and 104. During the time between the N+1th clock interruption and the N+2th clock interruption, the client 1 and the client 2 respectively perform logic processing on the Nth frame message in the received downlink data packet to display the corresponding screen effect, as shown in step 105 and step 106. During the time between the N+1th clock interruption and the N+2th clock interruption, the client 1 also sends the uplink N+1th frame message to the server, as shown in step 107. Then, during the N+2th clock interruption, the server broadcasts the received uplink N+1th frame message to the client 1 and the client 2 through the downlink data packet, as shown in step 108 and step 109.

[0047] In games that use frame synchronization technology, in order to distinguish frame data (or data frames) of different time slices, the server adds a logical sequence number to the player operation in the battlefield (or game, or field of view) obtained within a unit of time, and broadcasts (or synchronizes) the logical sequence number together with the player operation to all clients in the battlefield (or game, or field of view). Among them, the logical sequence number used to distinguish data frames of different time slices is also called a frame sequence number or frame index.

[0048] It can also be understood that the server logically divides the set of player operations within a unit of time to obtain logical frames (or game frames), referred to as "frames". For example, for a game that transmits 15 frames per second (15 Frames Per Second, 15FPS), the server caches (can also be understood as collecting) player operations in the battlefield (or game, or field of view) every 66ms and divides them into a logical frame, and assigns a frame number, or frame index, to each logical frame. When the server and client perform frame synchronization, they both use logical frames as the smallest unit.

[0049] The embodiment of the present application provides a schematic diagram of a frame synchronization method, such as Figure 2As shown, the method can be executed by a client. The method includes:

[0050] Exemplarily, the client obtains the predicted network delay time, and updates the display interval of the client based on the predicted network delay time and the remaining display time; the client plays the cached display frames based on the display interval to refresh the game screen.

[0051] The predicted network delay time refers to the predicted network delay time of a data frame; or, the predicted network delay time refers to the predicted maximum network delay time of subsequent data frames; or, the predicted network delay time refers to the predicted network delay time of the next data frame.

[0052] Network latency refers to the time it takes from the server broadcasting a data frame until the client receives the data frame.

[0053] The data frame includes operation data of at least one client.

[0054] The cache display frame refers to an image frame obtained after data processing of the received data frame.

[0055] The remaining display time refers to the playback time corresponding to at least one cached display frame; or, the remaining display time refers to the time during which the client 20 can still display stably without receiving new data frames; or, the remaining display time refers to the difference between the display time of the received data frames and the display time of the image frames already displayed by the client 20.

[0056] The display time of the received data frames refers to the product of the number of received data frames and the server frame interval.

[0057] The server frame interval refers to how often the server 10 broadcasts a data frame; or, the server frame interval refers to the time it takes for the client 20 to stably update the game screen after receiving the frame data.

[0058] The display interval is used to indicate the playback speed of the cached display frames, or the display interval is used to indicate the playback time corresponding to the cached display frames in a single update, or the display interval is used to indicate the duration corresponding to the playback of the cached display frames in a single refresh.

[0059] For example, a display interval of 0.1s means that when the game screen is updated once, the cached display frames within 0.1s are played, that is, each time the game screen is updated, 0.1s is advanced forward and the cached display frames within 0.1s are played.

[0060] The basic display interval refers to the preset duration corresponding to playing the cached display frames during a single refresh; or, the basic display interval refers to the preset time difference between the time when the client updates the game screen and the time when the game screen was last updated; or, the basic display interval is the default value of the display interval corresponding to the client. The basic display interval is a fixed update time set by the client itself. For example, the refresh frequency of the client is fixed, and the basic display interval is set to 0.5s. When the client refreshes the game screen for the first time, it advances 0.5s and plays the cached display frames within 0.5s.

[0061] It should be noted that the basic display interval is a default value of the display interval, or the basic display interval is a value of the display interval in a stable display state.

[0062] In the frame synchronization architecture, the server 10 continuously sends data frames to the client 20. Assuming that the frame rate of the server 10 is 10, that is, 10 data frames are sent per second, in an ideal state, the client 20 will receive a data frame every 0.1 seconds, and the client 20 can refresh the game screen within 0.1 seconds after calculation. Figure 2 As shown in Figure (a), when the network delay is stable, the client 20 receives a data frame every 0.1 seconds, and can display it stably and smoothly in the next 0.1 seconds. However, in reality, network delay is not a stable value. In most cases, it fluctuates within a range. In special cases, the data frame delay jitter may increase for a period of time. For example, the network delay of data frame 4 suddenly becomes much larger, and the client 20 does not display data for a period of time after displaying data frame 3, and the screen of the client 20 becomes stuck. After receiving data frame 4, due to the stuck period, the subsequent network delay of the client 20 relative to the server 10 becomes larger.

[0063] Exemplarily, the client 20 determines the playback status of the client 20 based on the difference between the predicted network delay time and the remaining display time; the client 20 updates the display interval based on the playback status.

[0064] The playback state refers to the state when the client 20 plays the cached display frame.

[0065] Optionally, the playback state includes a frame chasing state and a stable display state.

[0066] The frame chasing state refers to a state in which cached display frames are played at a double speed, and the cached display frames in the client 20 are reduced by accelerating the playback.

[0067] The stable display state refers to a state in which cached display frames are played at a basic display interval.

[0068] In some embodiments, in response to the difference between the remaining display time and the predicted network delay time being less than or equal to the first time threshold, the client 20 determines that the state of the client 20 is a stable display state; the client 20 updates the display interval to a basic display interval based on the state parameters of the stable display state.

[0069] For example, the first time threshold is 5s, the remaining display time is 7s, and the predicted network delay time is 3s. When the difference between the remaining display time and the predicted network delay time is less than the first time threshold, it is determined that the state of the client 20 is a stable display state.

[0070] like Figure 2 As shown in FIG. 2( b ), 7 cached display frames are cached in the client 20. When the client 20 is in a stable display state, the value of the display interval is the value of the basic display interval. Assuming that the value of the basic display interval is 0.1 s, the client 20 advances one basic display interval each time the game screen is updated, that is, the cached display frame within 0.1 s is played each time the game screen is updated.

[0071] In some embodiments, in response to the difference between the remaining display time and the predicted network delay time being greater than a first time threshold, the client 20 determines that the client's state is a frame chasing state; based on the state parameters of the frame chasing state, the display interval is updated to the product of the basic display interval and the magnification.

[0072] For example, the first time threshold is 5s, the remaining display time is 10s, and the predicted network delay time is 3s. When the difference between the remaining display time and the predicted network delay time (7s) is greater than the first time threshold (5s), it is determined that the playback state of the client 20 is a frame chasing state.

[0073] like Figure 2 As shown in Figure (c) in the figure, 7 cached display frames are cached in the client 20. Assuming that the value of the basic display interval is 0.1s and the magnification is 2, that is, the cached display frames are played at a multiple of 2, when the client 20 is in the frame chasing state, the client 20 advances two basic display intervals each time the game screen is updated, that is, the cached display frames within 0.2s are played at a multiple speed each time the game screen is updated, so that the cached display frames in the client 20 are consumed quickly, achieving the purpose of reducing the cached display frames and reducing the operation delay of the player.

[0074] For example, it is assumed that the update frequency of the client 20 is 10 times / second, the basic display interval is 0.1s, and the multiplier is 2. Then, when the client 20 is in a stable display state, the display interval value is 0.1s, and the client 20 plays the cached display frame within 0.1s each time the game screen is updated, that is, the content of the cached display frame of 1s in the cache is played within 1s; when the client 20 is in a frame chasing state, the display interval value is 0.2s, and the client 20 plays the content of the cached display frame of 0.2s at a double speed within 0.1s each time the game screen is updated, that is, the content of the cached display frame of 2s in the cache is played within 1s, so that the cached display frames in the client 20 are consumed quickly, achieving the purpose of reducing the cached display frames and reducing the operation delay of the player.

[0075] In summary, the method provided in this embodiment obtains the predicted network delay time; based on the predicted network delay time and the remaining display time, updates the display interval of the client, the display interval is used to indicate the duration corresponding to playing the cached display frame in a single refresh; based on the display interval, the cached display frame is played to refresh the game screen. The cached display frames in this application can effectively combat the problem of game screen freeze caused by network delays within a period of time. In addition, based on the predicted network delay time and the remaining display time corresponding to the cached display frames, the number of cached display frames in the client can be dynamically adjusted, thereby reducing the player's operation feedback delay.

[0076] Figure 3 The computer system 300 includes a first terminal 310 , a server 320 , and a second terminal 330 .

[0077] The first terminal 310 has installed and runs an application program supporting frame synchronization, such as a frame synchronization game, etc. The first terminal 310 is a terminal used by a first player.

[0078] In some embodiments, the first terminal 310 may be considered as a first player using the first terminal 310 .

[0079] The first terminal 310 is connected to the server 320 via a wireless network or a wired network.

[0080] The server 320 includes one of a server, a plurality of servers, a cloud computing platform and a virtualization center. Schematically, the server 320 includes a processor 321 and a memory 322, and the memory 322 includes a receiving module 3221, a display module 3222 and a control module 3223. The server 320 is used to provide background services for applications supporting frame synchronization. Optionally, the server 320 undertakes the main computing work, and the first terminal 310 and the second terminal 330 undertake the secondary computing work; or, the server 320 undertakes the secondary computing work, and the first terminal 310 and the second terminal 330 undertake the main computing work; or, the server 320, the first terminal 310 and the second terminal 330 adopt a distributed computing architecture for collaborative computing.

[0081] An application supporting frame synchronization is installed and run on the second terminal 330. The second terminal 330 is a terminal used by the second player.

[0082] In some embodiments, the second terminal 330 may be considered as a second player using the second terminal 330 .

[0083] Optionally, the first player and the second player are in or not in the same field of view, or the first player and the second player are in or not in the same game, or the first player and the second player are in or not in the same virtual scene. Optionally, the first player and the second player may belong to the same team, the same organization, have a friend relationship, or have temporary communication permissions.

[0084] Optionally, the applications installed on the first terminal 310 and the second terminal 330 are the same, or the applications installed on the two terminals are the same type of applications of different control system platforms. The first terminal 310 may refer to one of a plurality of terminals, and the second terminal 330 may refer to one of a plurality of terminals. This embodiment is only illustrated by the first terminal 310 and the second terminal 330. The device types of the first terminal 310 and the second terminal 330 are the same or different, and the device types include but are not limited to: at least one of a smart phone, a tablet computer, an e-book reader, a laptop computer, a desktop computer, a television, an augmented reality (AR) terminal, a virtual reality (VR) terminal, a mixed reality (MR) terminal, an extended reality (XR) terminal, a baffle reality (BR) terminal, a cinematic reality (CR) terminal, and a deceived reality (DR) terminal. The following embodiments are illustrated by taking the terminal including a smart phone as an example.

[0085] Those skilled in the art will appreciate that the number of the above terminals or players may be more or less. For example, the above terminal or player may be only one, or the above terminals or players may be dozens or hundreds, or more. The embodiment of the present application does not limit the number of terminals or players and the device type.

[0086] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the information involved in this application is obtained with full authorization, and the terminal and server only cache the information during the program operation, and will not solidify the storage and reuse the relevant data of the information.

[0087] Figure 4 : is a flowchart of a frame synchronization method provided by an exemplary embodiment of the present application. The method can be executed by a client. The method includes:

[0088] Step 402: Obtain predicted network delay time.

[0089] The predicted network delay time refers to the predicted network delay time of a data frame; or, the predicted network delay time refers to the predicted maximum network delay time of subsequent data frames; or, the predicted network delay time refers to the predicted network delay time of the next data frame.

[0090] Network latency refers to the time it takes from the server broadcasting a data frame until the client receives the data frame.

[0091] The data frame includes operation data of at least one client.

[0092] Step 404: Based on the predicted network delay time and the remaining display time, update the display interval of the client.

[0093] The remaining display time refers to the playback time corresponding to at least one cached display frame; or, the remaining display time refers to the time during which the client can still display stably without receiving new data frames; or, the remaining display time refers to the difference between the display time of the received data frames and the display time of the image frames already viewed by the client.

[0094] The display time of the received data frames refers to the product of the number of received data frames and the server frame interval.

[0095] The server frame interval refers to how often the server broadcasts a data frame; or, the server frame interval refers to the time it takes for the client to stably update the game screen after receiving the frame data.

[0096] The display interval is used to indicate the playback speed of the cached display frames, or the display interval is used to indicate the duration corresponding to the playback of the cached display frames during a single refresh.

[0097] For example, a display interval of 0.1s means that when the game screen is updated once, the cached display frames within 0.1s are played, that is, each time the game screen is updated, 0.1s is advanced forward and the cached display frames within 0.1s are played.

[0098] Exemplarily, the client determines the display interval for playing the cached display frames according to the predicted network delay time and the number of cached display frames, thereby realizing controllable playback of the cached display frames. When there are many cached display frames, the display interval is increased to quickly consume the cached display frames; when there are few cached display frames, the display interval is reduced to slow down the consumption of cached display frames.

[0099] Step 406: Play the cached display frames based on the display interval to refresh the game screen.

[0100] Exemplarily, after determining the display interval, the client selects cached display frames for interpolated playback based on the display interval, thereby refreshing the game screen.

[0101] Interpolation playback refers to selecting cached display frames within the display interval for playback according to the display interval.

[0102] For example, assuming that the display interval is 0.1s, the client advances 0.1s each time the game screen is updated, that is, the content of the cached display frame of 1s in the cache is played within 1s, thereby refreshing the game screen.

[0103] In summary, the method provided in this embodiment obtains the predicted network delay time; based on the predicted network delay time and the remaining display time, updates the display interval of the client, the display interval is used to indicate the duration corresponding to playing the cached display frame in a single refresh; based on the display interval, the cached display frame is played to refresh the game screen. The cached display frames in this application can effectively combat the problem of game screen freeze caused by network delays within a period of time. In addition, based on the predicted network delay time and the remaining display time corresponding to the cached display frames, the number of cached display frames in the client can be dynamically adjusted, thereby reducing the player's operation feedback delay.

[0104] Figure 5 : is a flowchart of a frame synchronization method provided by an exemplary embodiment of the present application. The method can be executed by a client. The method includes:

[0105] Step 502: Obtain predicted network delay time.

[0106] The predicted network delay time refers to the predicted network delay time of a data frame; or, the predicted network delay time refers to the predicted maximum network delay time of subsequent data frames; or, the predicted network delay time refers to the predicted network delay time of the next data frame.

[0107] Network latency refers to the time it takes from the server broadcasting a data frame until the client receives the data frame.

[0108] The data frame includes operation data of at least one client.

[0109] In some embodiments, the step of obtaining the predicted network delay time includes: the client obtains historical network delay records; the client predicts the predicted network delay time based on the historical network delay records.

[0110] Historical network delay records are used to record network delay time.

[0111] Exemplarily, the client obtains the mean value of the head network delay time and the mean value of the tail network delay time from the network delay time series; the client subtracts the mean value of the tail network delay time from the mean value of the head network delay time to obtain the predicted network delay time.

[0112] The mean network delay time at the head refers to the mean network delay time at the head of the network delay time series.

[0113] The mean tail network delay time refers to the mean of the network delay time arranged at the tail of the network delay time series.

[0114] Exemplarily, the steps of obtaining the mean value of the head network delay time and the mean value of the tail network delay time include:

[0115] The client arranges the network delay times recorded in the historical network delay records in descending order to obtain a network delay time series; the client extracts the first N network delay times arranged at the head of the network delay time series and calculates the average to obtain the head network delay time average; the client extracts the last N network delay times arranged at the tail of the network delay time series and calculates the average to obtain the tail network delay time average; N is a positive integer.

[0116] In some embodiments, the client extracts the first N network delay times arranged at the head of the network delay time sequence, and calculates the average after removing the largest M network delay times to obtain the average network delay time at the head; the client extracts the last N network delay times arranged at the tail of the network delay time sequence, and calculates the average after removing the M smallest network delay times to obtain the average network delay time at the tail; M is a positive integer.

[0117] Step 504: Determine the playback status of the client based on the difference between the predicted network delay time and the remaining display time.

[0118] The remaining display time refers to the playback time corresponding to at least one cached display frame; or, the remaining display time refers to the time during which the client can still display stably without receiving new data frames; or, the remaining display time refers to the difference between the display time of the received data frames and the display time of the image frames already viewed by the client.

[0119] The display time of the received data frames refers to the product of the number of received data frames and the server frame interval.

[0120] The server frame interval refers to how often the server broadcasts a data frame; or, the server frame interval refers to the time it takes for the client to stably update the game screen after receiving the frame data.

[0121] The playback state refers to the state of the client when it plays the cached display frames.

[0122] Optionally, the playback state includes a frame chasing state and a stable display state.

[0123] The frame chasing state refers to the state of playing cached display frames at double speed, which reduces the cached display frames in the client by accelerating the playback.

[0124] The stable display state refers to a state in which cached display frames are played at a basic display interval.

[0125] Exemplarily, the client determines the playback status of the client based on the difference between the predicted network delay time and the remaining display time.

[0126] Optionally, in response to a difference between the remaining display time and the predicted network delay time being greater than a first time threshold, the client determines that the state of the client is a frame chasing state.

[0127] For example, the first time threshold is 5s, the remaining display time is 10s, and the predicted network delay time is 3s. When the difference between the remaining display time and the predicted network delay time (7s) is greater than the first time threshold (5s), the client state is determined to be a frame chasing state.

[0128] Optionally, in response to a difference between the remaining display time and the predicted network delay time being less than or equal to a first time threshold, the client determines that the state of the client is a stable display state.

[0129] For example, the first time threshold is 5s, the remaining display time is 7s, and the predicted network delay time is 3s. When the difference between the remaining display time and the predicted network delay time is less than the first time threshold, it is determined that the client state is a stable display state.

[0130] Step 506: Based on the state parameters of the frame tracking state, update the value of the display interval to the product of the basic display interval and the magnification.

[0131] The display interval is used to indicate the playback speed of the cached display frames, or the display interval is used to indicate the duration corresponding to the playback of the cached display frames during a single refresh.

[0132] Exemplarily, the client updates the display interval based on the playback status.

[0133] When it is determined that the client is in the frame chasing state, the client updates the display interval to the product of the basic display interval and the magnification based on the state parameters of the frame chasing state.

[0134] The basic display interval refers to the preset duration of playing cached display frames in a single refresh; or, the basic display interval refers to the preset time difference between the time when the client updates the game screen and the last time the game screen was updated. The basic display interval is a fixed update time set by the client itself. For example, if the basic display interval is set to 0.5s, the first time the client updates the game screen, it will advance 0.5s and play the cached display frames within 0.5s.

[0135] The magnification refers to the multiple of the speed at which cached display frames are played.

[0136] In some embodiments, the state parameters of the frame chasing state include a frame chasing level.

[0137] The client determines the frame chasing level based on the difference between the remaining display time and the predicted network delay time.

[0138] The frame chasing level is used to indicate the difference between the remaining display time and the predicted network delay time.

[0139] The client determines the magnification according to the frame chasing level of the frame chasing state; based on the magnification, the client updates the value of the display interval to the product of the basic display interval and the magnification.

[0140] For example, when the difference between the remaining display time and the predicted network delay time belongs to the first numerical range, the value of the multiplier is determined to be the first multiplier; when the difference between the remaining display time and the predicted network delay time belongs to the second numerical range, the value of the multiplier is determined to be the second multiplier.

[0141] For example, Figure 6 The schematic diagram of the frame chasing state shown in the figure shows that 7 cached display frames are cached in the client. Assuming that the value of the basic display interval is 0.1s, that is, before updating the display interval, each refresh plays the cached display frame within 0.1s. When the client enters the frame chasing state, assuming the magnification is 2, the client advances two basic display intervals each time the game screen is updated, that is, the cached display frame of 0.2s is played at a double speed each time the game screen is updated, so that the cached display frames in the client are consumed quickly, achieving the purpose of reducing cached display frames and reducing player operation delays.

[0142] In some embodiments, when the client is in a frame chasing state and the difference between the remaining display time and the predicted network delay time decreases to less than a second time threshold, the client is switched from the frame chasing state to a stable display state.

[0143] The second time threshold is smaller than the first time threshold, and the stable display state refers to a state in which cached display frames are played at a basic display interval.

[0144] In some embodiments, when the client is in a frame chasing state and the difference between the remaining display time and the predicted network delay time is reduced to less than a second time threshold, the client is switched from the frame chasing state to a stable display state to avoid playing the cached display frame completely.

[0145] Optionally, when the client is in a frame chasing state and the difference between the remaining display time and the predicted network delay time is reduced to less than a second time threshold, the total value of the display interval is updated to the difference between the remaining display time and the predicted network delay time, in order to leave enough cached display frames to resist the next predicted network delay time.

[0146] The total value of the display interval refers to the sum of the display intervals of the playback cache display frames.

[0147] For example, if the remaining display time is 10s and the predicted network delay time is 3s, when the client is in the frame chasing state, the total value of the display interval is controlled to be less than or equal to 7s, so that the cached display frames are kept at a certain number to resist network fluctuations, avoiding the game screen from being stuck when the network delay occurs in the next frame.

[0148] Step 508: Based on the state parameter of the stable display state, update the value of the display interval to the value of the basic display interval.

[0149] Exemplarily, the client updates the display interval based on the playback status.

[0150] In the case where it is determined that the state of the client is a stable display state, the value of the display interval is updated to a value of the basic display interval based on the state parameter of the stable display state.

[0151] The stable display state refers to a state in which cached display frames are played at a basic display interval.

[0152] It should be noted that the basic display interval is a default value of the display interval, or the basic display interval is a value of the display interval in a stable display state.

[0153] For example, Figure 7 In the schematic diagram of the stable display state shown, there are 7 cached display frames cached in the client, assuming that the value of the basic display interval is 0.1s, that is, before the display interval is updated, each refresh plays the cached display frame within 0.1s. When the client is in a stable display state, the value of the display interval is the value of the basic display interval, and the client advances one basic display interval each time the game screen is updated, that is, the cached display frame within 0.1s is played each time the game screen is updated.

[0154] In some embodiments, when the client is in a stable display state and the difference between the remaining display time and the basic display interval is less than zero, the value of the display interval is updated to the value of the remaining display time.

[0155] That is, when the number of cached display frames is less than the number of cached display frames corresponding to the basic display interval, the value of the display interval is updated to the value of the remaining display time. For example, if the basic display interval is 0.3s and the remaining display time is only 0.2s, the value of the display interval is updated to the value of the remaining display time, that is, the game screen is updated to the latest progress.

[0156] Step 510: Play the cached display frames based on the display interval to refresh the game screen.

[0157] Exemplarily, after determining the display interval, the client selects cached display frames for interpolated playback based on the display interval, thereby refreshing the game screen.

[0158] For example, assuming that the client's update frequency is 10 times / second, the basic display interval is 0.1s, and the multiplier is 2. Then, when the client is in a stable display state, the display interval value is 0.1s, and the client plays the cached display frame within 0.1s each time the game screen is updated, that is, the content of the cached display frame of 1s in the cache is played within 1s; when the client is in a frame-chasing state, the display interval value is 0.2s, and the client plays the content of the cached display frame of 0.2s at a double speed within 0.1s each time the game screen is updated, that is, the content of the cached display frame of 2s in the cache is played within 1s, so that the cached display frames in the client are consumed quickly, achieving the purpose of reducing the cached display frames and reducing the player's operation delay.

[0159] In summary, the method provided in this embodiment obtains the predicted network delay time; based on the predicted network delay time and the remaining display time, updates the display interval of the client, the display interval is used to indicate the duration corresponding to playing the cached display frame in a single refresh; based on the display interval, the cached display frame is played to refresh the game screen. The cached display frames in this application can effectively combat the problem of game screen freeze caused by network delays within a period of time. In addition, based on the predicted network delay time and the remaining display time corresponding to the cached display frames, the number of cached display frames in the client can be dynamically adjusted, thereby reducing the player's operation feedback delay.

[0160] The method provided in this embodiment determines the playing status of the client based on the predicted network delay time and the remaining display time, and updates the display interval of the client according to the playing status. The present application adjusts the display interval according to the playing status of the client, and can dynamically adjust the number of cached display frames in the client, thereby reducing the player's operation feedback delay and improving the user experience.

[0161] The method provided in this embodiment determines the playback state of the client based on the predicted network delay time and the remaining display time. In the frame chasing state, the cached display frames are played in multiples; in the stable display state, the cached display frames are played at the basic display interval. This application shows two playback states of the client, and adjusts the corresponding display intervals according to different states, thereby dynamically adjusting the number of cached display frames in the client, thereby reducing the player's operation feedback delay and improving the user experience.

[0162] The method provided in this embodiment obtains historical network delay records, arranges the network delay times recorded in the historical network delay records in descending order, obtains a network delay time series, obtains the head network delay time mean and the tail network delay time mean from the network delay time series; subtracts the tail network delay time mean from the head network delay time mean to obtain the predicted network delay time. The present application predicts the possible network delay time based on the historical network delay records, and adjusts the number of cached display frames based on the predicted network delay time, thereby effectively combating the problem of game screen freezes caused by network delays within a period of time, and reducing the player's operation feedback delay, thereby improving the user experience.

[0163] Figure 8 801 , a display strategy module 802 , and a display module 803 are provided in the client, and the jitter prediction module 801 , the display strategy module 802 , and the display module 803 are connected in sequence.

[0164] The jitter prediction module 801 is used to predict network delay time.

[0165] The jitter prediction module 801 obtains historical network delay records; the jitter prediction module 801 predicts the predicted network delay time based on the historical network delay records.

[0166] The jitter prediction module 801 arranges the network delay times recorded in the historical network delay records in descending order through the client to obtain a network delay time series; the jitter prediction module 801 extracts the first N network delay times arranged at the head of the network delay time series and calculates the average to obtain the head network delay time average; the jitter prediction module 801 extracts the last N network delay times arranged at the tail of the network delay time series and calculates the average to obtain the tail network delay time average; N is a positive integer. The jitter prediction module 801 subtracts the tail network delay time average from the head network delay time average to obtain the predicted network delay time.

[0167] The display strategy module 802 is used to determine the playback status of the client based on the difference between the predicted network delay time and the remaining display time, and update the display interval based on the playback status.

[0168] In response to the difference between the remaining display time and the predicted network delay time being greater than the first time threshold, the display strategy module 802 determines that the client is in a frame chasing state; based on the state parameters of the frame chasing state, the display strategy module 802 updates the value of the display interval to the product of the basic display interval and the magnification.

[0169] In response to the difference between the remaining display time and the predicted network delay time being less than or equal to the first time threshold, the display policy module 802 determines that the client state is a stable display state; the display policy module 802 updates the value of the display interval to the value of the basic display interval based on the state parameters of the stable display state.

[0170] The display module 803 is used to select cached display frames for playback based on the display interval, thereby refreshing the game screen.

[0171] In summary, the method provided in this embodiment obtains the predicted network delay time; based on the predicted network delay time and the remaining display time, updates the display interval of the client, the display interval is used to indicate the duration corresponding to playing the cached display frame in a single refresh; based on the display interval, the cached display frame is played to refresh the game screen. The cached display frames in this application can effectively combat the problem of game screen freeze caused by network delays within a period of time. In addition, based on the predicted network delay time and the remaining display time corresponding to the cached display frames, the number of cached display frames in the client can be dynamically adjusted, thereby reducing the player's operation feedback delay.

[0172] Fig. 9 : is a flowchart of a frame synchronization method provided by an exemplary embodiment of the present application. The method can be executed by a client. The method includes:

[0173] Step 901: Start.

[0174] Step 902: Calculate the remaining display time of the client.

[0175] The remaining display time refers to the playback time corresponding to at least one cached display frame; or, the remaining display time refers to the time during which the client can still display stably without receiving new data frames; or, the remaining display time refers to the difference between the display time of the received data frames and the display time of the image frames already viewed by the client.

[0176] Exemplarily, the client determines the remaining display time of the client according to the playback time corresponding to the cached display frame that has been cached.

[0177] Step 903: Determine whether the difference between the remaining display time and the predicted network delay time is greater than a first time threshold.

[0178] The predicted network delay time refers to the predicted network delay time of a data frame; or, the predicted network delay time refers to the predicted maximum network delay time of subsequent data frames; or, the predicted network delay time refers to the predicted network delay time of the next data frame.

[0179] Network latency refers to the time it takes from the server broadcasting a data frame until the client receives the data frame.

[0180] The data frame includes operation data of at least one client.

[0181] After determining the remaining display time, determine whether the difference between the remaining display time and the predicted network delay time is greater than the first time threshold. If the difference between the remaining display time and the predicted network delay time is greater than the first time threshold, execute step 907; if the difference between the remaining display time and the predicted network delay time is less than or equal to the first time threshold, execute step 904.

[0182] Step 904: Update the value of the display interval to the value of the basic display interval.

[0183] Exemplarily, when the difference between the remaining display time and the predicted network delay time is less than or equal to the first time threshold, it is determined that the state of the client is a stable display state.

[0184] The stable display state refers to a state in which cached display frames are played at a basic display interval.

[0185] In the case where it is determined that the state of the client is a stable display state, the value of the display interval is updated to a value of the basic display interval based on the state parameter of the stable display state.

[0186] Step 905: Determine whether the difference between the remaining display time and the basic display interval is less than zero.

[0187] In some embodiments, when the client is in a stable display state, it is determined whether the difference between the remaining display time and the basic display interval is less than zero. If the difference between the remaining display time and the basic display interval is less than zero, step 906 is executed; if the difference between the remaining display time and the basic display interval is greater than or equal to zero, step 910 is executed.

[0188] Step 906: Update the value of the display interval to the value of the remaining display time.

[0189] When the difference between the remaining display time and the basic display interval is less than zero, the value of the display interval is updated to the value of the remaining display time.

[0190] That is, when the number of cached display frames is less than the number of cached display frames corresponding to the basic display interval, the value of the display interval is updated to the value of the remaining display time. For example, if the basic display interval is 0.3s and the remaining display time is only 0.2s, the value of the display interval is updated to the value of the remaining display time, that is, the game screen is updated to the latest progress.

[0191] Step 907: Update the value of the display interval to the product of the basic display interval and the magnification.

[0192] Exemplarily, when the difference between the remaining display time and the predicted network delay time is less than or equal to the first time threshold, it is determined that the state of the client is a frame chasing state.

[0193] When it is determined that the client is in the frame chasing state, the client updates the display interval to the product of the basic display interval and the magnification based on the state parameters of the frame chasing state.

[0194] Optionally, the state parameters of the frame chasing state include a frame chasing level.

[0195] The frame chasing level is used to indicate the difference between the remaining display time and the predicted network delay time.

[0196] The client determines the magnification according to the frame chasing level of the frame chasing state; based on the magnification, the client updates the value of the display interval to the product of the basic display interval and the magnification.

[0197] For example, when the difference between the remaining display time and the predicted network delay time belongs to the first numerical interval, the value of the multiplier is determined to be the first multiplier; when the difference between the remaining display time and the predicted network delay time belongs to the second numerical interval, the value of the multiplier is determined to be the second multiplier.

[0198] Step 908: Determine whether the difference between the remaining display time and the predicted network delay time is reduced to be less than a second time threshold.

[0199] Exemplarily, when the client is in a frame chasing state, it is determined whether the difference between the remaining display time and the predicted network delay time is reduced to less than a second time threshold. If the difference between the remaining display time and the predicted network delay time is reduced to less than the second time threshold, step 909 is executed; if the difference between the remaining display time and the predicted network delay time is not reduced to less than the second time threshold, step 910 is executed.

[0200] Step 909: Update the total value of the display interval to the difference between the remaining display time and the predicted network delay time.

[0201] When the difference between the remaining display time and the predicted network delay time decreases to less than the second time threshold, the client is switched from the frame chasing state to the stable display state, thereby avoiding playing the cached display frames completely.

[0202] Optionally, when the client is in a frame chasing state and the difference between the remaining display time and the predicted network delay time is reduced to less than a second time threshold, the total value of the display interval is updated to the difference between the remaining display time and the predicted network delay time, in order to leave enough cached display frames to resist the next predicted network delay time.

[0203] The total value of the display interval refers to the sum of the display intervals of the playback cache display frames.

[0204] For example, if the remaining display time is 10s and the predicted network delay time is 3s, when the client is in the frame chasing state, the total value of the display interval is controlled to be less than or equal to 7s, so that the cached display frames are kept at a certain number to resist network fluctuations, avoiding the game screen from being stuck when the network delay occurs in the next frame.

[0205] Step 910: Refresh the game screen based on the display interval.

[0206] Exemplarily, after determining the display interval, the client selects cached display frames for interpolated playback based on the display interval, thereby refreshing the game screen.

[0207] Fig.10 A schematic diagram of the structure of a frame synchronization device provided by an exemplary embodiment of the present application is shown. The device can be implemented as all or part of a computer device through software, hardware, or a combination of both. The device includes:

[0208] An acquisition module 1001 is used to acquire a predicted network delay time, where the predicted network delay time is a predicted network delay time of a data frame, where the network delay time is the time required from when the server broadcasts the data frame until the client receives the data frame, and the data frame includes operation data of at least one client;

[0209] An updating module 1002 is used to update the display interval of the client based on the predicted network delay time and the remaining display time, wherein the remaining display time is the playback duration corresponding to at least one cached display frame, and the display interval is used to indicate the duration corresponding to playing the cached display frame during a single refresh, wherein the cached display frame is an image frame obtained after data processing of the received data frame;

[0210] The refresh module 1003 is used to play the cached display frame based on the display interval to refresh the game screen.

[0211] In some embodiments, the update module 1002 is used to determine the playback status of the client based on the difference between the predicted network delay time and the remaining display time, and the playback status refers to the state of the client when playing the cached display frame; and update the display interval based on the playback status.

[0212] In some embodiments, the update module 1002 is used to determine that the state of the client is the frame chasing state in response to the difference between the remaining display time and the predicted network delay time being greater than a first time threshold.

[0213] In some embodiments, the refresh module 1003 is used to update the value of the display interval to the product of the basic display interval and the magnification based on the state parameter of the frame tracking state.

[0214] The basic display interval refers to a preset duration corresponding to playing a cached display frame in a single refresh, and the magnification refers to a multiple when playing the cached display frame at a multiple speed.

[0215] In some embodiments, the update module 1002 is used to determine the frame chasing level based on the difference between the remaining display time and the predicted network delay time; determine the multiplier according to the frame chasing level of the frame chasing state; and based on the multiplier, update the value of the display interval to the product of the basic display interval and the multiplier.

[0216] In some embodiments, the update module 1002 is used to switch the client from the frame chasing state to the stable display state when the client is in the frame chasing state and the difference between the remaining display time and the predicted network delay time is reduced to less than a second time threshold.

[0217] The second time threshold is smaller than the first time threshold, and the stable display state refers to a state in which the cached display frame is played at a basic display interval.

[0218] In some embodiments, the updating module 1002 is used to determine that the state of the client is the stable display state in response to the difference between the remaining display time and the predicted network delay time being less than or equal to a first time threshold.

[0219] In some embodiments, the refresh module 1003 is used to update the value of the display interval to the value of the basic display interval based on the state parameter of the stable display state.

[0220] In some embodiments, the updating module 1002 is used to update the value of the display interval to the value of the remaining display time when the client is in the stable display state and the difference between the remaining display time and the basic display interval is less than zero.

[0221] In some embodiments, the acquisition module 1001 is used to acquire historical network delay records, where the historical network delay records are used to record network delay time; and to predict the predicted network delay time based on the historical network delay records.

[0222] In some embodiments, the acquisition module 1001 is used to obtain the head network delay time mean and the tail network delay time mean from the network delay time series; subtract the tail network delay time mean from the head network delay time mean to obtain the predicted network delay time.

[0223] The head network delay time mean refers to the mean of the network delay time arranged at the head in the network delay time sequence, and the tail network delay time mean refers to the mean of the network delay time arranged at the tail in the network delay time sequence.

[0224] In some embodiments, the acquisition module 1001 is used to arrange the network delay times recorded in the historical network delay records in descending order to obtain the network delay time series; extract the first N network delay times arranged at the head of the network delay time series and calculate the average to obtain the head network delay time average; extract the last N network delay times arranged at the tail of the network delay time series and calculate the average to obtain the tail network delay time average; N is a positive integer.

[0225] Fig.11 The block diagram of the structure of a computer device 1100 provided by an exemplary embodiment of the present application is shown. The computer device 1100 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV). The computer device 1100 may also be called a user device, a portable terminal, or other names.

[0226] Typically, the computer device 1100 includes a processor 1101 and a memory 1102 .

[0227] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0228] The memory 1102 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one instruction, which is used to be executed by the processor 1101 to implement the frame synchronization method provided in the embodiment of the present application.

[0229] In some embodiments, the computer device 1100 may further include: a peripheral device interface 1103 and at least one peripheral device. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1104 , a touch display screen 1105 , a camera assembly 1106 , an audio circuit 1107 and a power supply 1108 .

[0230] The peripheral device interface 1103 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral device interface 1103 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0231] The radio frequency circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1104 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1104 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 1104 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1104 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0232] The touch display screen 1105 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. The touch display screen 1105 also has the ability to collect touch signals on the surface or above the surface of the touch display screen 1105. The touch signal may be input to the processor 1101 as a control signal for processing. The touch display screen 1105 is used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the touch display screen 1105 may be one, and the front panel of the computer device 1100 is set; in other embodiments, the touch display screen 1105 may be at least two, which are respectively set on different surfaces of the computer device 1100 or are folded; in some embodiments, the touch display screen 1105 may be a flexible display screen, which is set on a curved surface or a folded surface of the computer device 1100. Even, the touch display screen 1105 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The touch display screen 1105 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0233] The camera assembly 1106 is used to capture images or videos. Optionally, the camera assembly 1106 includes a front camera and a rear camera. Typically, the front camera is used to realize video calls or selfies, and the rear camera is used to realize the shooting of photos or videos. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, and a wide-angle camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, and the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0234] The audio circuit 1107 is used to provide an audio interface between the user and the computer device 1100. The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1101 for processing, or input them into the radio frequency circuit 1104 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the computer device 1100. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0235] The power supply 1108 is used to power various components in the computer device 1100. The power supply 1108 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1108 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0236] In some embodiments, the computer device 1100 further includes one or more sensors 1109 , including but not limited to: an acceleration sensor 1110 , a gyroscope sensor 1111 , a pressure sensor 1112 , an optical sensor 1113 , and a proximity sensor 1114 .

[0237] The acceleration sensor 1110 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the computer device 1100. For example, the acceleration sensor 1110 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1101 can control the touch display screen 1105 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1110. The acceleration sensor 1110 can also be used for collecting game or user motion data.

[0238] The gyro sensor 1111 can detect the body direction and rotation angle of the computer device 1100, and the gyro sensor 1111 can cooperate with the acceleration sensor 1110 to collect the user's 3D actions on the computer device 1100. The processor 1101 can implement the following functions based on the data collected by the gyro sensor 1111: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0239] The pressure sensor 1112 can be set on the side frame of the computer device 1100 and / or the lower layer of the touch display screen 1105. When the pressure sensor 1112 is set on the side frame of the computer device 1100, it can detect the user's grip signal of the computer device 1100, and perform left and right hand recognition or shortcut operations according to the grip signal. When the pressure sensor 1112 is set on the lower layer of the touch display screen 1105, it can control the operability controls on the UI interface according to the user's pressure operation on the touch display screen 1105. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0240] The optical sensor 1113 is used to collect the ambient light intensity. In one embodiment, the processor 1101 can control the display brightness of the touch display screen 1105 according to the ambient light intensity collected by the optical sensor 1113. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 1105 is reduced. In another embodiment, the processor 1101 can also dynamically adjust the shooting parameters of the camera assembly 1106 according to the ambient light intensity collected by the optical sensor 1113.

[0241] The proximity sensor 1114, also called a distance sensor, is usually arranged on the front of the computer device 1100. The proximity sensor 1114 is used to collect the distance between the user and the front of the computer device 1100. In one embodiment, when the proximity sensor 1114 detects that the distance between the user and the front of the computer device 1100 is gradually decreasing, the processor 1101 controls the touch display screen 1105 to switch from the screen-on state to the screen-off state; when the proximity sensor 1114 detects that the distance between the user and the front of the computer device 1100 is gradually increasing, the processor 1101 controls the touch display screen 1105 to switch from the screen-off state to the screen-on state.

[0242] Those skilled in the art will understand that Fig.11 The structure shown in the figure does not constitute a limitation on the computer device 1100, and the computer device 1100 may include more or less components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0243] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the frame synchronization method provided by the above-mentioned method embodiments.

[0244] An embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the frame synchronization method provided by the above-mentioned method embodiments.

[0245] An embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; the computer program is read and executed from the computer-readable storage medium by a processor of a computer device, so that the computer device executes to implement the frame synchronization method provided by the above-mentioned method embodiments.

[0246] It will be understood that in the specific implementation of the present application, the data involved, historical data, and data related to user data processing such as portraits related to user identity or characteristics, when the above embodiments of the present application are applied to specific products or technologies, need to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0247] It should be noted that, unless otherwise clearly defined herein, all terms used in the claims are interpreted according to their ordinary meanings in the technical field. Unless otherwise clearly stated, all references to "an element, device, component, equipment, step, etc." will be openly interpreted as referring to at least one instance of an element, device, component, equipment, step, etc. Unless explicitly stated, the steps of any method disclosed herein do not have to be performed in the exact order disclosed.

[0248] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0249] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0250] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent switching, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A frame synchronization method, characterized in that: The method is executed by a client, and includes: Acquire a predicted network delay time, where the predicted network delay time is a predicted network delay time of a data frame, where the network delay time is the time required from when the server broadcasts the data frame until the client receives the data frame, and the data frame includes operation data of at least one client; Based on the predicted network delay time and the remaining display time, updating the display interval of the client, the remaining display time is the playback duration corresponding to at least one cached display frame, the display interval is used to indicate the duration corresponding to playing the cached display frame in a single refresh, and the cached display frame is an image frame obtained after data processing of the received data frame; The cached display frame is played based on the display interval to refresh the game screen.

2. The method according to claim 1, characterized in that The updating of the display interval of the client based on the predicted network delay time and the remaining display time includes: Determining a playback state of the client based on a difference between the predicted network delay time and the remaining display time, wherein the playback state refers to a state when the client plays the cached display frame; The display interval is updated based on the playback status.

3. The method according to claim 2, characterized in that The playing state includes a frame chasing state, and the frame chasing state refers to a state of playing the cached display frame at a double speed; The determining the playback state of the client based on the difference between the predicted network delay time and the remaining display time includes: In response to a difference between the remaining display time and the predicted network delay time being greater than a first time threshold, determining that the state of the client is the frame chasing state; The updating of the display interval based on the playback status comprises: Based on the state parameter of the frame chasing state, the value of the display interval is updated to the product of the basic display interval and the magnification; The basic display interval refers to a preset duration corresponding to the cached display frame played during a single refresh, and the magnification refers to a multiple of the speed at which the cached display frame is played.

4. The method according to claim 3, characterized in that The state parameter includes a frame chasing level; and the method further includes: Determining the frame chasing level based on the difference between the remaining display time and the predicted network delay time; The updating of the value of the display interval to the product of the basic display interval and the magnification based on the state parameter of the frame chasing state includes: Determining the magnification according to the frame chasing level of the frame chasing state; Based on the magnification, the value of the display interval is updated to the product of the basic display interval and the magnification.

5. The method according to claim 3, characterized in that: The method further comprises: When the client is in the frame chasing state and the difference between the remaining display time and the predicted network delay time decreases to less than a second time threshold, switching the client from the frame chasing state to a stable display state; The second time threshold is smaller than the first time threshold, and the stable display state refers to a state in which the cached display frame is played at the basic display interval.

6. The method according to claim 2, characterized in that The playing state includes a stable display state; The determining the playback state of the client based on the difference between the predicted network delay time and the remaining display time includes: In response to a difference between the remaining display time and the predicted network delay time being less than or equal to a first time threshold, determining that the state of the client is the stable display state; The updating of the display interval based on the playback status comprises: Based on the state parameter of the stable display state, the value of the display interval is updated to the value of a basic display interval.

7. The method according to claim 6, characterized in that The method further comprises: When the client is in the stable display state and the difference between the remaining display time and the basic display interval is less than zero, the value of the display interval is updated to the value of the remaining display time.

8. The method according to any one of claims 1 to 7, characterized in that: The obtaining of the predicted network delay time includes: Obtaining a historical network delay record, where the historical network delay record is used to record the network delay time; The predicted network delay time is obtained based on the historical network delay record prediction.

9. The method according to claim 8, characterized in that The predicting and obtaining the predicted network delay time based on the historical network delay record includes: Obtain the mean of the head network delay time and the mean of the tail network delay time from the network delay time series; Subtract the mean value of the tail network delay time from the mean value of the head network delay time to obtain the predicted network delay time; The head network delay time mean refers to the mean of the network delay time arranged at the head in the network delay time sequence, and the tail network delay time mean refers to the mean of the network delay time arranged at the tail in the network delay time sequence.

10. The method according to claim 9, characterized in that The obtaining the mean value of the head network delay time and the mean value of the tail network delay time from the network delay time series includes: Arrange the network delay times recorded in the historical network delay records in descending order to obtain the network delay time series; Extracting the first N network delay times arranged at the head of the network delay time sequence and calculating the average thereof to obtain the average network delay time at the head; The last N network delay times arranged at the tail of the network delay time sequence are extracted and averaged to obtain the tail network delay time average; N is a positive integer.

11. A frame synchronization device, characterized in that: The device comprises: An acquisition module, used to acquire a predicted network delay time, wherein the predicted network delay time is a predicted network delay time of a data frame, wherein the network delay time is the time required from the server broadcasting the data frame until the client receives the data frame, wherein the data frame includes operation data of at least one client; an updating module, configured to update a display interval of the client based on the predicted network delay time and the remaining display time, wherein the remaining display time is the playing time corresponding to at least one cached display frame, and the display interval is used to indicate the playing time corresponding to the cached display frame in a single refresh, and the cached display frame is an image frame obtained after data processing of the received data frame; A refresh module is used to play the cached display frame based on the display interval to refresh the game screen.

12. A computer device, characterized in that: The computer device comprises: a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the frame synchronization method according to any one of claims 1 to 10.

13. A computer storage medium, characterized in that: At least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by the processor to implement the frame synchronization method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product comprises a computer program, which is stored in a computer-readable storage medium; the computer program is read from the computer-readable storage medium and executed by a processor of a computer device, so that the computer device executes the frame synchronization method according to any one of claims 1 to 10.

Images