Game live broadcast synchronization method and its device, equipment, medium, and product

Abstract

The present application relates to a method for synchronizing game live broadcast and its device, equipment, medium and product, the method comprising: obtaining an operation data stream generated by an operator user operating a game in a live broadcast room, the operation data in the operation data stream being marked with its operation timestamp; obtaining a live video stream marked with the operation timestamp pushed by the anchor user in the live broadcast room; loading the demonstration animation of the game according to the operation data stream, and playing the live video stream, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp. The present application can keep the game demonstration animation in the game live broadcast scene and the live video stream displayed synchronously, avoiding the system overhead of the service cluster, and ensuring that the game can be operated collaboratively by multiple users.

Description

Technical Field

[0001] The present application relates to network live broadcast technology, and in particular to a game live broadcast synchronization method and its device, equipment, medium, and product. Background Art

[0002] Live interactive games are usually played in a live broadcast room on the Internet. The room displays the live video stream of the host user on one side and the demonstration animation produced by the operator user operating the game on the other side. The audience users can enjoy the live video stream of the host user and the demonstration animation of the game at the same time in the room. Therefore, the picture synchronization relationship between the live video stream and the demonstration animation is very important.

[0003] Since there are many user devices at the terminal in the live broadcast room, and the network conditions of the game service cluster and the live broadcast service cluster are also relatively complex, this synchronization technology is relatively difficult. In practice, due to the influence of network delay fluctuations, the live video stream and demonstration animation received by the same terminal device are often out of sync, or due to various factors such as network reconnection, the device entering the lock screen state, and the live broadcast program being switched to the background, the game progress recovery is inaccurate, and it is difficult to calibrate the progress with the live video stream, resulting in asynchrony. In addition, due to the accumulation of hardware calculation errors between different terminal devices, the demonstration images obtained by different terminal devices may also be quite different.

[0004] To solve the above problems, the common method in the industry is to synthesize the game demonstration animation with the live video stream, that is, to synthesize the game demonstration animation into the live video stream, so that the audience users in the live broadcast room can get the effect of synchronous playback of the demonstration animation and the video content of the anchor user by just playing the live video stream. The problem brought by this method is that the system consumption of the service cluster is increased due to the need for image synthesis, and it also makes it impossible to decouple the game from the live video stream, unable to liberate the interactive ability of the game, and unable to achieve multiple users as game operators to collaborate to complete the same game at the same time.

[0005] It can be seen that for the game live broadcast scenario, it is necessary to consider both the interactivity of the game operation and the synchronization effect. It is necessary to systematically consider the various problems it faces and extract effective implementation plans. Summary of the invention

[0006] The purpose of this application is to solve the above-mentioned problems and to provide a game live broadcast synchronization method and its corresponding devices, equipment, non-volatile readable storage medium, and computer program product.

[0007] According to one aspect of the present application, a game live broadcast synchronization method is provided, comprising the following steps:

[0008] Acquire an operation data stream generated by an operator user operating a game in a live broadcast room, wherein the operation data in the operation data stream is marked with its operation timestamp;

[0009] Obtaining a live video stream pushed by a host user in the live broadcast room and marked with the operation timestamp;

[0010] The demonstration animation of the game is loaded according to the operation data stream, and the live video stream is played, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp.

[0011] Optionally, before obtaining the operation data stream generated by the operator user operating the game in the live broadcast room, the following steps are included:

[0012] The terminal device of the anchor user in the live broadcast room responds to the operator user operating the game to form a corresponding progress screen and generate a corresponding operation data stream, the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the anchor user;

[0013] The terminal device of the anchor user loads and displays various progress screens of the game according to the operation data in the operation data stream to form a demonstration animation;

[0014] The terminal device of the anchor user marks the image frame synchronized with the image frame with the operator timestamp of the operation data according to the synchronous display relationship between the operation screen corresponding to the operation data and the image frame of the live video stream of the anchor user;

[0015] The terminal device of the anchor user broadcasts the operation data stream to the live broadcast room through the game server, and broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

[0016] Optionally, before obtaining the operation data stream generated by the operator user operating the game in the live broadcast room, the following steps are included:

[0017] The terminal device of the audience user in the live broadcast room generates a corresponding operation data stream in response to the operator user operating the game, the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the audience user;

[0018] The terminal device of the audience user broadcasts the operation data stream to the live broadcast room through the game server;

[0019] The terminal device of the anchor user loads and displays various progress screens of the game according to the operation data in the operation data stream received to form a demonstration animation;

[0020] The terminal device of the anchor user marks the image frame synchronized with the image frame with the operator timestamp of the operation data according to the synchronous display relationship between the operation screen corresponding to the operation data and the image frame of the live video stream of the anchor user;

[0021] The terminal device of the anchor user broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

[0022] Optionally, loading a demonstration animation of the game according to the operation data stream, and playing the live video stream, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp, includes:

[0023] Get the game running time between the current time and the game start time;

[0024] Determine target operation data corresponding to the ongoing screen to be rendered according to the running time of the game, and determine a target operation timestamp corresponding to the target operation data;

[0025] The ongoing picture and the image frame corresponding to the target operation timestamp are synchronously displayed according to the target operation timestamp.

[0026] Optionally, loading a demonstration animation of the game according to the operation data stream includes:

[0027] Determining whether the program instructions of the game include an original random number for determining a game level;

[0028] Replacing the original random number with a pseudo-random number of a corresponding sequence in a standardized pseudo-random number sequence;

[0029] The program instruction is executed according to the pseudo-random number to enter a game level corresponding to the pseudo-random number.

[0030] Optionally, loading a demonstration animation of the game according to the operation data stream includes:

[0031] Determining whether the program instructions of the game contain a parameter whose result value is a floating point number;

[0032] Convert the resulting value of the parameter to an integer or fixed-point number;

[0033] The program instructions are executed according to the converted result values.

[0034] Optionally, after loading the game demonstration animation according to the operation data stream and playing the live video stream, the method further includes:

[0035] Respond to the game live broadcast replay command and determine the replay start time;

[0036] Respectively acquiring the operation data stream and the live video stream whose operation timestamps are after the playback start time;

[0037] The operation data stream and the live video stream are played synchronously according to the corresponding relationship between the operation timestamps.

[0038] According to another aspect of the present application, a game live broadcast synchronization device is provided, comprising:

[0039] An operation data acquisition module is configured to acquire an operation data stream generated by an operator user operating a game in a live broadcast room, wherein the operation data in the operation data stream is marked with an operation timestamp;

[0040] A live video acquisition module, configured to acquire a live video stream annotated with the operation timestamp and pushed by the anchor user in the live broadcast room;

[0041] The synchronous playback processing module is configured to load the demonstration animation of the game according to the operation data stream and play the live video stream so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp.

[0042] According to another aspect of the present application, a game live broadcast synchronization device is provided, including a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the game live broadcast synchronization method described in the present application.

[0043] According to another aspect of the present application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the game live broadcast synchronization method in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the method are executed.

[0044] According to another aspect of the present application, a computer program product is provided, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the steps of the method described in any one of the embodiments of the present application are implemented.

[0045] Compared with the prior art, this application has many technical advantages, including but not limited to:

[0046] On the one hand, the present application provides two data streams in the game live broadcast scenario, one of which is the operation data stream generated by the operator user operating the game, and the other is the live video stream of the anchor user. The image frames in the live video stream are marked with the operation timestamps corresponding to the operation data in the operation data stream. Accordingly, regardless of whether the game operator user and the anchor user are the same user, when displayed in the terminal device, the demonstration animation and the live video stream can be played synchronously according to the operation timestamp, so that the pictures of the demonstration animation and the live video stream are always synchronized, effectively filtering the influence of the source device and network conditions of the operation data stream and the live video stream, and can also avoid various asynchronisms caused by abnormalities or differences in terminal devices.

[0047] On the other hand, the setting of two data streams in the present application effectively decouples the game's operating data from the live video stream, which not only liberates the game's interactive capabilities, allowing the game's operator user to be both the host user and any audience user, allowing multiple audience users to collaborate on the same game task in the live broadcast room, but also saves the performance overhead of the service cluster because there is no need for image synthesis, and has a deployment cost advantage. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A schematic diagram of the network architecture of the application environment of the technical solution of this application;

[0050] Figure 2 A flowchart of an embodiment of a game live broadcast synchronization method of the present application;

[0051] Figure 3 This is a timing flow chart of the implementation framework corresponding to the anchor user as the operator user in this application;

[0052] Figure 4 This is a flow chart corresponding to the host user as an operator user in the embodiment of the present application;

[0053] Figure 5 A timing flow chart of the implementation framework corresponding to the audience user as the operator user in this application;

[0054] Figure 6 This is a flow chart corresponding to the case where the audience user acts as an operator user in the embodiment of the present application;

[0055] Figure 7 This is a schematic diagram of a process for aligning operation data according to game running time in an embodiment of the present application;

[0056] Figure 8 This is a schematic diagram of a progress screen of an exemplary "Pixel Bird" game of the present application;

[0057] Fig. 9 A schematic diagram of a flow chart of processing random numbers in game program instructions in an embodiment of the present application;

[0058] Fig.10 A schematic diagram of a flow chart of processing floating point numbers in game program instructions in an embodiment of the present application;

[0059] Fig.11 This is a flowchart of replaying the game process in an embodiment of the present application;

[0060] Fig.12 A schematic diagram of a progress screen of an exemplary flying knife theme game of the present application;

[0061] Fig.13 This is a principle block diagram of the game live broadcast synchronization device of the present application;

[0062] Fig.14 This is a structural diagram of a game live broadcast synchronization device used in this application. DETAILED DESCRIPTION

[0063] See also Figure 1 The network architecture adopted by an exemplary application scenario of the present application includes a terminal device 80, a media server 81, an application server 82 and a game server 83. The application server 82 can be used to deploy a live game service. The media server 81 can be used to process tasks such as optimization and forwarding of live video streams for the live game service. The game server 83 can be used to deploy game products required to be called by the live game service. The terminal device 80 can be used by the anchor user and the audience user in the live broadcast room of the live game service. Any one or more of the anchor user and the audience user can be allowed to participate in the game activities provided by the game product as needed, thereby becoming the operator user of the corresponding game.

[0064] Under the coordination of the application server 82, the game live broadcast activity in the live broadcast room can be held normally. When the game live broadcast activity in the live broadcast room is held, the game server 83 can be responsible for broadcasting the operation data stream generated by the operator user operating the game on his terminal device to the live broadcast room, and the media server 81 can be responsible for broadcasting the live video stream of the anchor user to the live broadcast room. The operation data stream and the live video stream are both based on the operation timestamp corresponding to the moment when the operator user operates the game. When the operation data stream and the live video stream reach the terminal device 80 of any user in the live broadcast room, especially the audience user, the terminal device controls the game demonstration animation and the live video stream to keep synchronous display according to the operation timestamp of the operator user operating the game.

[0065] Based on the principles disclosed above, please refer to Figure 2 According to a game live broadcast synchronization method provided by the present application, in one embodiment thereof, the following steps are included:

[0066] Step S1100, obtaining an operation data stream generated by an operator user operating a game in a live broadcast room, wherein the operation data in the operation data stream is marked with its operation timestamp;

[0067] As mentioned above, the game server is responsible for broadcasting the operation data stream corresponding to the operation of the game by the operator user to the live broadcast room, and this operation data stream will be broadcast to the terminal devices of each audience user.

[0068] For any audience user, the operation data stream they receive is generally delivered in the form of data frames. The data encapsulated in each data frame is the operation data generated by the operator user operating the game in a unit time slot. Therefore, when the operator user's terminal device needs to submit the data frame to the cloud server, the operation timestamp corresponding to the unit time slot will be determined for the operation data in the data frame according to the corresponding unit time slot, and then the operation data of the unit time slot will be marked with its corresponding operation timestamp. In this way, the operation data of each unit time slot and its corresponding operation timestamp are encapsulated into a data frame together, which is broadcasted to the live broadcast room through the cloud server and received by the audience user's terminal device.

[0069] The operation data generally includes data corresponding to various input events when the operator plays the game, including but not limited to keyboard touch events, mouse touch events, screen touch events, voice input events, etc. Generally speaking, the operation data mainly focuses on any business data that will cause the progress of the game to change, which can be determined according to the different business logics of different games, and does not affect the embodiment of the creative spirit of this application. In some embodiments, to meet various other business needs, other business-related data can also be carried in the data frame of the operation data stream, and similarly, it can be flexibly processed as needed.

[0070] The unit time slot of the data frame in the operation data stream can be flexibly set according to actual business needs. For example, it can be set to a specific value of microseconds or milliseconds as needed. The smaller the granularity of this unit time slot, the higher the refresh speed of the screen generated based on the operation data can be.

[0071] In one embodiment, the unit time slot corresponding to each data frame can be determined according to the refresh frequency of the game screen of the operator user's terminal device. For example, when the refresh frequency is 120 Hz, the unit time slot can be determined as 1 / 120 = 17 milliseconds, so that when the audience user side displays the game progress screen according to the operation data, the effect can remain basically consistent with that of the operator user side.

[0072] In another embodiment, in order to save network bandwidth, the unit time slot can also be set according to a standard recognizable by the human eye. The standard for a smooth picture recognizable by the human eye is a minimum of 24 frames, which can be relaxed to 30 frames. Based on this, the unit time slot can be determined as 1 / 24=33 milliseconds.

[0073] That is to say, every time the unit time slot is reached, a corresponding operation timestamp is generated in the terminal device of the operator user, triggering an operation data submission event, and the operation data generated corresponding to the unit time slot and the operation timestamp are encapsulated into a data frame in the operation data stream, which is sent to the game server, and the game server broadcasts it to the terminal device of the audience user in the live broadcast room to be received. As a result, the terminal device of the audience user can obtain the operation data stream generated by the operator user operating the game, which can be used to restore the progress screen of the game.

[0074] Step S1200: obtaining a live video stream annotated with the operation timestamp pushed by the anchor user in the live broadcast room;

[0075] The operator user of the game can be either the anchor user or any audience user. When the operator user and the anchor user are the same user, the anchor user's terminal device is responsible for pushing the game's operation data stream and the anchor user's live video stream at the same time. Therefore, the operation timestamp of the operation data in the operation data stream can be used to quickly mark the corresponding image frame of the live video stream, so that the operation data stream and the live video frame can establish a synchronization relationship with respect to the operation timestamp. When the operator user is not the anchor user but a certain audience user, the operation data stream and the live video stream are generated in different terminal devices. In this case, the operation data stream and the live video stream are independently sent to the game server for broadcasting. In order to establish a synchronization relationship between the operation data stream and the live video stream with respect to the operation timestamp, after the anchor user's terminal device receives the operation data stream pushed by the game server, the corresponding progress screen of the game can be loaded according to the operation data in the operation data stream, and then according to the synchronization relationship between the progress screen and the image frame of the live video stream on display, the operation timestamp of the operation data is used to mark the image frame synchronized with it, and similarly, the operation data stream and the live video frame can establish a synchronization relationship with respect to the operation timestamp.

[0076] The live video stream marked with the operation timestamp is pushed by the host user's terminal device to the media server used by the game live broadcast service, and then broadcast to the entire live broadcast room by the media server. As a result, the audience user's terminal device can obtain the live video stream and decode and play it.

[0077] In one embodiment, the operation timestamp used by the image frame in the live video stream is pre-set in the live video stream by the anchor user's terminal device as the supplementary enhancement information (SEE) of the live video stream. When it reaches the audience user's terminal device, the data packet of the live video stream is decoded, and after the identification of the existence of the supplementary enhancement information is identified, the corresponding operation timestamp can be read according to the identification. There are many video encoding protocols that allow the addition of supplementary enhancement information. For example, H.264 and its later protocol versions all support the addition of supplementary enhancement information to the video stream and can be adopted.

[0078] Step S1300, loading the demonstration animation of the game according to the operation data stream, and playing the live video stream, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp.

[0079] After receiving the operation data stream of the game broadcasted by the game server, the audience user can restore the corresponding game screen according to the operation data in the operation data stream, and use the corresponding operation data to substitute into the business logic of the game to obtain the corresponding progress screen. The operation data used in reconstructing the progress screen is called in order according to its operation timestamp, so that the demonstration animation composed of each progress screen is consistent with the operation process of the operator user, and the operation and demonstration have the same screen process.

[0080] When the audience user receives the live video stream of the anchor user broadcasted by the media server, it can be decoded and played. Since the live video stream is marked with an operation timestamp, in order to keep the playback of the live video stream synchronized with the progress of the game demonstration animation, when rendering and loading the corresponding progress screen according to a sequence of operation timestamps, the image frame carrying the operation timestamp in the live video stream can be rendered and loaded synchronously, so that the progress screen and image frame corresponding to the same operation timestamp are displayed synchronously, and so on, so that the game demonstration animation and the live video stream maintain a synchronized playback effect, and this synchronized playback effect is consistent with the playback effect in the anchor user's terminal device.

[0081] In one embodiment, the game server preferentially broadcasts the operation data corresponding to a certain operation timestamp to the live broadcast room, and the terminal device of the anchor user will mark the corresponding image frame according to the operation timestamp and then push the image frame to the live broadcast room, so that the terminal device of the audience user first receives the operation data corresponding to the operation timestamp, but needs to wait for the image frame corresponding to the operation timestamp. In this regard, the terminal device of the audience user can establish a buffer area to cache the operation data that arrives first, wait for the image frame corresponding to the operation timestamp of the cached operation data to arrive, and then, according to the corresponding relationship of the operation timestamp, synchronously control the display of the progress screen and image frame carrying the same operation timestamp. It can be seen that by caching the operation data in the buffer area, an anti-jitter mechanism can be established to adapt to the situation where the operator user and the anchor user are not in the same terminal device, resulting in abnormal transmission of the operation data stream and the live video stream, so that the operation data stream and the live video stream arrive asynchronously at the terminal device of the same audience user, and can still be output synchronously, so that the corresponding game progress screen and the live video stream image frame maintain a synchronous display relationship.

[0082] It is not difficult to understand that since the game progress screen is reconstructed based on the operation data generated by the operator user, rather than transmitting the screen after the operator user operates the game in the form of a video stream, the present application realizes the decoupling of the game's interactive capabilities. The terminal device of each audience user can load the source program of the game and apply the corresponding operation data to drive its process, thereby obtaining the corresponding progress screen at each moment. In this case, the audience user of the current terminal device can also be one of the operator users. Therefore, in the live broadcast room of the game, multi-user collaborative operation is allowed based on the game, so that the game remains a multi-user collaborative game in the live broadcast room.

[0083] According to the above embodiments, the present application has many technical advantages, including but not limited to:

[0084] On the one hand, the present application provides two data streams in the game live broadcast scenario, one of which is the operation data stream generated by the operator user operating the game, and the other is the live video stream of the anchor user. The image frames in the live video stream are marked with the operation timestamps corresponding to the operation data in the operation data stream. Accordingly, regardless of whether the game operator user and the anchor user are the same user, when displayed in the terminal device, the demonstration animation and the live video stream can be played synchronously according to the operation timestamp, so that the pictures of the demonstration animation and the live video stream are always synchronized, effectively filtering the influence of the source device and network conditions of the operation data stream and the live video stream, and can also avoid various asynchronisms caused by abnormalities or differences in terminal devices.

[0085] On the other hand, the setting of two data streams in the present application effectively decouples the game's operating data from the live video stream, which not only liberates the game's interactive capabilities, allowing the game's operator user to be both the host user and any audience user, allowing multiple audience users to collaborate on the same game task in the live broadcast room, but also saves the performance overhead of the service cluster because there is no need for image synthesis, and has a deployment cost advantage.

[0086] Based on any embodiment of this application, please refer to Figure 3 The timing flow chart shown and Figure 4 The flowchart shown in the figure includes the following steps before obtaining the operation data stream generated by the operator user operating the game in the live broadcast room:

[0087] Step S2100: The terminal device of the anchor user in the live broadcast room responds to the operator user operating the game to form a corresponding progress screen and generate a corresponding operation data stream, wherein the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the anchor user;

[0088] In this embodiment, the anchor user is also an operator user, so the operation data stream and the live video stream will be generated in the terminal device of the anchor user. In the process of the anchor user, that is, the operator user, playing the game, each operation of the game will generate corresponding operation data, and the game will play the progress screen according to its inherent business logic, thereby forming a demonstration animation. When the operator user operates the game, corresponding operation data will be generated, and the operation data will further cause the game progress screen to change, thereby promoting the progress of the entire demonstration animation. Since the operation data is encapsulated according to the preset unit time slot, the operation timestamp corresponding to the unit time slot is used to mark the operation data.

[0089] As mentioned above, during the advancement of the entire game, the operation data corresponding to each unit time slot is encapsulated into a data frame, and a corresponding operation timestamp is generated to mark the operation data, thereby forming an operation data stream.

[0090] Step S2200: The terminal device of the anchor user marks the image frames synchronized with the image frames of the anchor user's live video stream according to the synchronous display relationship between the operation screen corresponding to the operation data and the image frames of the anchor user's live video stream with the operator timestamp of the operation data;

[0091] While the host user plays the game as an operator and plays the corresponding progress screen on the display of his terminal device, the host user's camera unit also continuously records the environment image, thereby playing each image frame. Therefore, for the terminal device, the synchronization relationship between the operation data and the image frame can be established based on the temporal correspondence between the operation timestamp and the image frame in the live video stream, that is, the temporal correspondence between the corresponding operation data and the image frame. Specifically, according to the time correspondence rendered to the display, the operation timestamp of the operation data used by the progress screen with the corresponding relationship can be used as the supplementary enhancement information of the corresponding image frame to mark the image frame, thereby establishing a synchronization association between the operation data and the image frame. The progress screen of the operation data with a synchronous association relationship and the image frame of the live video stream are displayed synchronously in the display effect of the display.

[0092] Step S2300: The terminal device of the anchor user broadcasts the operation data stream to the live broadcast room through the game server, and broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

[0093] In order to provide the operation data stream and live video stream to each audience user in the live broadcast room, the terminal device of the anchor user can, on the one hand, continuously push the operation data stream composed of the operation data to the game server, and the game server broadcasts the operation data stream to the live broadcast room of the anchor user to be received and processed by the terminal devices of each audience user; on the other hand, continuously push the live video stream composed of the image frames marked with operation timestamps to the media server, and the media server broadcasts the live video stream to the live broadcast room of the anchor user to be received and processed by the terminal devices of each audience user. In this way, each audience user can receive two data streams, namely the operation data stream and the live video stream, and can synchronously play the game progress screen and the image frames of the live video stream according to the operation timestamps carried therein.

[0094] According to the above embodiments, when the host user plays the game as an operator user, the operator data generated by his operation of the game and his live video stream can be processed in the same terminal device with a relatively small time difference. When the two are broadcast to the live broadcast room and sent to the terminal devices of each audience user for loading and display, each audience user can synchronously play the game's demonstration animation and live video stream based on the operation timestamp, so that the progress screen of the demonstration animation and the image frame of the live video stream can be displayed strictly synchronously, thereby obtaining the same display effect as the host user's terminal device.

[0095] Based on any embodiment of this application, please refer to Figure 5 The timing flow shown and Figure 6 The flowchart shown in the figure includes the following steps before obtaining the operation data stream generated by the operator user operating the game in the live broadcast room:

[0096] Step S3100: The terminal device of the audience user in the live broadcast room generates a corresponding operation data stream in response to the operator user operating the game, the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the audience user;

[0097] In this embodiment, the operator user is an audience user in the live broadcast room rather than the host user of the live broadcast room. Therefore, the operator user and the host user use different terminal devices respectively. The operator user's terminal device is responsible for processing the operation data stream generated by the operator user operating the game, and the host user's terminal device is responsible for the live video stream generated by the host user's camera unit. The two have different divisions of labor.

[0098] Similarly, when the audience user, that is, the operator user, plays the game, each operation of the game will generate corresponding operation data, and the game plays the progress screen according to its inherent business logic, thereby forming a demonstration animation. When the operator user operates the game, corresponding operation data will be generated, and the operation data will further cause the game progress screen to change, thereby advancing the progress of the entire demonstration animation. Since the operation data is encapsulated according to the preset unit time slot, the operation timestamp corresponding to the unit time slot is used to mark the operation data.

[0099] Step S3200: The terminal device of the audience user broadcasts the operation data stream to the live broadcast room through the game server;

[0100] Since the operation data stream is generated by the terminal device of the audience user who is the operator user, the terminal device of the audience user is responsible for pushing the operation data stream to the game server, and then the game server broadcasts the received operation data stream to the live broadcast room. It is not difficult to understand that all users in the live broadcast room, including each audience user and the anchor user, can receive the operation data stream.

[0101] Step S3300: The terminal device of the anchor user loads and displays various progress screens of the game according to the operation data in the operation data stream received to form a demonstration animation;

[0102] For the terminal device of the anchor user, after it receives the operation data stream pushed by the game server, it can incorporate the operation data in each data frame in the operation data stream into the inherent business logic of the game to produce an effect equivalent to operating the game locally according to the operation data. Specifically, for each operation data loaded in the order of operation timestamps, the terminal device of the anchor user uses the operation data to apply it to the business logic of the game to realize the loading of the operation data, drive the progress of the game, and advance the game to its progress screen, so as to form an output effect of a demonstration animation on the display of the terminal device.

[0103] Step S3400: The terminal device of the anchor user marks the image frames synchronized with the image frames of the anchor user's live video stream according to the synchronous display relationship between the operation screen corresponding to the operation data and the image frames of the anchor user's live video stream with the operator timestamp of the operation data;

[0104] It is not difficult to understand that, since the terminal device of the anchor user synchronously processes the display of the live video stream of the anchor user, the anchor user's display not only displays the playback screen of the live video stream, but also displays the demonstration animation of the game. Therefore, from the display effect, there is a time synchronization display relationship between the progress screen in the demonstration animation of the game and the image frame in the live video stream. This synchronization display relationship is manifested in the background processing as the consistency between the progress screen and the image frame in the playback time. According to this consistency, the operation timestamp carried by the operation data forming the progress screen can be used as supplementary enhancement information to mark the image frame with which it has a synchronous display relationship. In this way, each image frame in the live video stream establishes an associated correspondence relationship of the operation timestamp with each operation data in the operation data stream. By using this associated correspondence relationship, the synchronous display between the two can be realized later.

[0105] Step S3500: The terminal device of the anchor user broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

[0106] While the host user's terminal device is processing the loading and display of the operation data stream, it is also processing the live video stream, which needs to be pushed to the media server so as to be delivered to each viewer user. In this regard, the live video stream pushed by the host user's terminal device is a live video stream in which the image frames are marked with the operation timestamp. The live video stream marked with the operation timestamp is broadcast to the host user's live broadcast room through the media server, which can ensure that each viewer user can obtain two data streams, the operation data stream and the live video stream, so that the viewer user can synchronize the game screen and the image frames of the live video stream according to the operation timestamp carried by them.

[0107] According to the above embodiments, it can be known that in the case where the operator user is not the anchor user, the operation data stream of the game and the live video stream of the anchor user are generated asynchronously, but are synchronously controlled and processed at the same time in the terminal device of the anchor user. The terminal device uses the operation timestamp of the operation data to mark the image frame that has a synchronous display relationship with the ongoing screen of the operation data, so that the live video stream and the demonstration animation of the game establish synchronization information based on the operation timestamp, which can ensure that after the two data streams reach the terminal device of the audience user, the terminal device of the audience user can synchronously play the live video stream and the demonstration animation according to the operation timestamp to obtain the same display effect as the terminal device of the anchor user.

[0108] Based on any embodiment of this application, please refer to Figure 7, loading the demonstration animation of the game according to the operation data stream, and playing the live video stream, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp, including:

[0109] Step S1311, obtaining the game running time between the current time and the game start time;

[0110] When it is necessary to control the progress of the game according to the operation data in the operation data stream, so as to display the corresponding progress screen to advance the demonstration animation of the game, according to the inherent business logic of the game, it is necessary to perform logical operations with the operation timestamp of the operation data to determine the progress node of the game. For this purpose, the game running time can be determined first. The game running time can be determined by the difference between the current time and the game start time. Therefore, for each operation data, each time the operation timestamp of the operation data is applied, it is compared with the game start time and determined according to the game running time determined after the comparison. Therefore, the application of each operation data can be determined based on the game running time, without the help of a timer, avoiding the generation of cumulative calculation errors, making the time calculation result more accurate and the game progress more accurate.

[0111] Step S1312: determining target operation data corresponding to the ongoing screen to be rendered according to the game running time, and determining a target operation timestamp corresponding to the target operation data;

[0112] Based on the game running time, the operation data corresponding to the next ongoing screen to be rendered, i.e., the target operation data, can be determined, and the operation timestamp carried by the target operation data is also the target operation timestamp. It is not difficult to understand that if the operation data stream reaches the current terminal device with a delay, according to the algorithm of this embodiment, part of the operation data that times out due to the delay will be ignored, but the next ongoing screen rendered and displayed can maintain the greatest degree of synchronization with the actual progress of the game.

[0113] Step S1313: Synchronously displaying the progress picture and image frame corresponding to the target operation timestamp according to the target operation timestamp.

[0114] After the target operation timestamp is determined, on the one hand, the operation data corresponding to the target operation timestamp is applied to the game to drive it to display the corresponding progress screen, and on the other hand, the image frame carrying the target operation timestamp in the live video stream is synchronously rendered and displayed, thereby ensuring that the progress screen and the image frame are played and displayed synchronously with respect to the target operation timestamp, and so on, the demonstration animation of the game is played and displayed synchronously with the live video stream with respect to the operation timestamp.

[0115] According to the above embodiments, the following combination Figure 8 The "Pixel Bird" game corresponding to the game interface shown in the figure is used to formally illustrate the application principle of operating data:

[0116] Gameplay analysis: The player controls a bird and keeps it from colliding with the ground and randomly generated pillars. The bird's X-axis coordinate is fixed, and the pillar moves in the X-axis direction, making the bird appear to be flying forward. The pillar is divided into two halves, with a gap in the middle, and the bird can fly through it.

[0117] Assume the current game configuration: the screen height is 150. The column width is 10. The initial position X of the first column is 110, the Y range of the gap is 50-80, and it moves 10 pixels to the left per second. The initial position of the bird is X: 20, Y: 50, and the Y axis speed falls 5 pixels per second. After clicking the screen, the Y axis speed of the bird becomes 5 pixels per second and lasts for 1 second.

[0118] If the game has been running for 10 seconds, and the player clicked the screen at the 3rd and 8th seconds, the current game state can be calculated:

[0119] Column X-axis position: 110-10*10=10

[0120] Bird X-axis position: The bird only moves in the Y-axis direction, and the X-axis position remains unchanged at 20

[0121] Bird Y-axis position:

[0122] 1.0-3 seconds, the bird falls, calculate the position at the 3rd second: 50+(3*-5)=35

[0123] 2.3-4 seconds, the bird rises, calculate the position at the 4th second: 35+(1*5)=40

[0124] 3.4-8 seconds, the bird falls, calculate the position at the 8th second: 40+(4*-5)=20

[0125] 4.8-9 seconds, the bird rises, calculate the position at 9 seconds: 20+(1*5)=25

[0126] 5.9-10 seconds, the bird falls, calculate the position at 10 seconds: 25+(1*-5)=20

[0127] So the bird's current Y-axis position is 20;

[0128] Based on the bird's position, the column's position, width, and the gap range, we can calculate whether the bird collides with the column:

[0129] The coordinates of the four points of the column gap rectangle are: (5,50), (5,80), (15,80), (15,50)

[0130] The coordinates of the four points in the rectangle of the lower half of the column are: (5,0), (5,50), (15,50), (15,0)

[0131] Coordinates of the four points in the upper rectangle of the column: (5,80), (5,150), (15,150), (15,80)

[0132] The bird's coordinates are: (20,20),

[0133] Therefore, the bird's coordinates do not collide with the upper and lower parts of the pillar, and the game can continue to advance.

[0134] According to the above embodiments and their vivid examples, it can be known that the target operation timestamp that needs to be called is determined based on the game running time, and then the progress screen and image frame corresponding to the target operation timestamp are synchronously displayed, so that the game demonstration animation and live video stream are always synchronized to the latest status, and no cumulative error will be generated, thereby ensuring a high-quality synchronous playback effect, and allowing the audience user to keep the latest playback status synchronized as much as possible with the operator user and the anchor user.

[0135] In the inherent business logic of the game, there are often parameters for switching game levels. Since the game designer wants the game level to be randomly determined, random numbers are often used, which are called original random numbers. Original random numbers can be understood as true random numbers, which are usually generated uniformly by the game server and then sent to each terminal device loaded with the game. In this case, the original random numbers obtained by the operator user and other non-operator users are different, which causes different terminal devices to enter different game levels, resulting in the inability to synchronize the game.

[0136] For the above problems, based on any embodiment of this application, please refer to Fig. 9 , loading the demonstration animation of the game according to the operation data stream, including:

[0137] Step S1321, determining whether the program instructions of the game include an original random number for determining a game level;

[0138] In order to identify the true random numbers in the inherent business logic of the game, a monitoring module can be preset in the live broadcast application to detect the program instructions of the loaded game to determine whether the program instructions to be executed contain the original random numbers used to determine the game levels. When the original random numbers do not exist, the inherent business logic of the game can continue. When the original random numbers exist, the program instructions need to be specially processed.

[0139] Step S1322: replacing the original random number with a pseudo-random number of a corresponding sequence in a standardized pseudo-random number sequence;

[0140] The principle of special processing of the program instructions is to replace the original random number with a determined pseudo-random number. To this end, a standardized pseudo-random number sequence can be pre-set and pre-stored in the terminal devices of each user in the live broadcast room. When the original random number needs to be replaced, the pseudo-random number of the corresponding sequence is called in sequence to replace the original random number, so that the random number based on which the program instructions are executed changes, so that the terminal devices of each user can obtain the same random number for each original random number and switch to the same game level, ensuring the steady progress of synchronous control.

[0141] In one embodiment, when the operator user starts the game for the first time, the game server can give a random number seed and synchronize the random number seed to the live broadcast room program of the terminal device of each user in the live broadcast room. Each terminal device generates a corresponding pseudo-random number sequence locally according to a preset algorithm. Since the algorithms are consistent, the pseudo-random number sequences of each terminal device are also consistent. When switching game levels during the game advancement process, the same pseudo-random number can be obtained according to the same pseudo-random number sequence. Using the same pseudo-random number, each terminal device can enter the same game level.

[0142] The algorithm for generating the pseudo-random number sequence may use a linear congruential method or a square median method, or any other feasible method.

[0143] Step S1323: Execute the program instruction according to the pseudo-random number to enter the game level corresponding to the pseudo-random number.

[0144] After the original random number is replaced by the pseudo-random number, the program instruction can be executed, and the program instruction uses the pseudo-random number to switch the game to a corresponding game level, thereby loading and displaying the corresponding progress screen of the game level.

[0145] According to the above embodiments, in order to ensure that the game has the necessary certainty when switching game levels according to random numbers in terminal devices of different users, so as to avoid different users entering different game levels and make the game level entered by each user consistent with the game level entered by the operator user, a standardized pseudo-random number sequence is used to replace the original random numbers of the game program instructions accordingly, thereby ensuring that the games running in the terminal devices of different users can switch to the same game level based on the same pseudo-random number when advancing under the drive of the same operation data stream, and ensuring that the game advancement logic is also synchronized in the terminal devices of different users.

[0146] In the inherent business logic of the game, various floating-point operations are often carried out. The reason for the generation of these floating-point numbers may be obtained after the operation of different data, or it may be obtained by running various functions such as trigonometric functions, and finally the result value represented by floating-point numbers will be obtained. The existence of these floating-point numbers, due to the different hardware environments and system platforms of different terminal devices, leads to different floating-point calculation results, and then the error will be magnified in the cumulative calculation process, resulting in inconsistent results in the final presentation, thus causing various differences between the animations and even the demonstration animations of different terminal devices.

[0147] For the above problems, based on any embodiment of this application, please refer to Fig.10 , loading the demonstration animation of the game according to the operation data stream, including:

[0148] Step S1331, determining whether the program instructions of the game contain a parameter whose result value is a floating point number;

[0149] In order to identify the parameters in the inherent business logic of the game that may cause the result value to be a floating point number, a monitoring module may be preset in the live broadcast application to detect the program instructions of the loaded game, determine whether the result value after the execution of the corresponding program instructions is a floating point number, and when a floating point number may be obtained, determine to intervene in the program instructions. When the result value is expected to be a non-floating point number, no processing is required.

[0150] Step S1332, converting the result value of the parameter into an integer or a fixed-point number;

[0151] For program instructions that will obtain result values ​​represented by floating-point numbers after execution, they can be converted to ensure that the result values ​​obtained can be represented in the form of integers or fixed-point numbers. During specific processing, different situations can be distinguished and responded to flexibly.

[0152] In one embodiment, when the result value is a value obtained according to a trigonometric function, the corresponding result value can be determined by looking up a table so that the result value is a fixed-point number. To this end, a table of mapping relationships between angles and values ​​of a trigonometric function is determined in advance, and in this expression, the values ​​corresponding to each angle are expressed as fixed-point numbers. When the angle information of a trigonometric function is given in a program instruction, the table can be looked up to determine the fixed-point number corresponding to the angle, thereby completing the expression of the result value of the trigonometric function in a fixed-point number.

[0153] In another embodiment, the result value obtained after the execution of the program instruction is multiplied by a larger integer, such as 1000, and then truncated, for example, all decimals are truncated to retain as integers, or a fixed number of digits after the decimal point are taken to become a fixed-point number, thereby converting the result value into an integer or fixed-point number representation.

[0154] Similarly, for the result values ​​of trigonometric functions, even if a table query is not used, truncation can be achieved by multiplying by a larger integer to obtain the result value in the form of an integer or fixed-point number.

[0155] Step S1333, execute the program instruction according to the converted result value.

[0156] After the conversion and representation of the result value in the program instruction is completed, the program instruction is executed, thereby avoiding the differences in floating-point operations between different terminal devices.

[0157] According to the above embodiments, by converting the result values ​​of floating-point numbers in the game program instructions into integers or fixed-point numbers for calculation, the calculation differences between the hardware and operating systems of different terminal devices can be eliminated, so that different terminal devices can always maintain a high degree of consistency in the process of using operation data streams to drive the progress of the same game, thereby ensuring the synchronization of game content. In the game live broadcast scenario, the risk of the same game being out of sync due to differences in software and hardware of different terminal devices is eliminated.

[0158] Based on any embodiment of this application, please refer to Fig.11 , after loading the game demonstration animation according to the operation data stream and playing the live video stream, it includes:

[0159] Step S4100, responding to the game live broadcast playback instruction and determining the playback start time;

[0160] Any user in the live broadcast room, such as the audience user of this application, including the anchor user and the audience user, can replay the historical data of the game through the playback control, so as to review the synchronized screen of any time node during the game progress. When the user drags the playback control to the moment indicated by a certain position, the game live broadcast playback command is triggered. In response to the command, the live broadcast room application can determine the operation timestamp corresponding to the moment as the playback start time.

[0161] Step S4200, respectively obtaining the operation data stream and the live video stream whose operation timestamps are after the playback start time;

[0162] The live broadcast room application then submits corresponding playback requests to the game server and the media server respectively, and the playback request carries the operation timestamp specified by the playback start time. The game server pushes the operation data stream after the operation timestamp to the terminal device of any user according to the operation timestamp. Similarly, the media server pushes the operation data stream after the operation timestamp to the terminal device of any user according to the operation timestamp, so that the terminal device of any user can obtain the operation data stream and live video stream after the playback start time.

[0163] Step S4300: Synchronously play the operation data stream and the live video stream according to the corresponding relationship between the operation timestamps.

[0164] After obtaining the operation data stream and live video stream after the playback start time, the operation data stream and live video stream with the operation timestamp after the playback start time can be synchronously played according to the synchronization relationship of the operation timestamps in accordance with the method disclosed in the previous text of this application, thereby realizing playback.

[0165] According to the above embodiments, by specifying the playback start time, the operation data stream and the live video stream with the operation timestamp after the playback start time can be obtained for synchronous playback, so that different terminal devices in the live broadcast room can abnormally replay the game process without interfering with each other, but the demonstration animation corresponding to the operation data stream in each terminal device is always synchronized with the screen of the live video stream.

[0166] The game live broadcast synchronization method of the present application can be applied to the game live broadcast scene to obtain better user experience. Fig.12 As shown, a flying knife-themed game is activated by the operator user in the live broadcast room. The current live broadcast screen is at the lower right corner of the graphical user interface. The anchor user or the audience user can take a seat and operate the game as the operator user. The anchor user can give a real-time commentary on the game operation. Any other user in the live broadcast room can watch the game process of the operator user and can simultaneously watch the live screen presented by the anchor user's live video stream.

[0167] In the game screen, there is a target that is constantly rotating. The operator can click the screen to launch a throwing knife to the circular target above, and it will be inserted into the target and rotate with the target. The target rotates in the opposite direction at a certain frequency, interfering with the operator. The remaining throwing knives are displayed on the left side of the target. If all the throwing knives are inserted into the target, the level is passed. If the throwing knife collides with other throwing knives on the target, the level is failed.

[0168] The game adopts the method of the present application, calibrates the game progress and the live broadcast progress through SEI transmission operation timestamp, realizes game logic operation with the idea of ​​time advancement, calculates the angle of the game target and the position of the flying knife, realizes the video playback function for progress calibration, and uses pseudo-random and fixed-point operations to ensure the operation error between different devices, thus providing an excellent user experience.

[0169] See also Fig.13 According to one aspect of the present application, a game live broadcast synchronization device is provided, including an operation data acquisition module 1100, a live video acquisition module 1200, and a synchronous playback processing module 1300, wherein: the operation data acquisition module 1100 is configured to obtain an operation data stream generated by an operator user operating a game in a live broadcast room, and the operation data in the operation data stream is marked with its operation timestamp; the live video acquisition module 1200 is configured to obtain a live video stream marked with the operation timestamp pushed by the anchor user in the live broadcast room; the synchronous playback processing module 1300 is configured to load a demonstration animation of the game according to the operation data stream, and play the live video stream, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp.

[0170] On the basis of any embodiment of the present application, the game live broadcast synchronization device of the present application includes: an anchor operation unit, which is configured so that the terminal device of the anchor user in the live broadcast room responds to the operator user operating the game to form a corresponding progress screen and generate a corresponding operation data stream, the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the anchor user; an anchor synchronization unit, which is configured so that the terminal device of the anchor user marks the image frames synchronized with it with the operator timestamp of the operation data according to the synchronous display relationship between the progress screen corresponding to the operation data and the image frames of the live broadcast video stream of the anchor user; an anchor push unit, which is configured so that the terminal device of the anchor user broadcasts the operation data stream to the live broadcast room through the game server, and broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

[0171] On the basis of any embodiment of the present application, before obtaining the operation data stream generated by the operator user operating the game in the live broadcast room, it includes: an audience operation unit, which is configured so that the terminal device of the audience user in the live broadcast room responds to the operator user operating the game to generate a corresponding operation data stream, the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the audience user; a game push unit, which is configured so that the terminal device of the audience user broadcasts the operation data stream to the live broadcast room through a game server; an anchor loading unit, which is configured so that the terminal device of the anchor user loads and displays various progress screens of the game according to the operation data in the operation data stream received by it to form a demonstration animation; an anchor synchronization unit, which is configured so that the terminal device of the anchor user marks the image frames synchronized with it with the operator timestamp of the operation data according to the synchronous display relationship between the progress screen corresponding to the operation data and the image frames of the live broadcast video stream of the anchor user; and an anchor push unit, which is configured so that the terminal device of the anchor user broadcasts the live video stream marked with the operation timestamp to the live broadcast room through a media server.

[0172] Based on any embodiment of the present application, the synchronous playback processing module 1300 includes: a duration determination unit, configured to obtain the game running duration between the current time and the game start time; a target determination unit, configured to determine the target operation data corresponding to the ongoing screen to be rendered according to the game running duration, and determine the target operation timestamp corresponding to the target operation data; a synchronization coordination unit, configured to synchronously display the ongoing screen and image frame corresponding to the target operation timestamp according to the target operation timestamp.

[0173] Based on any embodiment of the present application, the synchronous playback processing module 1300 includes: a random number identification unit, configured to determine whether the program instructions of the game contain an original random number for determining a game level; a random number replacement unit, configured to replace the original random number with a pseudo-random number of a corresponding sequence in a standardized pseudo-random number sequence; and a random number application unit, configured to execute the program instructions according to the pseudo-random number to enter the game level corresponding to the pseudo-random number.

[0174] Based on any embodiment of the present application, the synchronous playback processing module 1300 includes: a floating-point number recognition unit, configured to determine whether the program instructions of the game contain parameters whose result values ​​are floating-point numbers; a floating-point number replacement unit, configured to convert the result values ​​of the parameters into integers or fixed-point numbers; and a floating-point number application unit, configured to execute the program instructions according to the converted result values.

[0175] Based on any embodiment of the present application, the game live broadcast synchronization device of the present application includes: a playback response module, which is configured to respond to the game live broadcast playback instruction and determine the playback start time; a playback loading module, which is configured to respectively obtain the operation data stream and the live video stream with an operation timestamp after the playback start time; and a playback playing module, which is configured to synchronously play the operation data stream and the live video stream according to the corresponding relationship of the operation timestamps.

[0176] Another embodiment of the present application also provides a game live broadcast synchronization device. Fig.14 As shown, a schematic diagram of the internal structure of a game live synchronization device. The game live synchronization device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable non-volatile readable storage medium of the game live synchronization device stores an operating system, a database, and computer-readable instructions. The database may store an information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a game live synchronization method.

[0177] The processor of the game live synchronization device is used to provide computing and control capabilities to support the operation of the entire game live synchronization device. The memory of the game live synchronization device may store computer-readable instructions, which, when executed by the processor, may enable the processor to execute the game live synchronization method of the present application. The network interface of the game live synchronization device is used to connect and communicate with the terminal.

[0178] Those skilled in the art will understand that Fig.14 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the game live broadcast synchronization device to which the scheme of the present application is applied. The specific game live broadcast synchronization device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0179] In this embodiment, the processor is used to execute Fig.13 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to realize data transmission between user terminals or servers. The non-volatile readable storage medium in this embodiment stores the program code and data required to execute all modules in the game live broadcast synchronization device of this application, and the server can call the program code and data of the server to execute the functions of all modules.

[0180] The present application also provides a non-volatile readable storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the game live broadcast synchronization method of any embodiment of the present application.

[0181] The present application also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any embodiment of the present application when executed by one or more processors.

[0182] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0183] In summary, the present application can enable the game demonstration animation in the game live broadcast scene to be displayed synchronously with the live video stream, avoid occupying the system overhead of the service cluster, and ensure that the game can be operated collaboratively by multiple users.

Claims

1. A method for synchronizing game live broadcast, characterized in that: include: Acquire an operation data stream generated by an operator user operating a game in a live broadcast room, wherein the operation data in the operation data stream is marked with its operation timestamp; Obtaining a live video stream pushed by a host user in the live broadcast room and marked with the operation timestamp; Loading the demonstration animation of the game according to the operation data stream, and playing the live video stream, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp; Before obtaining the operation data stream generated by the operator user operating the game in the live broadcast room, it includes: The terminal device of the audience user in the live broadcast room generates a corresponding operation data stream in response to the operator user operating the game, the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the audience user; The terminal device of the audience user broadcasts the operation data stream to the live broadcast room through the game server; The terminal device of the anchor user loads and displays various progress screens of the game according to the operation data in the operation data stream received to form a demonstration animation; The terminal device of the anchor user marks the image frame synchronized with the image frame with the operator timestamp of the operation data according to the synchronous display relationship between the operation screen corresponding to the operation data and the image frame of the live video stream of the anchor user; The terminal device of the anchor user broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

2. The game live broadcast synchronization method according to claim 1, characterized in that: Before obtaining the operation data stream generated by the operator user operating the game in the live broadcast room, it includes: The terminal device of the anchor user in the live broadcast room responds to the operator user operating the game to form a corresponding progress screen and generate a corresponding operation data stream, the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the anchor user; The terminal device of the anchor user loads and displays various progress screens of the game according to the operation data in the operation data stream to form a demonstration animation; The terminal device of the anchor user marks the image frame synchronized with the image frame with the operator timestamp of the operation data according to the synchronous display relationship between the operation screen corresponding to the operation data and the image frame of the live video stream of the anchor user; The terminal device of the anchor user broadcasts the operation data stream to the live broadcast room through the game server, and broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

3. The game live broadcast synchronization method according to claim 1, characterized in that: The method comprises: loading a demonstration animation of the game according to the operation data stream, and playing the live video stream, so that the progress screen in the demonstration animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp, including: Get the game running time between the current time and the game start time; Determine target operation data corresponding to the ongoing screen to be rendered according to the running time of the game, and determine a target operation timestamp corresponding to the target operation data; The ongoing picture and the image frame corresponding to the target operation timestamp are synchronously displayed according to the target operation timestamp.

4. The game live broadcast synchronization method according to claim 1, characterized in that: Loading the demonstration animation of the game according to the operation data stream includes: Determining whether the program instructions of the game include an original random number for determining a game level; Replacing the original random number with a pseudo-random number of a corresponding sequence in a standardized pseudo-random number sequence; The program instruction is executed according to the pseudo-random number to enter a game level corresponding to the pseudo-random number.

5. The game live broadcast synchronization method according to claim 1, characterized in that: Loading the demonstration animation of the game according to the operation data stream includes: Determining whether the program instructions of the game contain a parameter whose result value is a floating point number; Convert the resulting value of the parameter to an integer or fixed-point number; The program instructions are executed according to the converted result values.

6. The game live broadcast synchronization method according to any one of claims 1 to 5, characterized in that: After loading the game demonstration animation according to the operation data stream and playing the live video stream, the method further comprises: Respond to the game live broadcast replay command and determine the replay start time; Respectively acquiring the operation data stream and the live video stream whose operation timestamps are after the playback start time; The operation data stream and the live video stream are played synchronously according to the corresponding relationship between the operation timestamps.

7. A game live broadcast synchronization device, characterized in that: include: An operation data acquisition module is configured to acquire an operation data stream generated by an operator user operating a game in a live broadcast room, wherein the operation data in the operation data stream is marked with an operation timestamp; A live video acquisition module, configured to acquire a live video stream annotated with the operation timestamp and pushed by the anchor user in the live broadcast room; a synchronous playback processing module, configured to load the demo animation of the game according to the operation data stream, and to play the live video stream, so that the progress screen in the demo animation and the image frame in the live video stream are displayed synchronously based on the operation timestamp; The audience operation unit is configured as a terminal device of an audience user in the live broadcast room to generate a corresponding operation data stream in response to an operator user operating a game, wherein the operation data in the operation data stream is marked with an operation timestamp, and the operator user is the audience user; A game push unit, configured to broadcast the operation data stream to the live broadcast room via a game server through the terminal device of the audience user; The anchor loading unit is configured to load and display various progress screens of the game according to the operation data in the operation data stream received by the anchor user's terminal device to form a demonstration animation; An anchor synchronization unit is configured to mark the image frames synchronized with the anchor user's live video stream with the operator timestamp of the operation data according to the synchronous display relationship between the operation screen corresponding to the operation data and the image frames; The anchor push unit is configured so that the terminal device of the anchor user broadcasts the live video stream marked with the operation timestamp to the live broadcast room through the media server.

8. A game live broadcast synchronization device, comprising a central processing unit and a memory, characterized in that: The central processor is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 6.

9. A non-volatile readable storage medium, characterized in that: A computer program is stored in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the method according to any one of claims 1 to 6 are executed.

Images