Live broadcast sound and picture synchronization method and device

Abstract

The embodiment of the invention provides a live broadcast sound and picture synchronization method, which is used for an audience terminal, and comprises the following steps: obtaining a live broadcast stream, the live broadcast stream comprising supplemental enhancement information, and the supplemental enhancement information comprising a live broadcast material identifier; acquiring corresponding live broadcast material information from a preset queue according to the live broadcast material identifier; wherein the preset queue comprises one or more pieces of live broadcast material information issued by a server; determining a live broadcast material card according to the corresponding live broadcast material information; and performing live broadcast display based on the live broadcast stream and the live broadcast material card. According to the technical scheme of the embodiment of the invention, the live broadcast material identifier is written into the supplemental enhancement information SEI of the live broadcast stream, and after the live broadcast stream is decoded, the live broadcast material card and the decoded audio and video content can be synchronously displayed based on the SEI, so that the synchronization of live broadcast audio and picture is ensured, and the watching experience is effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of live broadcast technology, and in particular, to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for synchronizing audio and video in a live broadcast. Background Art

[0002] With the rapid popularization of Internet technology, online live broadcasting is being accepted and loved by more and more people. Online live broadcasting involves: live broadcasting platform, anchor terminal and audience terminal. The anchor terminal can provide multimedia content (such as video and audio) to the audience terminal through the live broadcasting platform, and can also receive multimedia content (such as bullet screen and gifts) provided by the audience terminal through the live broadcasting platform, so as to achieve the effect of live broadcast and interaction.

[0003] In online live broadcasts, the host can display various live broadcast materials (such as clothing, teaching equipment, fruits, etc.) and further introduce the live broadcast materials through explanations. However, in current live broadcast technologies, the display time of live broadcast materials is earlier than the host's introduction time, resulting in asynchrony between live broadcast audio and video, and poor user experience.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a live audio and video synchronization method, apparatus, computer equipment, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a live audio and video synchronization method for a viewer terminal, the method comprising: Acquire a live stream, wherein the live stream includes supplementary enhancement information, and the supplementary enhancement information includes a live material identifier; According to the live broadcast material identifier, corresponding live broadcast material information is obtained from a preset queue; wherein the preset queue includes one or more live broadcast material information sent by the server; Determine a live broadcast material card according to the corresponding live broadcast material information; Based on the live broadcast stream and the live broadcast material card, a live broadcast presentation is performed.

[0007] Optionally, the preset queue is obtained by the following operations: Acquire live broadcast material information sent by the server, where the live broadcast material information carries a timestamp; Comparing the timestamp of the live broadcast material information with the timestamp of the live broadcast material information sent last time; When the timestamp of the live broadcast material information is greater than the timestamp of the live broadcast material information sent last time, obtaining the current length of the preset queue; When the current length of the preset queue is less than a preset threshold, the live broadcast material information is added to the preset queue.

[0008] Optionally, the live broadcast material information includes material content and material status, and the material status includes: adding, removing, switching or updating; Correspondingly, determining the live broadcast material card according to the corresponding live broadcast material information includes: Determine the material content and material status based on the corresponding live broadcast material information; When the material status is adding or switching, a new live material card is generated based on the material content to be displayed in the live broadcast room; When the material status is removed, based on the material content, remove the live material card corresponding to the live broadcast room; When the material status is updated, the live material card corresponding to the live broadcast room is updated based on the material content.

[0009] Another aspect of an embodiment of the present application provides a live broadcast audio and video synchronization method for a host terminal, the method comprising: Sending a material display instruction to the server, the material display instruction including a live material identifier, the server being used to: obtain corresponding live material information according to the live material identifier and send it to the viewer terminal; Polling the server to obtain a current polling result, wherein the current polling result includes the live broadcast material identifier; Update the supplementary enhancement information based on the live broadcast material identifier, and add the updated supplementary enhancement information to the live broadcast stream; The live broadcast stream is pushed to the server so as to be forwarded to the viewer terminal through the server, and the viewer terminal is used to perform live broadcast presentation based on the live broadcast stream and the corresponding live broadcast material information.

[0010] Optionally, the polling result carries a timestamp; updating the supplementary enhancement information based on the live broadcast material identifier includes: Compare the timestamp of the current polling result with the timestamp of the previous polling result; When the timestamp of the current polling result is greater than the timestamp of the previous polling result, comparing the live broadcast material identifier with the live broadcast material identifier in the previous polling result; If the two are different, updating the supplementary enhancement information based on the live broadcast material identifier; The current polling result is cached in a preset area.

[0011] Optionally, updating the supplemental enhancement information based on the live broadcast material identifier further includes: Determining whether the last round of updating of the supplemental enhancement information is completed; If the previous round of updating is not completed, terminating the previous round of updating, and updating the supplementary enhancement information based on the live broadcast material identifier; When the last round of updating has been completed, the supplementary enhancement information is updated based on the live broadcast material identifier.

[0012] Optionally, the live audio and video synchronization method further includes: Get the current live broadcast status, where the live broadcast status includes starting or stopping material display; When the current live broadcast state is to start material display, performing a polling operation on the server; When the current live broadcast status is to terminate the material display, the polling operation is terminated.

[0013] Another aspect of an embodiment of the present application provides a live audio and video synchronization method for a server, the method comprising: Receiving a material display instruction sent by an anchor terminal, wherein the material display instruction includes a live broadcast material identifier; Based on the live broadcast material identifier, corresponding live broadcast material information is acquired, and the live broadcast material information is sent to the viewer terminal; receiving a polling request from the anchor terminal, and sending a polling result to the anchor terminal, wherein the polling result includes the live broadcast material identifier; Receive the live stream provided by the anchor terminal, and forward the live stream to the audience terminal; wherein the live stream includes supplementary enhancement information, the supplementary enhancement information includes the live material identifier, and the audience terminal is used to: perform live display based on the live stream and the corresponding live material information.

[0014] Optionally, the live audio and video synchronization method further includes: Based on the material display instruction, determine the current live broadcast state, where the live broadcast state includes starting material display or terminating material display; The current live broadcast status is pushed to the anchor terminal.

[0015] Another aspect of the embodiment of the present application provides a live audio and video synchronization device for a viewer terminal, the device comprising: A first acquisition module, configured to acquire a live stream, wherein the live stream includes supplementary enhancement information, and the supplementary enhancement information includes a live material identifier; A second acquisition module is used to acquire corresponding live broadcast material information from a preset queue according to the live broadcast material identifier; wherein the preset queue includes one or more live broadcast material information sent by the server; A third acquisition module is used to determine a live broadcast material card according to the corresponding live broadcast material information; A display module is used to perform live display based on the live stream and the live material card.

[0016] Another aspect of the embodiment of the present application provides a live broadcast audio and video synchronization device for a host terminal, the device comprising: A sending module, used for sending a material display instruction to a server, wherein the material display instruction includes a live material identifier, and the server is used for: obtaining corresponding live material information according to the live material identifier and sending it to the viewer terminal; A polling module, used for polling the server to obtain a current polling result, wherein the current polling result includes the live broadcast material identifier; An inserting module, configured to update the supplementary enhancement information based on the live broadcast material identifier, and add the updated supplementary enhancement information to the live broadcast stream; The streaming module is used to push the live streaming to the server so as to forward it to the viewer terminal through the server. The viewer terminal is used to perform live broadcasting based on the live streaming and the corresponding live broadcast material information.

[0017] Another aspect of an embodiment of the present application provides a live audio and video synchronization device for a server, the device comprising: A receiving module, used to receive a material display instruction sent by an anchor terminal, wherein the material display instruction includes a live broadcast material identifier; A sending module, used for obtaining corresponding live broadcast material information based on the live broadcast material identifier, and sending the corresponding live broadcast material information to the viewer terminal; A sending module, configured to receive a polling request from the anchor terminal and send a polling result to the anchor terminal, wherein the polling result includes the live broadcast material identifier; A forwarding module is used to receive the live stream provided by the anchor terminal and forward the live stream to the audience terminal; wherein the live stream includes supplementary enhancement information, the supplementary enhancement information includes the live material identifier, and the audience terminal is used to: perform live display based on the live stream and the corresponding live material information.

[0018] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0019] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0020] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0021] The above technical solution adopted in the embodiment of the present application may have the following advantages: The audience terminal can obtain the live stream from the server, and parse the supplementary enhancement information therein to obtain the live material identifier. According to the live material identifier, the corresponding live material information can be obtained from the preset queue. Among them, the preset queue includes one or more live material information issued by the server. According to the corresponding live material information, the live material card can be determined, so as to perform live display based on the live stream and the live material card. It can be seen that the embodiment of the present application writes the live material identifier into the supplementary enhancement information (SEI) of the live stream, so that after the audience terminal decodes the live stream, it can synchronously display the live material card and the decoded audio and video content (i.e., the host's explanation) based on the SEI, ensuring the synchronization of live audio and video, and effectively improving the viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0023] Figure 1 The operating environment diagram of the live audio and video synchronization method according to the first embodiment of the present application is schematically shown; Figure 2 The flowchart of the live audio and video synchronization method according to the first embodiment of the present application is schematically shown; Figure 3 The following schematically shows a flow chart of obtaining a preset queue according to Embodiment 1 of the present application; Figure 4 Schematically shows Figure 2 Flow chart of sub-steps of step S204; Figure 5The flowchart of the live audio and video synchronization method according to the second embodiment of the present application is schematically shown; Figure 6 The operating environment diagram of the live audio and video synchronization method according to the second embodiment of the present application is schematically shown; Figure 7 Schematically shows Figure 5 Flow chart of sub-steps of step S504; Figure 8 Schematically shows Figure 7 Flow chart of sub-steps of step S704; Fig. 9 The newly added flow chart of the live audio and video synchronization method according to the second embodiment of the present application is schematically shown; Fig.10 A flowchart schematically shows a method for synchronizing live audio and video according to Embodiment 3 of the present application. Fig.11 This is an application example diagram of the live audio and video synchronization method according to an embodiment of the present application; Fig.12 This is an application example diagram of the live audio and video synchronization method according to an embodiment of the present application; Fig.13 A block diagram of a live audio and video synchronization device according to a fourth embodiment of the present application is schematically shown; Fig.14 A block diagram of a live audio and video synchronization device according to Embodiment 5 of the present application is schematically shown; Fig.15 A block diagram schematically shows a live audio and video synchronization device according to Embodiment 6 of the present application; and Fig.16 The hardware architecture diagram of the computer device according to the seventh embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0025] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0026] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0027] First, the following terms are explained: SEI (Supplemental Enhancement Information): Supplemental enhancement information is a mechanism used to transmit additional information in video coding standards (such as H.264 and H.265). SEI can embed additional data in the video bitstream.

[0028] Long link: A network communication mechanism based on Socket technology that achieves relatively real-time data communication by establishing a persistent connection.

[0029] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: During live streaming, the host can display various live materials (such as clothing, teaching equipment, fruits, etc.) and further introduce the live materials through explanations.

[0030] However, the applicant has learned that the relevant live broadcast technology pushes live broadcast material information to the audience terminal through a long link. Due to the high real-time nature of the long link, the live broadcast material will be displayed earlier than the host's introduction time, resulting in the live broadcast audio and video being out of sync and the user experience being poor.

[0031] To this end, an embodiment of the present application provides a technical solution for synchronization of live audio and video. In this technical solution: (1) When the host is explaining the live material, the server can record the host's operation behavior (i.e., the live material of the explanation) and send the detailed information of the live material to the audience terminal through broadcast. The host terminal will poll the server in real time before pushing the stream and pull the host's operation behavior from the server. The acquired host operation behavior data is encapsulated into a fixed format and written into SEI, and the SEI is inserted into the live stream. The live stream is pushed to the server and forwarded to the audience terminal via the server. (2) After decoding the live stream, the audience terminal can synchronously display the live material card and the decoded audio and video content (i.e., the host's explanation) according to the decoded SEI, so as to achieve synchronization of live audio and video and effectively improve the live viewing experience. See below for details.

[0032] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0033] like Figure 1 As shown, the operating environment diagram includes: service platform 2, anchor terminals (4A, 4B, ..., 4M), and audience terminals (6A, 6B, ..., 6N). In the live broadcast scenario, the anchor terminal (4A, 4B, ..., 4M) logs in to the service platform 2, and the service platform 2 pushes the live broadcast data to the audience terminal (6A, 6B, ..., 6N) in real time.

[0034] Service platform 2 can provide live broadcast service, which can be a single server, a server cluster or a cloud computing service center. The server can include a business server, a video cloud server, etc.

[0035] The anchor terminal (4A, 4B, ..., 4M) is used to generate live broadcast data in real time and perform streaming operations on the live broadcast data. The live broadcast data may include audio data or video data. The anchor terminal may be an electronic device such as a smart phone or a tablet computer. Of course, the anchor terminal may be a virtual computing instance in the service platform 2.

[0036] The audience terminal (6A, 6B, ..., 6N) may be configured to receive the live broadcast data of the anchor terminal in real time. The audience terminal (6A, 6B, ..., 6N) may be any type of computing device, such as a smart phone, a tablet device, a laptop computer, a smart TV, a vehicle-mounted terminal, etc. The audience terminal (6A, 6B, ..., 6N) may have a built-in browser or a special program, and receive the live broadcast data through the browser or the special program to output content to the user. The content may include video, audio, comments, text data and / or the like.

[0037] The audience terminal (6A, 6B, ..., 6N) may include a player. The player outputs (e.g., displays, presents) content to the user. The content may include video, audio, comments, text data and / or the like. The audience terminal (6A, 6B, ..., 6N) may include an interface, which may include an input element (touch screen). For example, the input element may be configured to receive user instructions, and the user instructions may enable the audience terminal (6A, 6B, ..., 6N) to perform various operations, such as sending a bullet screen, entering a comment, giving a gift, etc.

[0038] The anchor terminals (4A, 4B, ..., 4M), the audience terminals (6A, 6B, ..., 6N) and the service platform 2 may be connected via a network. The network may include various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls and / or proxy devices, etc. The network may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof and / or the like. The network may include wireless links, such as cellular links, satellite links, Wi-Fi links and / or the like.

[0039] It should be noted that Figure 1 The number of the host terminal and the audience terminal is only illustrative and is not used to limit the scope of patent protection of this application. According to actual conditions, there can be any number of host terminals and audience terminals, and the host terminal and the audience terminal can be the same terminal.

[0040] The technical solutions of the present application are described below through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described here.

[0041] Embodiment 1 Figure 2 The flowchart of the live audio and video synchronization method according to the first embodiment of the present application is schematically shown.

[0042] like Figure 2 As shown, the live audio and video synchronization method can be used in a viewer terminal (such as 6A), and can specifically include steps S200-206, wherein: Step S200: acquiring a live stream, wherein the live stream includes supplementary enhancement information, and the supplementary enhancement information includes a live material identifier.

[0043] Step S202: acquiring corresponding live broadcast material information from a preset queue according to the live broadcast material identifier; wherein the preset queue includes one or more live broadcast material information sent by the server.

[0044] Step S204: determining a live broadcast material card according to the corresponding live broadcast material information.

[0045] Step S206: Perform live broadcast presentation based on the live broadcast stream and the live broadcast material card.

[0046] The live audio and video synchronization method provided in this embodiment is that the audience terminal can obtain the live stream from the server, and parse the supplementary enhancement information therein to obtain the live material identifier. According to the live material identifier, the corresponding live material information can be obtained from the preset queue. Among them, the preset queue includes one or more live material information issued by the server. According to the corresponding live material information, the live material card can be determined, so as to perform live display based on the live stream and the live material card. It can be seen that the embodiment of the present application writes the live material identifier into the supplementary enhancement information (SEI) of the live stream, so that after the audience terminal decodes the live stream, it can synchronously display the live material card and the decoded audio and video content (i.e. the host's explanation) based on the SEI, ensuring the synchronization of live audio and video, and effectively improving the viewing experience.

[0047] The following combination Figure 2 , each step in steps S200~S206 and other optional steps are described in detail.

[0048] Step S200 , obtaining a live stream, the live stream including supplementary enhancement information, and the supplementary enhancement information including a live material identifier.

[0049] The anchor can display various types of live broadcast materials during the live broadcast, such as clothing, teaching equipment, fruits or various commodities, and explain them. The live broadcast stream generated by the anchor terminal will be pushed to the server. The audience terminal can obtain the live broadcast stream from the server. The live broadcast stream may include audio and video content and supplementary enhancement information. The audio and video content may include the anchor's explanation screen and the anchor's explanation audio for a specific live broadcast material. The supplementary enhancement information may include the live broadcast material identifier of the specific live broadcast material.

[0050] Step S202 , according to the live broadcast material identifier, obtain corresponding live broadcast material information from a preset queue; wherein the preset queue includes one or more live broadcast material information sent by the server.

[0051] The live broadcast material information may include detailed data of the live broadcast material for display. Before the live broadcast begins, the anchor terminal or other terminals may import the live broadcast material information corresponding to each of the multiple live broadcast materials into the preset database of the server in advance to improve the response speed during the live broadcast. When the anchor starts to display and explain the live broadcast material, he may send a material display instruction to the server, and the material display instruction may include a live broadcast material identifier. The server may quickly obtain the corresponding live broadcast material information according to the live broadcast material identifier and send it to the audience terminal through the broadcast mechanism. For example, the anchor may display and explain a live broadcast material every minute (min), 0~1min may correspond to live broadcast material A, 1~2min may correspond to live broadcast material B, and 2~3min may correspond to live broadcast material C. Accordingly, at the beginning, the anchor terminal may send a material display instruction including the live broadcast material A identifier to the server, and the server may obtain the live broadcast material information corresponding to the live broadcast material A and send it to the audience terminal, and the audience terminal may cache it to the preset queue. At 1min, the anchor terminal may send a material display instruction carrying the live broadcast material B identifier to the server, and the server may obtain the live broadcast material information corresponding to the live broadcast material B and send it to the audience terminal, and the audience terminal may cache it to the preset queue. By analogy, it can be known that the viewer terminal can cache one or more live broadcast material information sent by the server through a preset queue.

[0052] In actual applications, live streaming may have a certain delay due to various reasons such as network transmission, server load, encoding and decoding processing. If the audience terminal displays the live material information immediately after obtaining it, the display time of the live material may be earlier than the introduction time of the live material, resulting in asynchrony of the live audio and video, leading to a poor viewing experience. To this end, the audience terminal can set a preset queue to cache one or more material information sent by the server, and on the other hand, it can obtain the corresponding live material information from the preset queue based on the live material identifier in the SEI. The live material identifier of the SEI can indicate the live material targeted by the current live stream segment (audio and video content), so the live material information and the audio and video content can be matched based on the SEI to achieve live audio and video synchronization. For example, the live stream can arrive at the audience terminal with a delay of 2 minutes, but the SEI corresponding to the live stream segment 0~1min includes the live material A identifier, and the SEI corresponding to the live stream segment 1~2min includes the live material B identifier. After the audience terminal decodes the live stream, it can obtain the corresponding live material information from the preset queue based on the live material A identifier for displaying live material A. At 1 minute, the corresponding live broadcast material information can be obtained from the preset queue based on the live broadcast material B identifier to display the live broadcast material B.

[0053] In this embodiment, based on the live broadcast material identification of SEI, the live broadcast material information can be matched with the live broadcast stream segment to facilitate subsequent alignment of the live broadcast audio and video.

[0054] The above embodiment describes that the viewer terminal can cache one or more live broadcast material information sent by the server through a preset queue. An exemplary solution will be provided below.

[0055] In an optional embodiment, if Figure 3 As shown, the preset queue can be obtained by following the steps below: Step S300, obtaining live broadcast material information sent by the server, wherein the live broadcast material information carries a timestamp.

[0056] Step S302: compare the timestamp of the live broadcast material information with the timestamp of the live broadcast material information sent last time.

[0057] Step S304: when the timestamp of the live broadcast material information is greater than the timestamp of the live broadcast material information sent last time, obtain the current length of the preset queue.

[0058] Step S306: When the current length of the preset queue is less than a preset threshold, the live broadcast material information is added to the preset queue.

[0059] Exemplarily, when the viewer terminal obtains the live broadcast material information newly sent by the server, it can compare the timestamp of the live broadcast material information with the timestamp of the live broadcast material information previously sent to the local cache. If the timestamp of the newly sent live broadcast material information is less than or equal to the timestamp of the local cache, it means that the same data has been processed before, and the newly sent live broadcast material information can be discarded. If the timestamp of the newly sent live broadcast material information is greater than the timestamp of the local cache, it means that the live broadcast material information is new and needs to be cached in the preset queue. Before caching, the current length of the preset queue can also be obtained. When the current length of the preset queue is less than or equal to the preset threshold (such as 20), the live broadcast material information can be added to the preset queue, otherwise it can be discarded.

[0060] In this embodiment, by comparing the timestamp of the live broadcast material information and the current length of the preset queue, repeated processing can be avoided, the cache can be effectively managed, and the data processing efficiency can be improved to optimize the live broadcast experience.

[0061] The above embodiment describes how to align live material information and live stream segments based on SEI. The following describes how to execute business logic to synchronously display live material and audio and video content.

[0062] Step S204 , determine the live broadcast material card according to the corresponding live broadcast material information.

[0063] Live material information may include material content and material status. Material content may include the name, category, specification parameters, pictures, videos, jump links, ratings, etc. associated with the live material. Material status may include adding, removing, switching, or updating. Live material may be displayed in the form of cards, links, QR codes, etc. The following is an exemplary introduction using cards as an example.

[0064] In an optional embodiment, if Figure 4 As shown, step S204 may include: Step S400: determining the material content and material status based on the corresponding live broadcast material information.

[0065] Step S402, when the material status is adding or switching, a new live material card is generated based on the material content to be displayed in the live broadcast room.

[0066] Step S404, when the material status is removed, remove the live material card corresponding to the live broadcast room based on the material content.

[0067] Step S406, when the material status is updated, based on the material content, update the live material card corresponding to the live broadcast room.

[0068] Exemplarily, the live broadcast material information can be parsed to obtain the corresponding material content and material status. When the material status is added or switched, a new live broadcast material card can be generated according to the material content and displayed in the live broadcast room or replace the original live broadcast material card in the live broadcast room. When the material status is removed, the corresponding live broadcast material card in the live broadcast room can be determined according to the material content, and the removal operation can be performed to terminate the material display. When the material status is updated, the corresponding live broadcast material card in the live broadcast room can be determined according to the material content, and the content on the live broadcast material card can be updated, such as updating the picture and rating of the live broadcast material.

[0069] In this embodiment, different business logics can be executed according to the material content and material status of the live material information, and live material cards can be added, removed, switched or updated in real time to improve the smoothness and accuracy of viewing and reduce errors.

[0070] Step S206 , based on the live stream and the live material card, perform live display.

[0071] As mentioned above, based on the live broadcast material identifier in SEI, the live broadcast stream segment and the live broadcast material card can be aligned, so that when the live broadcast material card is displayed, the audio and video content (the host's explanation picture and the host's explanation audio) of the live broadcast material card can be displayed together, thereby achieving synchronization of live broadcast audio and video and effectively improving the live broadcast viewing experience.

[0072] Embodiment 2 Figure 5 The flowchart of the live audio and video synchronization method according to the second embodiment of the present application is schematically shown.

[0073] like Figure 5 As shown, the live audio and video synchronization method can be used in an anchor terminal (such as 4A), and specifically may include steps S500-506, wherein: Step S500: Send a material display instruction to a server, wherein the material display instruction includes a live material identifier. The server is used to obtain corresponding live material information according to the live material identifier and send it to the viewer terminal.

[0074] Step S502: poll the server to obtain a current polling result, where the current polling result includes the live broadcast material identifier.

[0075] Step S504: updating the supplementary enhancement information based on the live broadcast material identifier, and adding the updated supplementary enhancement information to the live broadcast stream.

[0076] Step S506: Push the live stream to the server so as to forward it to the viewer terminal through the server. The viewer terminal is used to: perform live broadcasting based on the live stream and the corresponding live broadcast material information.

[0077] In the live audio and video synchronization method provided in this embodiment, the anchor terminal can send a material display instruction carrying a live material identifier to the server, so that the server can send the corresponding live material information to the audience terminal based on the live material identifier. At the same time, the anchor terminal polls the server to obtain the current polling result, and the current polling result includes the live material identifier. Update the supplementary enhancement information based on the live material identifier, and add the updated supplementary enhancement information to the live stream. Push the live stream to the server to forward it to the audience terminal through the server, and the audience terminal can perform live display based on the live stream and the corresponding live material information. It can be seen that the embodiment of the present application writes the live material identifier into the supplementary enhancement information (SEI) of the live stream, so that the live material card and the decoded audio and video content (i.e. the anchor's explanation) can be synchronized based on the SEI, ensuring the synchronization of live audio and video, and effectively improving the viewing experience.

[0078] The following combination Figure 5 , each step in steps S500~S506 and other optional steps are explained in detail.

[0079] Step S500 , sending a material display instruction to the server, the material display instruction including a live material identifier, the server is used to: obtain corresponding live material information according to the live material identifier and send it to the viewer terminal.

[0080] When the anchor starts to show and explain the live broadcast material, he can send a material display instruction including the live broadcast material identifier to the server through the anchor terminal. Figure 6 As shown, the anchor terminal may include a collection and editing device and a streaming device. The collection and editing device may be used to collect and process live content, such as video collection, audio playback, video editing, content processing, etc. The streaming device may be used to transmit the collected live content to a server. In some embodiments, Figure 6 As shown in the figure, an assistant broadcast device can also be configured, which can be operated by the on-site assistant broadcaster to send a material display instruction to the server. The host terminal and the assistant broadcast terminal work together. The host focuses on presenting the live content, and the assistant broadcaster can manage the live resources, so as to further improve the live broadcast efficiency and viewing experience. The server can parse the material display instruction and obtain the live broadcast material identifier. According to the live broadcast material identifier, the corresponding live broadcast material information is quickly and accurately obtained and sent to the audience terminal through the broadcast mechanism.

[0081] Step S502 , polling the server to obtain a current polling result, wherein the current polling result includes the live broadcast material identifier.

[0082] As mentioned above, when the anchor is showing and explaining the live broadcast material, he will send a material display instruction (live broadcast material identifier) ​​to the server, and the server can record the anchor's operation behavior accordingly. As the streaming end, the anchor terminal can poll the server in real time to pull the anchor's operation behavior data, that is, the live broadcast material identifier, from the server. The polling operation can be performed based on the live broadcast status. An exemplary solution is provided below.

[0083] In an optional embodiment, if Fig. 9 As shown, the live audio and video synchronization method may also include: Step S900, obtaining the current live broadcast status, where the live broadcast status includes starting material display or terminating material display.

[0084] Step S902: When the current live broadcast state is to start material display, a polling operation is performed on the server.

[0085] Step S904, when the current live broadcast state is to terminate the material display, terminate the polling operation.

[0086] In this embodiment, the current live broadcast status can be queried through the server or locally. The live broadcast status can include starting material display or terminating material display. When the current live broadcast status is starting material display, a polling operation can be performed on the server. When the current live broadcast status is terminating material display, the polling operation can be stopped. In this way, resource utilization can be optimized, system efficiency can be improved, and burden can be reduced.

[0087] Step S504 , updating the supplementary enhancement information based on the live broadcast material identifier, and adding the updated supplementary enhancement information to the live broadcast stream.

[0088] Exemplarily, the live broadcast material identifier can be encapsulated into a fixed format and inserted into the SEI, and can be inserted repeatedly multiple times (such as 10 times) to ensure that multiple SEIs are inserted to prevent freezes or network delays from causing SEI update failures, resulting in the inability to align live stream segments and live broadcast materials according to SEI. The anchor end polls in real time and generates SEI based on the results of each polling, and embeds these SEIs into each segment of the live stream to ensure that each live stream segment is aligned with the corresponding live broadcast material identifier. Exemplarily, the SEI of the 0~1min segment of the live stream can include the identifier of live broadcast material A, and the SEI of the 1~2min segment of the live stream can include the identifier of live broadcast material B.

[0089] Some exemplary schemes are provided below.

[0090] In an optional embodiment, if Figure 7 As shown, step S504 may include: Step S700, comparing the timestamp of the current polling result with the timestamp of the previous polling result.

[0091] Step S702: When the timestamp of the current polling result is greater than the timestamp of the previous polling result, the live broadcast material identifier is compared with the live broadcast material identifier in the previous polling result.

[0092] Step S704: when the two are different, update the supplementary enhancement information based on the live broadcast material identifier.

[0093] Step S706: Cache the current polling result in a preset area.

[0094] Exemplarily, the timestamp of the current polling result can be compared with the timestamp of the previous polling result cached locally. When the timestamp of the current polling result is less than or equal to the previous polling result, it means that the same data has been processed and the current polling result can be directly discarded. When the timestamp of the current polling result is greater than the previous polling result, it means that the polling result is newer and needs to be processed. The live broadcast material identifier of the current polling result can be compared with the live broadcast material identifier of the previous polling result. When the two are different, it means that the live broadcast material identifier is the latest and needs to be inserted into SEI and cached locally. For example: the operation of inserting the live broadcast material identifier into SEI is executed in a loop 10 times, and the current polling result is cached locally. When the next polling result is obtained, it can be compared with the current polling result that has been cached locally, and so on.

[0095] In this embodiment, by comparing the timestamp of the polling result and the live broadcast material identifier, the real-time nature of the data can be ensured, and the data processing efficiency and system stability can be improved.

[0096] In an optional embodiment, if Figure 8 As shown, step SS704 may include: Step S800: determining whether the last round of updating of the supplementary enhancement information is completed.

[0097] Step S802: If the previous round of updating is not completed, terminate the previous round of updating, and update the supplementary enhancement information based on the live broadcast material identifier.

[0098] Step S804: when the previous round of updating has been completed, update the supplementary enhancement information based on the live broadcast material identifier.

[0099] Exemplarily, before executing the update, it can be determined whether the last round of insertion of the supplemental enhancement information has been completed. If it has not been completed, the last round of update can be directly terminated, and the SEI can be updated based on the live material identifier. If it has been completed, the live material identifier can be directly inserted into the SEI.

[0100] In this embodiment, the correctness of the update operation can be guaranteed, data conflicts or errors caused by repeated operations can be avoided, and it can be ensured that each insertion is based on the latest live broadcast material identifier, which can further improve the stability of the system.

[0101] Step S506 , pushing the live stream to the server so as to forward it to the viewer terminal through the server, and the viewer terminal is used to: perform live display based on the live stream and the corresponding live material information.

[0102] For example, the anchor terminal can push the live stream to the server, so as to forward it to the viewer terminal through the server. The viewer terminal can perform live broadcasting based on the live stream and the corresponding live broadcast material information previously sent by the server.

[0103] In this embodiment, the live stream segment and the live material card can be aligned based on the live material identifier in the SEI, so that when the live material card is displayed, the audio and video content (the host's commentary picture and the host's commentary audio) of the live material card can be displayed together, thereby achieving synchronization of live audio and video and effectively improving the live viewing experience.

[0104] Embodiment 3 Fig.10 The flowchart of the live audio and video synchronization method according to the second embodiment of the present application is schematically shown.

[0105] like Fig.10 As shown, the live audio and video synchronization method can be used in a server (such as service platform 2), and can specifically include steps S1000-1006, wherein: Step S1000: receiving a material display instruction sent by a host terminal, wherein the material display instruction includes a live broadcast material identifier.

[0106] Step S1002: based on the live broadcast material identifier, obtain corresponding live broadcast material information, and send the corresponding live broadcast material information to the viewer terminal.

[0107] Step S1004: receiving a polling request from the anchor terminal, and sending a polling result to the anchor terminal, wherein the polling result includes the live broadcast material identifier.

[0108] Step S1006, receiving the live stream provided by the anchor terminal, and forwarding the live stream to the viewer terminal; wherein the live stream includes supplementary enhancement information, and the supplementary enhancement information includes the live material identifier. The viewer terminal is used to: perform live display based on the live stream and the corresponding live material information.

[0109] In the live audio and video synchronization method provided in this embodiment, the server can receive a material display instruction carrying a live material identifier provided by the anchor terminal, so as to send the corresponding live material information to the audience terminal based on the live material identifier. At the same time, when the anchor terminal polls the server, the polling result can be returned to the anchor terminal, and the polling result includes the live material identifier. The server can obtain the live stream provided by the anchor terminal, and the live stream includes supplementary enhancement information. The supplementary enhancement information includes the live material identifier. The server can forward the live stream to the audience terminal, and the audience terminal can perform a live display based on the live stream and the corresponding live material information. It can be seen that the embodiment of the present application writes the live material identifier into the supplementary enhancement information (SEI) of the live stream, so that the live material card and the decoded audio and video content (i.e., the anchor's explanation) can be synchronized based on the SEI, ensuring the synchronization of live audio and video, and effectively improving the viewing experience.

[0110] In an optional embodiment, the live broadcast audio and video synchronization method may further include: determining the current live broadcast status based on the material display instruction, the live broadcast status including starting material display or terminating material display; and pushing the current live broadcast status to the anchor terminal.

[0111] In this embodiment, when the server receives the material display instruction from the anchor terminal, it can determine the current live broadcast status based on the material display instruction. The live broadcast status can include starting material display or terminating material display. The current live broadcast status is pushed to the anchor terminal. This can improve the management efficiency and controllability of the live broadcast and further optimize the live broadcast experience.

[0112] In order to make this application easier to understand, the following Fig.11 and Fig.12 An exemplary application is provided.

[0113] S11: The anchor terminal sends a material display instruction to the server, where the material display instruction includes a live broadcast material identifier.

[0114] S12: The server parses the material display instruction to obtain the live material identifier, sends the corresponding live material information to the viewer terminal based on the live material identifier, and the viewer terminal caches the corresponding live material information into a preset queue. The server determines the current live status and broadcasts the current live status to the anchor terminal.

[0115] S13: The anchor terminal performs a polling operation on the server based on the current live broadcast status to obtain a current polling result, which includes a live broadcast material identifier.

[0116] S14: The anchor terminal inserts the live broadcast material identifier into the SEI, and inserts the SEI into the live broadcast stream.

[0117] S15: The anchor terminal pushes the live stream to the server, so that the server forwards it to the viewer terminal.

[0118] S16: The viewer terminal parses the SEI to obtain a live broadcast material identifier, and obtains corresponding live broadcast material information from a preset queue based on the live broadcast material identifier.

[0119] S17: The viewer terminal determines the live broadcast material card based on the material content and material status in the corresponding live broadcast material information.

[0120] S18: Perform live broadcast presentation based on the live stream and live material cards.

[0121] In this exemplary application: (1) When the host is explaining the live broadcast material, the server can record the host's operation behavior (i.e., the live broadcast material being explained) and send the detailed information of the live broadcast material to the viewer terminal through broadcast. The host terminal will poll the server in real time before pushing the stream and pull the host's operation behavior from the server. The acquired host operation behavior data is encapsulated into a fixed format and written into SEI, and the SEI is inserted into the live broadcast stream. The live broadcast stream is pushed to the server and forwarded to the viewer terminal via the server. (2) After decoding the live broadcast stream, the viewer terminal can synchronously display the live broadcast material card and the decoded audio and video content (i.e., the host's explanation) according to the decoded SEI, so as to achieve synchronization of live broadcast audio and video, and effectively improve the live broadcast viewing experience.

[0122] Embodiment 4 Fig.13 The block diagram of the live audio and video synchronization device according to the fourth embodiment of the present application is schematically shown. The device can be used in the audience terminal and can also be divided into one or more program modules. One or more program modules are stored in the storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.13 As shown, the device 1300 may include: a first acquisition module 1310, a second acquisition module 1320, a third acquisition module 1330 and a display module 1340, wherein: A first acquisition module 1310 is used to acquire a live stream, where the live stream includes supplementary enhancement information, and the supplementary enhancement information includes a live material identifier; The second acquisition module 1320 is used to acquire corresponding live broadcast material information from a preset queue according to the live broadcast material identifier; wherein the preset queue includes one or more live broadcast material information sent by the server; The third acquisition module 1330 is used to determine a live broadcast material card according to the corresponding live broadcast material information; The display module 1340 is used to perform live display based on the live stream and the live material card.

[0123] As an optional embodiment, the preset queue is obtained by the following operations: Acquire live broadcast material information sent by the server, where the live broadcast material information carries a timestamp; Comparing the timestamp of the live broadcast material information with the timestamp of the live broadcast material information sent last time; When the timestamp of the live broadcast material information is greater than the timestamp of the live broadcast material information sent last time, obtaining the current length of the preset queue; When the current length of the preset queue is less than a preset threshold, the live broadcast material information is added to the preset queue.

[0124] As an optional embodiment, the live broadcast material information includes material content and material status, and the material status includes: adding, removing, switching or updating; Correspondingly, determining the live broadcast material card according to the corresponding live broadcast material information includes: Determine the material content and material status based on the corresponding live broadcast material information; When the material status is adding or switching, a new live material card is generated based on the material content to be displayed in the live broadcast room; When the material status is removed, based on the material content, remove the live material card corresponding to the live broadcast room; When the material status is updated, the live material card corresponding to the live broadcast room is updated based on the material content.

[0125] Embodiment 5 Fig.14 The block diagram of the live audio and video synchronization device according to the fifth embodiment of the present application is schematically shown. The device can be used in the anchor terminal and can also be divided into one or more program modules. One or more program modules are stored in the storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.14 As shown, the device 1400 may include: a sending module 1410, a polling module 1420, an inserting module 1430 and a streaming module 1440, wherein: The sending module 1410 is used to send a material display instruction to the server, wherein the material display instruction includes a live material identifier, and the server is used to: obtain corresponding live material information according to the live material identifier and send it to the viewer terminal; A polling module 1420, configured to poll the server to obtain a current polling result, wherein the current polling result includes the live broadcast material identifier; An inserting module 1430 is configured to update the supplementary enhancement information based on the live broadcast material identifier, and add the updated supplementary enhancement information to the live broadcast stream; The streaming module 1440 is used to push the live streaming to the server so as to forward it to the viewer terminal through the server. The viewer terminal is used to perform live display based on the live streaming and the corresponding live material information.

[0126] As an optional embodiment, the polling result carries a timestamp; and updating the supplementary enhancement information based on the live broadcast material identifier includes: Compare the timestamp of the current polling result with the timestamp of the previous polling result; When the timestamp of the current polling result is greater than the timestamp of the previous polling result, comparing the live broadcast material identifier with the live broadcast material identifier in the previous polling result; If the two are different, updating the supplementary enhancement information based on the live broadcast material identifier; The current polling result is cached in a preset area.

[0127] As an optional embodiment, updating the supplementary enhancement information based on the live broadcast material identifier further includes: Determining whether the last round of updating of the supplemental enhancement information is completed; If the previous round of updating is not completed, terminating the previous round of updating, and updating the supplementary enhancement information based on the live broadcast material identifier; When the last round of updating has been completed, the supplementary enhancement information is updated based on the live broadcast material identifier.

[0128] As an optional embodiment, the device 1400 is further used for: Get the current live broadcast status, where the live broadcast status includes starting or stopping material display; When the current live broadcast state is to start material display, performing a polling operation on the server; When the current live broadcast status is to terminate the material display, the polling operation is terminated.

[0129] Embodiment 6 Fig.15The block diagram of the live audio and video synchronization device according to the sixth embodiment of the present application is schematically shown. The device can be used in a server and can also be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.15 As shown, the device 1500 may include: a receiving module 1510, a sending module 1520, a sending module 1530 and a forwarding module 1540, wherein: The receiving module 1510 is used to receive a material display instruction sent by the anchor terminal, where the material display instruction includes a live broadcast material identifier; The sending module 1520 is used to obtain corresponding live broadcast material information based on the live broadcast material identifier, and send the corresponding live broadcast material information to the viewer terminal; The sending module 1530 is used to receive the polling request of the anchor terminal and send the polling result to the anchor terminal, wherein the polling result includes the live broadcast material identifier; The forwarding module 1540 is used to receive the live stream provided by the anchor terminal and forward the live stream to the audience terminal; wherein the live stream includes supplementary enhancement information, the supplementary enhancement information includes the live material identifier, and the audience terminal is used to: perform live display based on the live stream and the corresponding live material information.

[0130] As an optional embodiment, the device 1500 is further used for: Based on the material display instruction, determine the current live broadcast state, where the live broadcast state includes starting material display or terminating material display; The current live broadcast status is pushed to the anchor terminal.

[0131] Embodiment 7 Fig.16 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the method for synchronizing audio and video in live broadcast according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.16As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed in the computer device 10000, such as the program code of the live audio and video synchronization method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0132] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0133] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0134] It should be pointed out that Fig.16 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0135] In this embodiment, the live audio and video synchronization method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0136] Embodiment 8 An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the live audio and video synchronization method in the embodiment are implemented.

[0137] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the live audio and video synchronization method in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0138] Embodiment 9 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0139] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0140] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for synchronizing audio and video during live broadcast, characterized in that: For a viewer terminal, the method comprises: Acquire a live stream, wherein the live stream includes supplementary enhancement information, and the supplementary enhancement information includes a live material identifier; According to the live broadcast material identifier, corresponding live broadcast material information is obtained from a preset queue; wherein the preset queue includes one or more live broadcast material information sent by the server; Determine a live broadcast material card according to the corresponding live broadcast material information; Based on the live broadcast stream and the live broadcast material card, a live broadcast presentation is performed.

2. The method according to claim 1, characterized in that The preset queue is obtained by the following operations: Acquire live broadcast material information sent by the server, where the live broadcast material information carries a timestamp; Comparing the timestamp of the live broadcast material information with the timestamp of the live broadcast material information sent last time; When the timestamp of the live broadcast material information is greater than the timestamp of the live broadcast material information sent last time, obtaining the current length of the preset queue; When the current length of the preset queue is less than a preset threshold, the live broadcast material information is added to the preset queue.

3. The method according to claim 1, characterized in that The live broadcast material information includes material content and material status, and the material status includes: adding, removing, switching or updating; Correspondingly, determining the live broadcast material card according to the corresponding live broadcast material information includes: Determine the material content and material status based on the corresponding live broadcast material information; When the material status is adding or switching, a new live material card is generated based on the material content to be displayed in the live broadcast room; When the material status is removed, based on the material content, remove the live material card corresponding to the live broadcast room; When the material status is updated, the live material card corresponding to the live broadcast room is updated based on the material content.

4. A method for synchronizing audio and video during live broadcast, characterized in that: For an anchor terminal, the method includes: Sending a material display instruction to the server, the material display instruction including a live material identifier, the server being used to: obtain corresponding live material information according to the live material identifier and send it to the viewer terminal; Polling the server to obtain a current polling result, wherein the current polling result includes the live broadcast material identifier; Update the supplementary enhancement information based on the live broadcast material identifier, and add the updated supplementary enhancement information to the live broadcast stream; The live broadcast stream is pushed to the server so as to be forwarded to the viewer terminal through the server, and the viewer terminal is used to perform live broadcast presentation based on the live broadcast stream and the corresponding live broadcast material information.

5. The method according to claim 4, characterized in that The polling result carries a timestamp; based on the live broadcast material identifier, the supplementary enhancement information is updated, including: Compare the timestamp of the current polling result with the timestamp of the previous polling result; When the timestamp of the current polling result is greater than the timestamp of the previous polling result, comparing the live broadcast material identifier with the live broadcast material identifier in the previous polling result; If the two are different, updating the supplementary enhancement information based on the live broadcast material identifier; The current polling result is cached in a preset area.

6. The method according to claim 4, characterized in that Updating the supplementary enhancement information based on the live broadcast material identifier also includes: Determining whether the last round of updating of the supplemental enhancement information is completed; If the previous round of updating is not completed, terminating the previous round of updating, and updating the supplementary enhancement information based on the live broadcast material identifier; When the last round of updating has been completed, the supplementary enhancement information is updated based on the live broadcast material identifier.

7. The method according to claim 4, characterized in that Also includes: Get the current live broadcast status, where the live broadcast status includes starting or stopping material display; When the current live broadcast state is to start material display, performing a polling operation on the server; When the current live broadcast status is to terminate the material display, the polling operation is terminated.

8. A method for synchronizing audio and video during live broadcast, characterized in that: For a server, the method comprises: Receiving a material display instruction sent by an anchor terminal, wherein the material display instruction includes a live broadcast material identifier; Based on the live broadcast material identifier, corresponding live broadcast material information is acquired, and the corresponding live broadcast material information is sent to the viewer terminal; receiving a polling request from the anchor terminal, and sending a polling result to the anchor terminal, wherein the polling result includes the live broadcast material identifier; Receive the live stream provided by the anchor terminal, and forward the live stream to the audience terminal; wherein the live stream includes supplementary enhancement information, the supplementary enhancement information includes the live material identifier, and the audience terminal is used to: perform live display based on the live stream and the corresponding live material information.

9. The method according to claim 8, characterized in that Also includes: Based on the material display instruction, determine the current live broadcast state, where the live broadcast state includes starting material display or terminating material display; The current live broadcast status is pushed to the anchor terminal.

10. A live audio and video synchronization device, characterized in that: For a viewer terminal, the device comprises: A first acquisition module, configured to acquire a live stream, wherein the live stream includes supplementary enhancement information, and the supplementary enhancement information includes a live material identifier; A second acquisition module is used to acquire corresponding live broadcast material information from a preset queue according to the live broadcast material identifier; wherein the preset queue includes one or more live broadcast material information sent by the server; A third acquisition module is used to determine a live broadcast material card according to the corresponding live broadcast material information; A display module is used to perform live display based on the live stream and the live material card.

11. A live audio and video synchronization device, characterized in that: For a host terminal, the device includes: A sending module, configured to send a material display instruction to a server, wherein the material display instruction includes a live material identifier, and the server is configured to: obtain corresponding live material information according to the live material identifier and send the information to a viewer terminal; A polling module, used for polling the server to obtain a current polling result, wherein the current polling result includes the live broadcast material identifier; An inserting module, configured to update the supplementary enhancement information based on the live broadcast material identifier, and add the updated supplementary enhancement information to the live broadcast stream; The streaming module is used to push the live streaming to the server so as to forward it to the viewer terminal through the server. The viewer terminal is used to perform live broadcasting based on the live streaming and the corresponding live broadcast material information.

12. A live audio and video synchronization device, characterized in that: For a server, the device comprises: A receiving module, used to receive a material display instruction sent by an anchor terminal, wherein the material display instruction includes a live broadcast material identifier; A sending module, used for obtaining corresponding live broadcast material information based on the live broadcast material identifier, and sending the corresponding live broadcast material information to the viewer terminal; A sending module, configured to receive a polling request from the anchor terminal and send a polling result to the anchor terminal, wherein the polling result includes the live broadcast material identifier; A forwarding module is used to receive the live stream provided by the anchor terminal and forward the live stream to the audience terminal; wherein the live stream includes supplementary enhancement information, the supplementary enhancement information includes the live material identifier, and the audience terminal is used to: perform live display based on the live stream and the corresponding live material information.

13. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images