Chorusing method and device, and storage medium

Abstract

Embodiments of the present disclosure provide a chorusing method and device, and a storage medium. The method comprises: by means of a first non-chorus role client participating in livestreaming interactive connection, caching received multimedia stream data of chorus role clients among clients participating in livestreaming interactive connection other than said first non-chorus role client; and on the basis of time information comprised in the multimedia stream data of the chorus role clients, aligning the multimedia stream data of the chorus role clients and then playing back the multimedia stream data. By caching and aligning the multimedia stream data of the chorus role clients on the non-chorus role client participating in interactive connection, it can be ensured that a chorusing effect is presented on said non-chorus role client, thereby satisfying a multi-person chorusing requirement in a livestreaming room, and improving the connection experience in the livestreaming room; moreover, since only the multimedia stream data of the chorus role clients is processed, normal real-time interaction between the non-chorus role clients can also be carried out on this basis, and is not affect by chorusing.

Description

Chorus method, device and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202311873350.4 filed on December 29, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] The embodiments of the present disclosure relate to a chorus method, device, and storage medium. Background Art

[0003] Currently, in live broadcast rooms such as video live broadcast rooms or voice live broadcast rooms, the host can conduct real-time live broadcast and live broadcast interactions with guests by establishing a live broadcast session, so that the audience in the live broadcast room can watch the live broadcast interaction content.

[0004] In one scenario, the host or guest participating in the live broadcast can sing a song, while other users connected to the microphone usually do not participate in the singing. If a multi-person chorus is performed, due to factors such as the network and device performance, there may be a certain delay in any party receiving the multimedia stream data of the other party, and the delay may also be uneven, resulting in a poor chorus effect and unable to meet the needs of multi-person chorus in the chat room. Summary of the Invention

[0005] Embodiments of the present disclosure provide a chorus method, device, and storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides a chorus method, which is applied to a first non-chorus role client participating in a live broadcast and a microphone connection. The method includes:

[0007] Receiving multimedia stream data of other clients participating in the live broadcast and microphone connection except the first non-chorus role client;

[0008] caching the multimedia streaming data of the chorus role client among the multimedia streaming data of the other clients;

[0009] According to the time information included in the multimedia streaming data of each chorus role client, the multimedia streaming data of each chorus role client is aligned and then played.

[0010] In a second aspect, an embodiment of the present disclosure provides a chorus method, which is applied to a lead singing role client participating in a live broadcast, and the method includes:

[0011] Receive multimedia streaming data from other clients participating in the live broadcast except the lead singer client, and play the multimedia streaming data of the other clients in silence;

[0012] Collecting multimedia stream data of the lead singer role client during the singing process, and adding time information to the multimedia stream data of the lead singer role client according to the singing progress;

[0013] The multimedia stream data of the lead singer role client with added time information is transmitted to the other clients in real time.

[0014] In a third aspect, an embodiment of the present disclosure provides a chorus method, which is applied to a client that plays the first chorus role in a live broadcast, and the method includes:

[0015] Receiving multimedia stream data from other clients participating in the live broadcast except the first chorus role client;

[0016] Playing the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and playing the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in silence;

[0017] Collecting multimedia streaming data of the first chorus role client during the singing process of following the lead singer, and adding time information to the multimedia streaming data of the first chorus role client according to the singing progress;

[0018] The multimedia stream data of the first chorus role client with added time information is transmitted to the other clients in real time.

[0019] In a fourth aspect, an embodiment of the present disclosure provides a chorus device, comprising:

[0020] A receiving unit, configured to receive multimedia stream data of other clients participating in the live broadcast and microphone connection except the first non-chorus role client;

[0021] a processing unit, configured to cache multimedia streaming data of the chorus role client among the multimedia streaming data of the other clients;

[0022] The playing unit is used to align the multimedia stream data of each chorus role client according to the time information included in the multimedia stream data of each chorus role client and then play it.

[0023] In a fifth aspect, an embodiment of the present disclosure provides a chorus device, including:

[0024] A receiving unit, configured to receive multimedia stream data from clients other than the lead singer client participating in the live broadcast;

[0025] A playing unit, configured to play the multimedia stream data of the other clients in silence;

[0026] A collection unit, configured to collect multimedia stream data of the lead singer client during a singing process, and add time information to the multimedia stream data of the lead singer client according to a singing progress;

[0027] The sending unit is used to transmit the multimedia stream data of the lead singer role client with added time information in real time to other clients participating in the joint microphone connection.

[0028] In a sixth aspect, an embodiment of the present disclosure provides a chorus device, including:

[0029] A receiving unit, configured to receive multimedia stream data of other clients participating in the live broadcast except the first chorus role client;

[0030] A playback unit, configured to play the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and to play the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in a silent state;

[0031] a collection unit, configured to collect multimedia stream data of the first chorus role client during the singing process following the lead singer, and add time information to the multimedia stream data of the first chorus role client according to the singing progress;

[0032] The sending unit is used to transmit the multimedia stream data of the first chorus role client with added time information to the other clients in real time.

[0033] In a seventh aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0034] The memory stores computer-executable instructions;

[0035] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the chorus method described in the first aspect and various possible designs of the first aspect, or the chorus method described in the second aspect and various possible designs of the second aspect, or the chorus method described in the third aspect and various possible designs of the third aspect.

[0036] In an eighth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the chorus method described in the first aspect and various possible designs of the first aspect, or the chorus method described in the second aspect and various possible designs of the second aspect, or the chorus method described in the third aspect and various possible designs of the third aspect is implemented.

[0037] In a ninth aspect, an embodiment of the present disclosure provides a computer program product comprising computer-executable instructions. When a processor executes the computer-executable instructions, the chorus method described in the first aspect and various possible designs of the first aspect, or the chorus method described in the second aspect and various possible designs of the second aspect, or the chorus method described in the third aspect and various possible designs of the third aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] FIG1 is a scene example diagram of a chorus method provided by an embodiment of the present disclosure;

[0040] FIG2 is a schematic flow chart of a chorus method provided by an embodiment of the present disclosure;

[0041] FIG3 is a schematic diagram of a chorus method provided by an embodiment of the present disclosure;

[0042] FIG4 is a schematic flow chart of a chorus method provided by another embodiment of the present disclosure;

[0043] FIG5 is a schematic flow chart of a chorus method provided by another embodiment of the present disclosure;

[0044] FIG6 is a structural block diagram of a chorus device provided in an embodiment of the present disclosure;

[0045] FIG7 is a structural block diagram of a chorus device provided by another embodiment of the present disclosure;

[0046] FIG8 is a structural block diagram of a chorus device provided by another embodiment of the present disclosure;

[0047] FIG9 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0049] In one scenario, the host or guest in the live broadcast room can sing a song, while other users connected to the microphone usually do not participate in the singing. If a multi-person chorus is performed, due to factors such as the network and device performance, there may be a certain delay when any party receives the multimedia streaming data of the other party, and the delay may also be uneven. From the perspective of non-chorus parties or audiences, the progress of each singer may be different, and it is impossible to present a good chorus effect, which cannot meet the needs of multi-person chorus in the live broadcast room.

[0050] In order to solve the above technical problems, the embodiment of the present disclosure can divide the clients participating in the live broadcast into chorus role clients and non-chorus role clients from the perspective of roles. The chorus role clients may include the lead singer (or main singer) role clients and the secondary singer role clients.

[0051] Among them, after the non-chorus role client participating in the live broadcast receives the multimedia stream data of other clients participating in the live broadcast except the non-chorus role client, the multimedia stream data of the chorus role client can be cached and aligned, thereby ensuring that the chorus effect is presented on the non-chorus role client participating in the live broadcast, meeting the needs of multi-person chorus in the live broadcast room, and improving the connection experience of the live broadcast room. Moreover, since the above-mentioned processing is only performed on the multimedia stream data of the chorus role client, normal real-time interaction between the non-chorus role clients can also be carried out on this basis without being affected by the chorus.

[0052] After receiving the multimedia stream data from other clients participating in the live broadcast except the lead singer client, the lead singer client plays the multimedia stream data of each other client in silence; collects the local multimedia stream data during the singing process, and adds time information to the local multimedia stream data according to the progress of the singing; transmits the local multimedia stream data with the added time information to each other client in real time, realizing the lead singing function of the local end without being interfered by the other clients participating in the live broadcast; at the same time, the local multimedia stream data also provides a chorus basis for the chorus client, so that the chorus client user can sing along, and the addition of time information also facilitates the caching and alignment operations of non-chorus role clients.

[0053] After receiving the multimedia stream data of other clients participating in the live broadcast except the chorus role client, the chorus role client plays the multimedia stream data of the lead singer role client among the multimedia stream data of other clients, and plays the multimedia stream data of non-chorus role clients and other chorus role clients in silence; collects the local multimedia stream data in the process of following the lead singer, and adds time information to the local multimedia stream data according to the progress of the singing; transmits the local multimedia stream data with the added time information in real time to the other clients participating in the live broadcast, so that the client can sing along with the lead singer without being interfered by the other clients participating in the live broadcast. At the same time, adding time information also facilitates the non-chorus role clients to perform caching and alignment operations.

[0054] The chorus method provided in the above embodiment is applied to the scenario shown in Figure 1. The live broadcast room can be a video live broadcast room or a voice live broadcast room. From the perspective of the role, the live broadcast room may include clients participating in the live broadcast and audience clients not participating in the live broadcast. The clients participating in the live broadcast include chorus role clients and non-chorus role clients. The chorus role clients are clients participating in the live broadcast chorus, and may include lead singer (or main singer) role clients and chorus role clients. The non-chorus role clients, lead singer role clients and chorus role clients participating in the live broadcast can respectively execute the above-mentioned corresponding live broadcast room chorus methods.

[0055] It should be noted that the user information and data involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0056] The chorus method disclosed herein will be described in detail below with reference to specific embodiments.

[0057] With reference to Figure 2, Figure 2 is a flow chart of the chorus method provided by an embodiment of the present disclosure. In the chorus scenario of the live broadcast room, the live broadcast room can be a video live broadcast room or a voice live broadcast room. From the perspective of the role, the live broadcast room may include clients participating in the microphone connection and audience clients not participating in the microphone connection, wherein the clients participating in the microphone connection include chorus role clients and non-chorus role clients, and the chorus role clients are clients participating in the microphone connection chorus, and may include lead singer (or main singer) role clients and secondary singer role clients; from another perspective, the live broadcast room may include host clients and guest clients participating in the microphone connection, as well as audience clients not participating in the microphone connection, wherein the host client and the guest client may be any of the above roles (wherein there is only one lead singer role client, if the host client or a guest client is the lead singer role client, then the other clients participating in the microphone connection cannot be the lead singer role client).

[0058] The chorus method of this embodiment can be applied to a first non-chorus role client participating in a live broadcast and a microphone connection, wherein the first non-chorus role client is any one of the non-chorus role clients participating in the live broadcast and a microphone connection. The chorus method includes:

[0059] S201: Receive multimedia stream data of other clients participating in the live broadcast except the first non-chorus role client.

[0060] In this embodiment, each client participating in the live broadcast and microphone connection can receive multimedia streaming data from other clients in addition to itself, but different roles process the multimedia streaming data of other clients differently. Each client participating in the microphone connection can upload its own multimedia streaming data to the server, and through the server, each client participating in the microphone connection can receive the multimedia streaming data of other clients.

[0061] The first non-chorus role client may receive multimedia stream data of the chorus role client, including multimedia stream data of the lead vocal role client and multimedia stream data of the secondary vocal role client.

[0062] In addition, if there are multiple non-chorus role clients participating in the live broadcast, the first non-chorus role client can also receive multimedia stream data from other second non-chorus role clients.

[0063] Among them, in a video live broadcast room, the multimedia stream data can be audio and video stream data; in a voice live broadcast room, the media stream data can be audio stream data.

[0064] S202: Cache the multimedia streaming data of the chorus role client in the multimedia streaming data of the other clients.

[0065] In this embodiment, since the chorus role client needs to sing along with the multimedia stream data of the lead role client, the multimedia stream data of the chorus role client is somewhat delayed relative to the multimedia stream data of the lead role client. For example, as shown in FIG3 , the multimedia stream data of the lead role client has been sung to the 10th second of the song, corresponding to the data in the gray portion of the multimedia stream data of the lead role client in FIG3 , while the multimedia stream data of the chorus role client 1 may have only been sung to the 8th second of the song, corresponding to the data in the gray portion of the multimedia stream data of the chorus role client 1 in FIG3 . The singing progress of the multimedia stream data of different chorus role clients may also be different. The multimedia stream data of another chorus role client 2 may be sung to the 9th second of the song, corresponding to the data in the gray portion of the multimedia stream data of the chorus role client 2 in FIG3 . In order to present a chorus effect on non-chorus role clients, the non-chorus role clients may cache the multimedia stream data of the chorus role clients to perform the following alignment operation.

[0066] As for the multimedia streaming data of other second non-chorus role clients, since they are not related to the chorus and do not require alignment operations, they do not need to be cached. Instead, they can be played directly in real time after receiving the multimedia streaming data of other second non-chorus role clients.

[0067] S203: aligning the multimedia streaming data of each chorus role client according to the time information included in the multimedia streaming data of each chorus role client and then playing the multimedia streaming data.

[0068] In this embodiment, each chorus role client may add time information, such as the singing progress of each frame, to its respective multimedia stream data to facilitate alignment with the non-chorus role clients. The first non-chorus role client may then align the cached multimedia stream data of each chorus role client based on the time information included in the multimedia stream data of each chorus role client before playing it. The alignment method is not limited in this embodiment and any known method may be used.

[0069] Optionally, the first non-chorus role client can obtain multimedia data of the same singing progress from the multimedia streaming data of each chorus role client according to the time information included in the multimedia streaming data of each chorus role client for synchronous rendering and / or playback. For example, the multimedia data corresponding to the 0.1 second singing progress is obtained from the multimedia streaming data of each chorus role client for synchronous rendering and / or playback, and then the multimedia data corresponding to the 0.2 second singing progress is obtained for synchronous rendering and / or playback, and so on, thereby achieving alignment of the multimedia streaming data of each chorus role client on the first non-chorus role client and presenting a chorus effect.

[0070] The chorus method of this embodiment receives multimedia stream data of other clients participating in the live broadcast except the first non-chorus role client through the first non-chorus role client participating in the live broadcast; caches the multimedia stream data of the chorus role client in the multimedia stream data of other clients; and aligns the multimedia stream data of each chorus role client according to the time information included in the multimedia stream data of each chorus role client before playing. By caching and aligning the multimedia stream data of the chorus role client on the non-chorus role client participating in the live broadcast, it can be ensured that the chorus effect is presented on the non-chorus role client participating in the live broadcast, meeting the needs of multiple people singing in the live broadcast room and improving the connection experience of the live broadcast room. Moreover, since the above-mentioned processing is only performed on the multimedia stream data of the chorus role client, normal real-time interaction between the non-chorus role clients can also be carried out on this basis without being affected by the chorus.

[0071] On the basis of any of the above embodiments, since only the multimedia stream data of the chorus role in the multimedia stream data of each other client participating in the live broadcast and the chorus role is cached, aligned and played, and the multimedia stream data of the other second non-chorus role clients in the multimedia stream data of the other clients can be played in real time, as shown in Figure 3, the first non-chorus role client can hear in real time, or hear and see in real time the content of the multimedia stream data of the other second non-chorus role clients; In addition, the first non-chorus role client can also collect the multimedia stream data of the first non-chorus role client at this end and transmit it to each other client participating in the live broadcast and the other second non-chorus role clients can also play the received multimedia stream data of the first non-chorus role client in real time, thus ensuring normal real-time interaction between the non-chorus role clients, and the chorus role client can play the received multimedia stream data of the first non-chorus role client in silence, so that the multimedia stream data of the first non-chorus role client does not affect the singing of the chorus role client user. The transmission process can be implemented by the server or by other means, which is not limited in this embodiment.

[0072] On the basis of any of the above embodiments, since the first non-chorus role client can be a guest client of a non-chorus role or a host client of a non-chorus role, if the first non-chorus role client is a guest client of a non-chorus role, the processing of the above embodiment can be performed; if the first non-chorus role client is a host client of a non-chorus role, in addition to the processing of the above embodiment, the following process needs to be performed:

[0073] The aligned multimedia stream data of each chorus role client, the multimedia stream data of other non-chorus role clients participating in the joint microphone connection, and the multimedia stream data of the first non-chorus role client are merged, and the merged multimedia stream data is transmitted to the audience client.

[0074] In this embodiment, since the anchor client is also responsible for pushing the live broadcast room to the audience client, for the anchor client of the first non-chorus role, the aligned multimedia stream data of each chorus role client, the multimedia stream data of other non-chorus role clients participating in the microphone connection, and the multimedia stream data of the local end can be merged, and the merged multimedia stream data can be transmitted to the audience client, so that the audience client can also hear the chorus effect and the normal real-time interaction between the non-chorus role clients.

[0075] On the basis of any of the above embodiments, during the chorus process, the first non-chorus role client can also join the chorus, that is, switch from a non-chorus role to a chorus role (since the lead role client already exists, the non-chorus role generally cannot be directly switched to the lead role). A chorus joining control can be provided in the live broadcast room interface of the first non-chorus role client. The user can trigger the chorus joining command after triggering the chorus joining control, thereby switching from the non-chorus role client to the chorus role client. Accordingly, the above processing of the first non-chorus role client is no longer executed, but the processing of the chorus role client is executed, which is specifically as follows:

[0076] In response to the instruction to join the chorus, the role is switched from a non-chorus role to a chorus role;

[0077] Play the multimedia stream data of the lead singing role client among the multimedia stream data of all other clients participating in the live broadcast and play the multimedia stream data of the second non-chorus role client and other chorus role clients in silence;

[0078] Collect the local multimedia stream data during the lead singer's singing, and add time information to the local multimedia stream data according to the singing progress;

[0079] The local multimedia stream data with added time information is transmitted in real time to other clients participating in the joint microphone connection.

[0080] Among them, the processing process after switching to the chorus role can be found in the following embodiment of the live broadcast chorus method on the chorus role client side, which will not be repeated here.

[0081] In addition, at the end of the chorus, or upon receiving a chorus end instruction from the lead singer role client or the host client, the first non-chorus role client can resume the normal microphone connection process, that is, it will no longer cache and align the multimedia stream data of the chorus role client, but can play the multimedia stream data of all other clients participating in the live broadcast connection in real time.

[0082] In addition, if the first non-chorus role client exits the microphone connection, it can switch to the audience client, or exit the live broadcast room, which will not be repeated here.

[0083] Referring to Figure 4, Figure 4 is a flow chart of a chorus method provided by an embodiment of the present disclosure. The method of this embodiment can be applied to a lead singing role client participating in a live broadcast, and the chorus method includes:

[0084] S301, receiving multimedia streaming data of other clients participating in the live broadcast except the lead singing role client, and playing the multimedia streaming data of the other clients in silence.

[0085] In this embodiment, the lead vocalist client can receive multimedia streaming data from the chorus vocalist client. In addition, if there are non-chorus vocalist clients participating in the live broadcast room, the multimedia streaming data of the non-chorus vocalist clients can also be received. In order to prevent the multimedia streaming data of clients other than the lead vocalist client from interfering with the singing of the lead vocalist client user, the multimedia streaming data of other clients can be played in silence. Specifically, in a video live broadcast room (such as a video live broadcast room), only the video image of the multimedia streaming data can be played, and the audio can be muted; while in a voice live broadcast room (such as a voice live broadcast room), the audio can be muted.

[0086] S302, collecting multimedia streaming data of the lead singer role client during the singing process, and adding time information to the multimedia streaming data of the lead singer role client according to the singing progress.

[0087] In this embodiment, the lead singing role client can collect the multimedia streaming data of the lead singing role client during the singing process of the local user. Optionally, the multimedia streaming data of the lead singing role client can also include accompaniment.

[0088] In order to facilitate alignment of other clients such as non-chorus role clients to present a chorus effect, the lead role client can add time information to the multimedia stream data of the lead role client, such as the singing progress corresponding to each frame, etc., which can be expressed as a timestamp or as a percentage. Optionally, the time information can be obtained according to the playback progress of the accompaniment.

[0089] S303: Transmitting the multimedia stream data of the lead singer role client with added time information to the other clients in real time.

[0090] In this embodiment, the lead vocalist client can transmit the multimedia stream data of the lead vocalist client with time information added to it in real time to the other clients participating in the microphone connection. This data can be played by the secondary vocalist client for singing along, and can also be cached, aligned, and played by the non-chorus client. The transmission process can be implemented by the server or by other means, which is not limited in this embodiment.

[0091] On the basis of any of the above embodiments, the lead singing role client can be a guest client of the lead singing role, or it can be a host client of the lead singing role. If the lead singing role client is a guest client of the lead singing role, the processing of the above embodiment can be executed; if the lead singing role client is a host client of the lead singing role, in addition to the processing of the above embodiment, the following process needs to be executed:

[0092] caching the multimedia stream data of the client of the secondary vocal role and the multimedia stream data of the client of the lead vocal role among the multimedia stream data of each other client participating in the joint microphone connection;

[0093] Aligning the multimedia stream data of the chorus role client and the multimedia stream data of the lead vocal role client according to the time information to obtain aligned multimedia stream data of each chorus role client;

[0094] The aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the live broadcast are merged, and the merged multimedia stream data is transmitted to the audience client.

[0095] In this embodiment, since the anchor client is also responsible for pushing the live broadcast room to the audience client, for the anchor client of the lead singing role, in order to enable the audience client to also present the chorus effect, the anchor client of the lead singing role also needs to cache and align the multimedia stream data of each chorus role client, including caching the multimedia stream data of the secondary singing role client and the multimedia stream data of the lead singing role client, and further aligning the multimedia stream data of the secondary singing role client and the multimedia stream data of the lead singing role client according to time information to obtain the aligned multimedia stream data of each chorus role client, and then merge the aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the microphone connection, and transmit the merged multimedia stream data to the audience client, so that the audience client can also hear the chorus effect and the normal real-time interaction between the non-chorus role clients at the same time.

[0096] Based on any of the above embodiments, the lead singer role client can also control the end of the chorus. Optionally, a chorus end control can be set in the live broadcast room interface of the lead singer role client. The user can trigger the chorus end instruction by triggering the chorus end control, and the lead singer role client can resume the normal microphone connection process. The lead singer role client also sends a chorus end instruction to other clients participating in the live broadcast microphone connection, so that each other client also ends the chorus and resumes the normal microphone connection process.

[0097] Based on any of the above embodiments, before the chorus begins, any client participating in the live broadcast in the live broadcast room can initiate the chorus, for example, by requesting a song, and then the client switches to the lead singing role client; in addition, the host client can also specify the lead singing role client, or determine the lead singing role client through other means.

[0098] Based on any of the above embodiments, during the singing process, the lead singing role client generally cannot switch to other roles; of course, the lead singing role client may not be restricted from switching to other roles. The lead singing role client may specify or request other clients to become the lead singing role client, and then the two clients may exchange roles. Of course, other means may also be used to enable the lead singing role client to switch to other roles, which is not restricted in this embodiment.

[0099] The chorus method provided in the above embodiment is that the lead role client receives the multimedia stream data of other clients participating in the live broadcast except the lead role client, and plays the multimedia stream data of each other client in silence; collects the multimedia stream data of the lead role client during the singing process, and adds time information to the multimedia stream data of the lead role client according to the progress of the singing; and transmits the multimedia stream data of the lead role client with the added time information to each other client participating in the live broadcast in real time. The lead role client can realize the lead singing function of its own end through the above process without being interfered by the other clients participating in the live broadcast. At the same time, its own multimedia stream data also provides a chorus basis for the secondary role client, so that the secondary role client user can sing along, and by adding time information, it is also convenient for non-chorus role clients to perform caching and alignment operations, which can ensure that the chorus effect is presented on non-chorus role clients, meeting the needs of multiple people singing in the live broadcast room and improving the connection experience of the live broadcast room.

[0100] Referring to Figure 5, Figure 5 is a flow chart of a chorus method provided by an embodiment of the present disclosure. The method of this embodiment can be applied to the first chorus role client participating in a live broadcast and microphone connection. The chorus method includes:

[0101] S401: Receive multimedia stream data of other clients participating in the live broadcast except the first chorus role client.

[0102] In this embodiment, the first chorus role client can receive multimedia stream data from the lead role client and other second chorus role clients; in addition, if there are non-chorus role clients participating in the live broadcast room, the multimedia stream data of the non-chorus role clients can also be received.

[0103] S402: Play the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and play the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in silence.

[0104] In this embodiment, since the first chorus role client needs to sing along, the first chorus role client can play the multimedia stream data of the lead role client. In order to avoid interference from other clients participating in the live broadcast, the chorus role client can mute the multimedia stream data of other clients except the multimedia stream data of the lead role client, including muting the multimedia stream data of non-chorus role clients and other chorus role clients. Specifically, in a video live broadcast room (such as a video live broadcast room), only the video picture of the multimedia stream data can be played, and the audio can be muted; and in a voice live broadcast room, the audio can be muted.

[0105] S403: Collect multimedia streaming data of the first chorus role client during the singing process of following the lead singer, and add time information to the multimedia streaming data of the first chorus role client according to the singing progress.

[0106] In this embodiment, the first chorus role client can collect the multimedia stream data of the first chorus role client during the singing-along process of the user of the first chorus role client. In order to facilitate alignment of other clients such as non-chorus role clients to present a chorus effect, the first chorus role client can add time information to the multimedia stream data of the first chorus role client, such as the singing progress corresponding to each frame, etc., which can be expressed as a timestamp or as a percentage. Optionally, the time information can be obtained based on the multimedia stream data of the lead singer role client.

[0107] S404: Transmit the multimedia stream data of the first chorus role client with added time information to the other clients in real time.

[0108] In this embodiment, the first chorus role client can transmit the multimedia stream data of the first chorus role client with the time information added in real time to each other client participating in the live broadcast, so that the non-chorus role clients can cache, align, and play it. The transmission process can be implemented by the server or through other means, which is not limited in this embodiment.

[0109] Based on any of the above embodiments, the first secondary vocal client can be a guest client for the secondary vocal role, or can be a host client for the secondary vocal role. If the first secondary vocal client is a guest client for the secondary vocal role, the processing of the above embodiment can be performed; if the first secondary vocal client is a host client for the secondary vocal role, in addition to the processing of the above embodiment, the following process needs to be performed:

[0110] caching the multimedia stream data of the lead vocalist client and the second chorus vocalist client, as well as the multimedia stream data of the first chorus vocalist client, among the multimedia stream data of the other clients participating in the live broadcast;

[0111] Aligning the multimedia stream data of the lead vocalist client and the second chorus vocalist client, as well as the multimedia stream data of the first chorus vocalist client, according to the time information, to obtain aligned multimedia stream data of each chorus vocalist client;

[0112] The aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the live broadcast are merged, and the merged multimedia stream data is transmitted to the audience client.

[0113] In this embodiment, since the anchor client is also responsible for pushing the live broadcast room to the audience client, for the anchor client of the chorus role, in order to enable the audience client to also present the chorus effect, the anchor client of the chorus role also needs to cache and align the multimedia stream data of each chorus role client, including caching the multimedia stream data of the lead role client and other second chorus role clients in the multimedia stream data of each other client participating in the live broadcast and the multimedia stream data of the local end (the first chorus role client), and aligning based on time information to obtain the aligned multimedia stream data of each chorus role client, and then merge the aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the live broadcast and the merged multimedia stream data to the audience client, so that the audience client can also hear the chorus effect and the normal real-time interaction between the non-chorus role clients.

[0114] Based on any of the above embodiments, during the singing process, the first chorus role client can also exit the chorus. Optionally, a chorus exit control can be set in the live broadcast room interface of the first chorus role client. The user can trigger the chorus exit instruction by triggering the chorus exit control. The first chorus role client can respond to the chorus exit instruction and switch from the chorus role to the non-chorus role, and then execute the above-mentioned processing of the non-chorus role client, which may specifically include:

[0115] caching the multimedia stream data of the chorus role client among the multimedia stream data of all other clients participating in the live broadcast and microphone connection;

[0116] According to the time information included in the multimedia streaming data of each chorus role client, the multimedia streaming data of each chorus role client is aligned and then played.

[0117] For details, please refer to the above-mentioned live broadcast room chorus method embodiment on the non-chorus role client side, which will not be described here.

[0118] In the chorus method provided in the above embodiment, the first chorus role client receives multimedia streaming data from other clients participating in the live broadcast and microphone connection except the first chorus role client; plays the multimedia streaming data of the lead role client among the multimedia streaming data of the other clients, and plays the multimedia streaming data of the non-chorus role clients and other second chorus role clients in silence; collects the multimedia streaming data of the first chorus role client during the singing process following the lead singer, and adds time information to the multimedia streaming data of the first chorus role client according to the singing progress; and transmits the multimedia streaming data of the first chorus role client with the added time information in real time to each client participating in the microphone connection. Through the above process, the chorus role client can sing along with the lead singer without being interfered with by the other clients participating in the microphone connection. At the same time, by adding time information, it is also convenient for the non-chorus role client to perform caching and alignment operations, which can ensure that the chorus effect is presented on the non-chorus role client, meeting the needs of multiple choruses in the chat room and improving the chat room connection experience.

[0119] Corresponding to the chorus method for the non-chorus client side of the above embodiment, FIG8 is a block diagram of the structure of the chorus device provided by the embodiment of the present disclosure. For ease of illustration, only the parts relevant to the embodiment of the present disclosure are shown. Referring to FIG6 , the chorus device 600 includes: a receiving unit 601, a processing unit 602, and a playback unit 603.

[0120] The receiving unit 601 is configured to receive multimedia stream data of other clients participating in the live broadcast except the first non-chorus role client;

[0121] The processing unit 602 is configured to cache multimedia streaming data of the chorus role client among the multimedia streaming data of the other clients;

[0122] The playing unit 603 is configured to align the multimedia stream data of each chorus role client according to the time information included in the multimedia stream data of each chorus role client and then play the aligned multimedia stream data.

[0123] In one or more embodiments of the present disclosure, when aligning the multimedia stream data of each chorus role client according to the time information included in the multimedia stream data of each chorus role client and then playing it, the playback unit 603 is configured to:

[0124] According to the time information included in the multimedia streaming data of each chorus role client, multimedia data with the same singing progress is obtained from the multimedia streaming data of each chorus role client for synchronous rendering and / or playback.

[0125] In one or more embodiments of the present disclosure, the playback unit 603 is further configured to play in real time the multimedia streaming data of other non-chorus role clients in the multimedia streaming data of the other clients; and / or

[0126] The device further includes a collection unit 604 for collecting multimedia stream data of the first non-chorus role client;

[0127] The device further includes a sending unit 605, configured to transmit the multimedia stream data of the first non-chorus role client to the other clients in real time.

[0128] In one or more embodiments of the present disclosure, if the first non-chorus role client is a non-chorus role anchor client, the sending unit 605 is also used to merge the aligned multimedia stream data of each chorus role client, the multimedia stream data of the second non-chorus role client participating in the live broadcast, and the multimedia stream data of the first non-chorus role client, and transmit the merged multimedia stream data to the audience client.

[0129] In one or more embodiments of the present disclosure, the processing unit 602 is further configured to, in response to a chorus joining instruction, switch roles from a non-chorus role to a chorus role;

[0130] The playback unit 603 is further configured to play the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and to play the multimedia streaming data of the second non-chorus vocalist client and other chorus vocalist client in mute mode;

[0131] The collection unit 604 is further configured to collect local multimedia stream data during the singing process of the lead singer and add time information to the local multimedia stream data according to the singing progress;

[0132] The sending unit 605 is further configured to transmit the local multimedia stream data with time information added thereto to the other clients in real time.

[0133] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0134] Corresponding to the chorus method for the lead vocalist client described in the preceding embodiment, FIG7 is a block diagram of the chorus device provided in the present embodiment. For ease of illustration, only the portions relevant to the present embodiment are shown. Referring to FIG7 , the chorus device 700 includes a receiving unit 701, a playback unit 702, a collection unit 703, and a sending unit 704.

[0135] The receiving unit 701 is configured to receive multimedia stream data from other clients participating in the live broadcast except for the first chorus role client;

[0136] The playing unit 702 is configured to play the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and play the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in a muted state;

[0137] The collecting unit 703 is used to collect the multimedia streaming data of the lead singer client during the singing process, and add time information to the multimedia streaming data of the lead singer client according to the singing progress;

[0138] The sending unit 704 is used to transmit the multimedia stream data of the lead singer role client with added time information to the other clients in real time.

[0139] In one or more embodiments of the present disclosure, if the lead singing role client is a host singing role client, the device further includes a processing unit 705, configured to cache the multimedia stream data of the secondary singing role client and the multimedia stream data of the lead singing role client in the multimedia stream data of the other clients;

[0140] The sending unit 704 is further configured to align the multimedia stream data of the chorus role client and the multimedia stream data of the lead role client according to time information to obtain aligned multimedia stream data of each chorus role client; merge the aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the live broadcast, and transmit the merged multimedia stream data to the audience client.

[0141] In one or more embodiments of the present disclosure, the processing unit 705 is further configured to, in response to a chorus end instruction, resume a regular microphone connection process;

[0142] The sending unit 704 is further configured to send a chorus end instruction to each other client participating in the joint microphone connection, so that each other client can resume a normal joint microphone connection process.

[0143] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0144] Corresponding to the chorus method for the chorus role client side in the above embodiment, FIG8 is a block diagram of the structure of the chorus device provided in the embodiment of the present disclosure. For ease of illustration, only the portions relevant to the embodiment of the present disclosure are shown. Referring to FIG8 , the chorus device 800 includes: a receiving unit 801, a playback unit 802, a collection unit 803, and a sending unit 804.

[0145] The receiving unit 801 is configured to receive multimedia stream data from other clients participating in the live broadcast except for the first chorus role client;

[0146] The playing unit 802 is configured to play the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and play the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in a muted state;

[0147] The collecting unit 803 is used to collect multimedia streaming data of the first chorus role client during the singing process of following the lead singer, and add time information to the multimedia streaming data of the first chorus role client according to the singing progress;

[0148] The sending unit 804 is configured to transmit the multimedia stream data of the first chorus role client with added time information to the other clients in real time.

[0149] In one or more embodiments of the present disclosure, if the first chorus role client is a host client of a lead role, the device further includes a processing unit 805, configured to cache the multimedia stream data of the lead role client and the multimedia stream data of each of the other second chorus role clients, as well as the multimedia stream data of the first chorus role client, among the multimedia stream data of the other clients;

[0150] The sending unit 804 is further configured to align the multimedia stream data of the lead vocalist client and each second chorus vocalist client, as well as the multimedia stream data of the first chorus vocalist client, according to time information, to obtain aligned multimedia stream data of each chorus vocalist client; merge the aligned multimedia stream data of each chorus vocalist client and the multimedia stream data of the non-chorus vocalist client participating in the live broadcast, and transmit the merged multimedia stream data to the audience client.

[0151] In one or more embodiments of the present disclosure, the processing unit 805 is further configured to, in response to an exit chorus instruction, switch roles from a chorus role to a non-chorus role;

[0152] The processing unit 805 is further configured to cache the multimedia streaming data of the chorus role client in the multimedia streaming data of the other clients;

[0153] The playback unit 802 is further configured to align the multimedia streaming data of each chorus role client according to the time information included in the multimedia streaming data of each chorus role client and then play the aligned multimedia streaming data.

[0154] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0155] Referring to FIG9 , a schematic diagram of the structure of an electronic device 900 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 900 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG9 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0156] As shown in Figure 9, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0157] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG9 shows an electronic device 900 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0158] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of any one side of the above-mentioned embodiment of the present disclosure are performed.

[0159] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0160] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0161] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0162] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0164] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0165] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0166] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0167] In a first aspect, according to one or more embodiments of the present disclosure, a chorus method is provided, which is applied to a first non-chorus role client participating in a live broadcast and a microphone connection. The method includes:

[0168] Receiving multimedia stream data of other clients participating in the live broadcast and microphone connection except the first non-chorus role client;

[0169] caching the multimedia streaming data of the chorus role client among the multimedia streaming data of the other clients;

[0170] According to the time information included in the multimedia streaming data of each chorus role client, the multimedia streaming data of each chorus role client is aligned and then played.

[0171] According to one or more embodiments of the present disclosure, aligning the multimedia streaming data of each chorus role client and then playing it according to the time information included in the multimedia streaming data of each chorus role client includes:

[0172] According to the time information included in the multimedia streaming data of each chorus role client, multimedia data of the same singing progress is obtained from the multimedia streaming data of each chorus role client for synchronous rendering and / or playback.

[0173] According to one or more embodiments of the present disclosure, the method further includes:

[0174] Playing in real time the multimedia streaming data of the second non-chorus role client among the multimedia streaming data of the other clients; and / or

[0175] Collect multimedia stream data of the first non-chorus role client and transmit it to the other clients in real time.

[0176] According to one or more embodiments of the present disclosure, the method further includes:

[0177] If the first non-chorus role client is a non-chorus role anchor client, the aligned multimedia stream data of each chorus role client, the multimedia stream data of the second non-chorus role client participating in the live broadcast, and the multimedia stream data of the first non-chorus role client are merged, and the merged multimedia stream data is transmitted to the audience client.

[0178] According to one or more embodiments of the present disclosure, the method further includes:

[0179] In response to the instruction to join the chorus, the role is switched from a non-chorus role to a chorus role;

[0180] Playing the multimedia streaming data of the lead singing role client among the multimedia streaming data of the other clients, and playing the multimedia streaming data of the second non-chorus role client and other chorus role clients in silence;

[0181] Collect the local multimedia stream data during the lead singer's singing, and add time information to the local multimedia stream data according to the singing progress;

[0182] The local multimedia stream data with added time information is transmitted to the other clients in real time.

[0183] In a second aspect, according to one or more embodiments of the present disclosure, a chorus method is provided, which is applied to a lead singing role client participating in a live broadcast, and the method includes:

[0184] Receive multimedia streaming data from other clients participating in the live broadcast except the lead singer client, and play the multimedia streaming data of the other clients in silence;

[0185] Collecting multimedia stream data of the lead singer role client during the singing process, and adding time information to the multimedia stream data of the lead singer role client according to the singing progress;

[0186] The multimedia stream data of the lead singer role client with added time information is transmitted to the other clients in real time.

[0187] According to one or more embodiments of the present disclosure, the method further includes:

[0188] If the lead singing role client is the anchor client of the lead singing role, the multimedia streaming data of the secondary singing role client and the multimedia streaming data of the lead singing role client in the multimedia streaming data of the other clients are cached;

[0189] Aligning the multimedia stream data of the chorus role client and the multimedia stream data of the lead vocal role client according to the time information to obtain aligned multimedia stream data of each chorus role client;

[0190] The aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the joint microphone connection are merged, and the merged multimedia stream data is transmitted to the audience client.

[0191] According to one or more embodiments of the present disclosure, the method further includes:

[0192] In response to the chorus end instruction, the normal microphone connection process is resumed;

[0193] Send a chorus end instruction to each other client participating in the joint microphone connection, so that each other client can resume the normal joint microphone connection process.

[0194] In a third aspect, according to one or more embodiments of the present disclosure, a chorus method is provided, which is applied to a first chorus role client participating in a live broadcast, the method comprising:

[0195] Receiving multimedia stream data from other clients participating in the live broadcast except the first chorus role client;

[0196] Playing the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and playing the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in silence;

[0197] Collecting multimedia streaming data of the first chorus role client during the singing process of following the lead singer, and adding time information to the multimedia streaming data of the first chorus role client according to the singing progress;

[0198] The multimedia stream data of the first chorus role client with added time information is transmitted to the other clients in real time.

[0199] According to one or more embodiments of the present disclosure, the method further includes:

[0200] If the first chorus role client is the anchor client of the lead role, the multimedia stream data of the lead role client and each second chorus role client in the multimedia stream data of the other clients, as well as the multimedia stream data of the first chorus role client are cached;

[0201] Aligning the multimedia stream data of the lead vocalist client and each second chorus vocalist client, and the multimedia stream data of the first chorus vocalist client according to the time information, to obtain aligned multimedia stream data of each chorus vocalist client;

[0202] The aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role client are merged, and the merged multimedia stream data is transmitted to the audience client.

[0203] According to one or more embodiments of the present disclosure, the method further includes:

[0204] In response to the exit chorus instruction, the role is switched from the chorus role to the non-chorus role;

[0205] caching the multimedia streaming data of the chorus role client among the multimedia streaming data of the other clients;

[0206] According to the time information included in the multimedia streaming data of each chorus role client, the multimedia streaming data of each chorus role client is aligned and then played.

[0207] In a fourth aspect, according to one or more embodiments of the present disclosure, there is provided a chorus device, comprising:

[0208] A receiving unit, configured to receive multimedia stream data of other clients participating in the live broadcast and microphone connection except the first non-chorus role client;

[0209] a processing unit, configured to cache multimedia streaming data of the chorus role client among the multimedia streaming data of the other clients;

[0210] The playing unit is used to align the multimedia streaming data of each chorus role client according to the time information included in the multimedia streaming data of each chorus role client and then play it.

[0211] According to one or more embodiments of the present disclosure, when the playback unit aligns the multimedia streaming data of each chorus role client according to the time information included in the multimedia streaming data of each chorus role client and then plays it, it is configured to:

[0212] According to the time information included in the multimedia streaming data of each chorus role client, multimedia data with the same singing progress is obtained from the multimedia streaming data of each chorus role client for synchronous rendering and / or playback.

[0213] According to one or more embodiments of the present disclosure, the playback unit is further configured to play in real time the multimedia streaming data of other non-chorus role clients in the multimedia streaming data of the other clients; and / or

[0214] The device further includes a collection unit for collecting multimedia stream data of the first non-chorus role client;

[0215] The device further includes a sending unit configured to transmit the multimedia stream data of the first non-chorus role client to the other clients in real time.

[0216] According to one or more embodiments of the present disclosure, if the first non-chorus role client is a non-chorus role anchor client, the sending unit is also used to merge the aligned multimedia stream data of each chorus role client, the multimedia stream data of the second non-chorus role client participating in the live broadcast, and the multimedia stream data of the first non-chorus role client, and transmit the merged multimedia stream data to the audience client.

[0217] According to one or more embodiments of the present disclosure, the processing unit is further configured to, in response to a chorus joining instruction, switch roles from a non-chorus role to a chorus role;

[0218] The playback unit is further configured to play the multimedia streaming data of the lead singing role client among the multimedia streaming data of the other clients, and play the multimedia streaming data of the second non-chorus role client and other chorus role clients in silence;

[0219] The collection unit is further used to collect the local multimedia stream data during the singing process of the lead singer, and add time information to the local multimedia stream data according to the singing progress;

[0220] The sending unit is further configured to transmit the local multimedia stream data with time information added thereto to the other clients in real time.

[0221] In a fifth aspect, according to one or more embodiments of the present disclosure, a chorus device is provided, comprising:

[0222] A receiving unit, configured to receive multimedia stream data of other clients participating in the live broadcast except the first chorus role client;

[0223] A playback unit, configured to play the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and to play the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in a silent state;

[0224] A collection unit, configured to collect multimedia stream data of the lead singer client during a singing process, and add time information to the multimedia stream data of the lead singer client according to a singing progress;

[0225] The sending unit is used to transmit the multimedia stream data of the lead singer role client with added time information to the other clients in real time.

[0226] According to one or more embodiments of the present disclosure, if the lead singing role client is a host singing role client, the device further includes a processing unit configured to cache multimedia streaming data of the secondary singing role client and multimedia streaming data of the lead singing role client among the multimedia streaming data of the other clients;

[0227] The sending unit is further configured to align the multimedia stream data of the chorus role client and the multimedia stream data of the lead role client according to time information to obtain aligned multimedia stream data of each chorus role client; merge the aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the live broadcast, and transmit the merged multimedia stream data to the audience client.

[0228] According to one or more embodiments of the present disclosure, the processing unit is further configured to, in response to a chorus end instruction, resume a regular microphone connection process;

[0229] The sending unit is further configured to send a chorus end instruction to each other client participating in the joint microphone connection, so that each other client can resume a normal joint microphone connection process.

[0230] In a sixth aspect, according to one or more embodiments of the present disclosure, a chorus device is provided, comprising:

[0231] A receiving unit, configured to receive multimedia stream data of other clients participating in the live broadcast except the first chorus role client;

[0232] A playback unit, configured to play the multimedia streaming data of the lead vocalist client among the multimedia streaming data of the other clients, and to play the multimedia streaming data of the non-chorus vocalist clients and the second chorus vocalist clients in a silent state;

[0233] a collection unit, configured to collect multimedia stream data of the first chorus role client during the singing process following the lead singer, and add time information to the multimedia stream data of the first chorus role client according to the singing progress;

[0234] The sending unit is used to transmit the multimedia stream data of the first chorus role client with added time information to the other clients in real time.

[0235] According to one or more embodiments of the present disclosure, if the first secondary vocalist client is a lead vocalist client, the device further includes a processing unit configured to cache multimedia stream data of the lead vocalist client and other second secondary vocalist client clients, as well as multimedia stream data of the first secondary vocalist client, among multimedia stream data of the other clients;

[0236] The sending unit is further used to align the multimedia stream data of the lead role client and each second chorus role client, and the multimedia stream data of the first chorus role client according to time information to obtain aligned multimedia stream data of each chorus role client; merge the aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role clients participating in the joint microphone broadcast, and transmit the merged multimedia stream data to the audience client.

[0237] According to one or more embodiments of the present disclosure, the processing unit is further configured to, in response to an exit chorus instruction, switch roles from a chorus role to a non-chorus role;

[0238] The processing unit is further configured to cache the multimedia streaming data of the chorus role client among the multimedia streaming data of the other clients;

[0239] The playback unit is further configured to align the multimedia stream data of each chorus role client according to time information included in the multimedia stream data of each chorus role client and then play the aligned multimedia stream data.

[0240] In a seventh aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0241] The memory stores computer-executable instructions;

[0242] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the chorus method described in the first aspect and various possible designs of the first aspect, or the chorus method described in the second aspect and various possible designs of the second aspect, or the chorus method described in the third aspect and various possible designs of the third aspect.

[0243] In an eighth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the chorus method described in the first aspect and various possible designs of the first aspect, or the chorus method described in the second aspect and various possible designs of the second aspect, or the chorus method described in the third aspect and various possible designs of the third aspect is implemented.

[0244] In the ninth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising computer-executable instructions. When a processor executes the computer-executable instructions, the chorus method described in the first aspect and various possible designs of the first aspect, or the chorus method described in the second aspect and various possible designs of the second aspect, or the chorus method described in the third aspect and various possible designs of the third aspect is implemented.

[0245] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0246] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0247] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A chorus method, applied to a first non-chorus role client participating in a live connection for a joint live session, the method comprising: Receiving multimedia stream data of other clients except the first non-chorus role client participating in the live connection for the joint live session; Caching the multimedia stream data of the chorus role clients in the multimedia stream data of the other clients; According to the time information included in the multimedia stream data of each chorus role client, aligning the multimedia stream data of each chorus role client and then playing them.

2. The method according to claim 1, wherein, The step of, according to the time information included in the multimedia stream data of each chorus role client, aligning the multimedia stream data of each chorus role client and then playing them, includes: According to the time information included in the multimedia stream data of each chorus role client, obtaining multimedia data at the same singing progress from the multimedia stream data of each chorus role client for synchronous rendering and / or playing.

3. The method according to claim 1 or 2, further comprising: Playing in real time the multimedia stream data of the second non-chorus role client in the multimedia stream data of the other clients; And / or Collecting the multimedia stream data of the first non-chorus role client and transmitting it to the other clients in real time.

4. The method according to claim 1 or 2, further comprising: If the first non-chorus role client is the host client of the non-chorus role, then merging the aligned multimedia stream data of each chorus role client, the multimedia stream data of the second non-chorus role client participating in the live connection for the joint live session, and the multimedia stream data of the first non-chorus role client, and transmitting the merged multimedia stream data to the viewer client.

5. The method according to claim 1, further comprising: In response to a chorus joining instruction, performing a role switch, switching the first non-chorus role client from the non-chorus role to the supporting singer role; Playing the multimedia stream data of the lead singer role client in the multimedia stream data of the other clients, and playing the multimedia stream data of the second non-chorus role client and other supporting singer role clients in mute; Collecting the local multimedia stream data during the lead singer's singing process, and adding time information to the local multimedia stream data according to the singing progress; Transmitting the local multimedia stream data with added time information to the other clients in real time.

6. A chorus method, applied to a lead singer role client participating in a live connection for a joint live session, the method comprising: Receiving multimedia stream data of other clients except the lead singer role client participating in the live connection for the joint live session, and playing the multimedia stream data of the other clients in mute; Collecting the multimedia stream data of the lead singer role client during the singing process, and adding time information to the multimedia stream data of the lead singer role client according to the singing progress; Transmitting the multimedia stream data of the lead singer role client with added time information to the other clients in real time.

7. The method according to claim 6, further comprising: If the lead singer role client is the host client of the lead singer role, cache the multimedia stream data of the backup singer role client and the multimedia stream data of the lead singer role client in the multimedia stream data of the other clients; Align the multimedia stream data of the backup singer role client and the multimedia stream data of the lead singer role client according to the time information to obtain the aligned multimedia stream data of each chorus role client; Merge the aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role client participating in the live connection, and transmit the merged multimedia stream data to the viewer client.

8. The method according to claim 6 or 7, further comprising: In response to the chorus end instruction, resume the normal connection process; Send the chorus end instruction to each of the other clients participating in the connection so that each of the other clients resumes the normal connection process.

9. A chorus method applied to the first backup singer role client participating in the live connection, the method comprising: Receive the multimedia stream data of the other clients participating in the live connection except the first backup singer role client; Play the multimedia stream data of the lead singer role client in the multimedia stream data of the other clients, and play the multimedia stream data of the non-chorus role client and each second backup singer role client in mute; Collect the multimedia stream data of the first backup singer role client during the lead singer's singing process, and add time information to the multimedia stream data of the first backup singer role client according to the singing progress; Real-time transmit the multimedia stream data of the first backup singer role client with added time information to the other clients.

10. The method according to claim 9, further comprising: If the first backup singer role client is the host client of the lead singer role, cache the multimedia stream data of the lead singer role client, each second backup singer role client, and the multimedia stream data of the first backup singer role client in the multimedia stream data of the other clients; Align the multimedia stream data of the lead singer role client, each second backup singer role client, and the multimedia stream data of the first backup singer role client according to the time information to obtain the aligned multimedia stream data of each chorus role client; Merge the aligned multimedia stream data of each chorus role client and the multimedia stream data of the non-chorus role client, and transmit the merged multimedia stream data to the viewer client.

11. The method according to claim 9 or 10, further comprising: In response to the exit chorus instruction, perform role switching, switching from the backup singer role to the non-chorus role; Cache the multimedia stream data of the chorus role client in the multimedia stream data of the other clients; According to the time information included in the multimedia stream data of each chorus role client, align and play the multimedia stream data of each chorus role client.

12. A chorus device, comprising: A receiving unit, configured to receive multimedia stream data of other clients participating in the live connection for duet singing except for the first non-chorus role client; A processing unit, configured to cache the multimedia stream data of the chorus role clients in the multimedia stream data of the other clients; A playing unit, configured to play the multimedia stream data of each chorus role client after alignment according to the time information included in the multimedia stream data of each chorus role client.

13. A chorus device, comprising: A receiving unit, configured to receive multimedia stream data of clients participating in the live connection for duet singing except for the lead singer role client; A playing unit, configured to play the multimedia stream data of the other clients in mute; An acquisition unit, configured to acquire the multimedia stream data of the lead singer role client during the singing process, and add time information to the multimedia stream data of the lead singer role client according to the singing progress; A sending unit, configured to transmit the multimedia stream data of the lead singer role client with added time information to each of the other clients participating in the connection for duet singing in real time.

14. A chorus device, comprising: A receiving unit, configured to receive multimedia stream data of other clients participating in the live connection for duet singing except for the first backup singer role client; A playing unit, configured to play the multimedia stream data of the lead singer role client in the multimedia stream data of the other clients, and play the multimedia stream data of the non-chorus role clients and each second backup singer role client in mute; An acquisition unit, configured to acquire the multimedia stream data of the first backup singer role client during the singing process following the lead singer, and add time information to the multimedia stream data of the first backup singer role client according to the singing progress; A sending unit, configured to transmit the multimedia stream data of the first backup singer role client with added time information to the other clients in real time.

15. An electronic device, comprising: At least one processor and a memory, wherein, the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the method according to any one of claims 1-11.

16. A computer-readable storage medium stores computer-executable instructions, wherein, When the processor executes the computer execution instructions, the method according to any one of claims 1-11 is implemented.

17. A computer program product, comprising computer-executable instructions, wherein, When the processor executes the computer execution instructions, the method according to any one of claims 1-11 is implemented.

Images