Live broadcast stream synthesis method and device, equipment and storage medium

Abstract

The invention discloses a live broadcast stream synthesis method and device, equipment and a storage medium, and belongs to the field of live broadcast. The method comprises: in response to an audio acquisition instruction, acquiring audio data of a target audio through an integrated player, the target audio being an audio indicated by the audio acquisition instruction, the integrated player being integrated in an anchor client, and the integrated player being provided by an audio service party other than an operator of the anchor client; voice data are obtained, the voice data comprise data collected by a microphone connected with an anchor terminal, and the anchor terminal is a terminal where an anchor client is located; and sending the audio data and the voice data to a live broadcast server, wherein the audio data and the voice data are used by the live broadcast server to synthesize a live broadcast stream. The process of acquiring the audio data of the target audio does not depend on recording of a microphone, so that the audience client can always play background music.

Description

Technical Field

[0001] The present application relates to the field of live broadcasting, and in particular to a method, device, equipment and storage medium for synthesizing a live broadcast stream. Background Art

[0002] When broadcasting live through the broadcast client, the anchor usually provides background music in the live broadcast room to enhance the interest of the audience when watching the live broadcast.

[0003] In the related art, the anchor plays background music through a music client other than the anchor client during live broadcast. The background music played through the speaker can be recorded by a microphone connected to the anchor terminal where the anchor client is located, thereby obtaining voice data containing the audio data of the background music. The anchor client then sends the recorded voice data to the live broadcast server to generate a live broadcast stream, and the live broadcast server provides the live broadcast stream to the viewer client, so that the viewer client can play the background music according to the audio data in the live broadcast stream.

[0004] When the above method is used to obtain audio data, if the host uses headphones to play background music, the microphone will not be able to collect the audio data of the background music, resulting in the viewer client being unable to play the background music. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for synthesizing a live stream, which can enable the viewer client to always play background music. The technical solution is as follows:

[0006] According to one aspect of the present application, a method for synthesizing a live stream is provided, the method being applied to a host client, the method comprising:

[0007] In response to the audio acquisition instruction, audio data of a target audio is acquired through an integrated player, wherein the target audio is the audio indicated by the audio acquisition instruction, the integrated player is integrated in the host client, and the integrated player is provided by an audio service provider other than the operator of the host client;

[0008] Acquire voice data, where the voice data includes data collected by a microphone connected to an anchor terminal, where the anchor terminal is a terminal where the anchor client is located;

[0009] The audio data and the voice data are sent to a live broadcast server, and the audio data and the voice data are used by the live broadcast server to synthesize a live broadcast stream.

[0010] According to another aspect of the present application, a live stream synthesis device is provided, the device comprising:

[0011] An acquisition module, configured to acquire audio data of a target audio through an integrated player in response to an audio acquisition instruction, wherein the target audio is the audio indicated by the audio acquisition instruction, and the integrated player is integrated in the anchor client, and the integrated player is provided by an audio service provider other than the operator of the anchor client;

[0012] The acquisition module is further used to acquire voice data, where the voice data includes data collected by a microphone connected to an anchor terminal, where the anchor terminal is a terminal where the anchor client is located;

[0013] The sending module is used to send the audio data and the voice data to the live broadcast server, and the audio data and the voice data are used by the live broadcast server to synthesize the live broadcast stream.

[0014] In an optional design, the device further includes:

[0015] The acquisition module is used to acquire the audio file of the target audio in the audio server through the integrated player;

[0016] A decoding module, used for decoding the audio file of the target audio through the integrated player to obtain the audio data;

[0017] The acquisition module is further used to acquire the audio data transmitted by the integrated player to the anchor client;

[0018] Wherein, a communication connection is established between the audio server and the integrated player, and the audio server is maintained by the audio service provider.

[0019] In an optional design, the decoding module is used to:

[0020] The integrated player decodes the audio file of the target audio to obtain pulse code modulation (PCM) data corresponding to the audio file.

[0021] In an optional design, the integrated player is developed based on browser technology so as to be integrated into the anchor client.

[0022] In an optional design, the device further includes:

[0023] The transmission module is used to transmit the audio data to the native player in response to the audio playing instruction when the audio data is not played, so that the native player calls the playing device to play the audio data.

[0024] In an optional design, the device further includes:

[0025] an identification module, configured to obtain identification data by setting a filtering identification on the audio data when the audio data is played through a speaker;

[0026] A filtering module, used for filtering out the identification data from the acquired voice data in the process of acquiring the voice data, so as to obtain filtered voice data;

[0027] The sending module is used to send the audio data and the filtered voice data to the live broadcast server.

[0028] In an optional design, the audio data is multi-track audio data; the acquisition module is used to:

[0029] In response to the track switching instruction, obtaining audio data under a target track from the multi-track audio data through the integrated player, wherein the target track is the track indicated by the track switching instruction;

[0030] The sending module is used for:

[0031] The audio data and the voice data under the target audio track are sent to the live broadcast server.

[0032] In an optional design, the device further includes:

[0033] A processing module, configured to respond to an audio processing instruction and process the audio data of the target audio according to the audio processing instruction through the integrated player to obtain processed audio data;

[0034] The audio processing instruction includes at least one of an instruction for changing the pitch of the target audio, an instruction for adjusting the volume of the target audio, and an instruction for adjusting the sound effect of the target audio.

[0035] According to another aspect of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the live stream synthesis method as described above.

[0036] According to another aspect of the present application, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the live stream synthesis method as described above.

[0037] According to another aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the live stream synthesis method provided in various optional implementations of the above aspects.

[0038] The beneficial effects of the technical solution provided by this application include at least:

[0039] By integrating the integrated player provided by the audio service provider into the anchor client, it is possible to obtain the audio data of the target audio through the integrated player. The anchor client provides the audio data of the target audio to the live broadcast server to synthesize the live broadcast stream and provide it to the audience client, so that the audience client can play the target audio according to the synthesized live broadcast stream. Since the process of obtaining the audio data of the target audio does not need to rely on the recording of the microphone, the anchor client can still obtain the audio data of the target audio even when the target audio is played through headphones. In the case where the target audio is background music, it is possible to enable the audience client to always play background music. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 is a structural block diagram of a computer system provided by an exemplary embodiment of the present application;

[0042] Figure 2 is a schematic diagram of a process of synthesizing a live stream provided by an exemplary embodiment of the present application;

[0043] Figure 3 It is a flowchart of a method for synthesizing a live stream provided by an exemplary embodiment of the present application;

[0044] Figure 4 It is a flowchart of a method for synthesizing a live stream provided by an exemplary embodiment of the present application;

[0045] Figure 5 is a schematic diagram of a process of filtering voice data provided by an exemplary embodiment of the present application;

[0046] Figure 6is a schematic diagram of a user interface of an integrated player provided by an exemplary embodiment of the present application;

[0047] Figure 7 It is a structural schematic diagram of a live stream synthesis device provided by an exemplary embodiment of the present application;

[0048] Figure 8 It is a structural schematic diagram of a live stream synthesis device provided by an exemplary embodiment of the present application;

[0049] Fig. 9 It is a structural schematic diagram of a live stream synthesis device provided by an exemplary embodiment of the present application;

[0050] Fig.10 It is a structural schematic diagram of a live stream synthesis device provided by an exemplary embodiment of the present application;

[0051] Fig.11 It is a structural schematic diagram of a live stream synthesis device provided by an exemplary embodiment of the present application;

[0052] Fig.12 It is a schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application.

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0055] Figure 1 It is a structural block diagram of a computer system provided by an exemplary embodiment of the present application. The computer system 100 includes: a first terminal 110, a second terminal 120 and a server cluster 130.

[0056] The first terminal 110 is installed and runs an anchor client 111, which is a client that supports providing live content. In some embodiments, the types of the anchor client 111 include at least one of the following: live broadcast type, short video type, video on demand type, listening to music type, singing type, instant messaging type, food type, takeaway type, local service type, game type, and financial type. When the first terminal 110 runs the anchor client 111, the user interface of the anchor client 111 is displayed on the screen of the first terminal 110. The first terminal 110 is a terminal used by the first user 112, the first user 112 can be an anchor user, and the first terminal 110 can be called an anchor terminal. The first user 112 uses the anchor client 111 by logging in.

[0057] The second terminal 120 is installed and runs a viewer client 121, which is a client that supports watching live content. In some embodiments, the type of the viewer client 121 includes at least one of the following: live type, short video type, video on demand type, listening to music type, singing type, instant messaging type, food type, takeaway type, local service type, game type, and financial type. When the second terminal 120 runs the viewer client 121, the user interface of the viewer client 121 is displayed on the screen of the second terminal 120. Optionally, the viewer client 121 is exactly the same as the anchor client 111, or is the same client under different operating systems, or is a different version of the same client, or is a different client. The viewer client 121 and the anchor client 111 can be collectively referred to as a live client. The second terminal 120 is a terminal used by the second user 122, the second user 122 can be a viewer user, and the second terminal 120 can be referred to as a viewer terminal. The second user 122 uses the viewer client 121 by logging in.

[0058] The device types of the first terminal 110 and the second terminal 120 include at least one of a smart phone, a tablet computer, an e-book reader, an MP3 player, an MP4 player, a laptop computer, and a desktop computer.

[0059] Figure 1 Only two terminals are shown, but in different embodiments, there are multiple other terminals 140. In some embodiments, there is at least one other terminal 140 corresponding to the developer, and a development and editing platform of the live broadcast client is installed on the other terminal 140. The developer can edit and update the live broadcast client on the other terminal 140, and transmit the updated client installation package to the server cluster 130 through a wired or wireless network. The first terminal 110 and the second terminal 120 can download the client installation package from the server cluster 130 to update the client.

[0060] The first terminal 110 , the second terminal 120 , and other terminals 140 are connected to the server cluster 130 via a wireless network or a wired network.

[0061] The server cluster 130 includes at least one of a server, multiple servers, a cloud computing platform, and a virtualization center. The server cluster 130 is used to provide background services for the live broadcast client. Optionally, the server cluster 130 undertakes the main computing work, and the terminal undertakes the secondary computing work; or, the server cluster 130 undertakes the secondary computing work, and the terminal undertakes the main computing work; or, the server cluster 130 and the terminal adopt a distributed computing architecture for collaborative computing. Optionally, the above-mentioned terminal and server are both computer devices.

[0062] In an illustrative example, the server cluster 130 includes a server 131 and a server 136, and the server 131 includes a processor 132, a database 133, a live broadcast service module 134, and an input / output interface (I / O interface) 135. The processor 132 is used to load instructions stored in the server 131 and process data in the database 133 and the live broadcast service module 134; the database 133 is used to store user-related data of the first terminal 110, the second terminal 120, and other terminals 140, such as the user's avatar, nickname, virtual currency, etc.; the live broadcast service module 134 is used to provide services for live broadcast related functions in the live broadcast client; and the I / O interface 135 is used to establish communication and exchange data with the terminal through a wireless network or a wired network.

[0063] In combination with the above introduction to the implementation environment involved in the present application, the method provided in the embodiment of the present application will be described below.

[0064] Figure 2 FIG. 1 is a schematic diagram of a process of synthesizing a live stream provided by an exemplary embodiment of the present application. Figure 2 As shown, the live broadcast platform 201 includes an anchor client and a live broadcast server. The anchor client is integrated with an integrated player 202, and the integrated player 202 is provided by an audio service provider other than the operator of the live broadcast platform 201. In response to the audio acquisition instruction, the anchor client will obtain the audio file of the target audio in the audio server 203 through the integrated player 202. Among them, the target audio is the audio indicated by the audio acquisition instruction, and the audio server 203 has established a communication connection with the integrated player 202, and the audio server 203 is maintained by the audio service provider. After that, the anchor client decodes the audio file of the target audio through the decoder in the integrated player 202, and processes it through the processing device, so as to obtain the audio data of the target audio, such as the pulse code modulation (Pulse Code Modulation, PCM) data of the target audio. The anchor client can obtain the audio data of the target audio provided by the integrated player 202 through the PCM receiving device, and thus send the audio data to the live broadcast server. The anchor client will also obtain voice data and send the obtained voice data to the live broadcast server. The voice data includes data collected by a microphone 204 connected to the anchor terminal, where the anchor terminal is the terminal where the anchor client is located. In some embodiments, when the anchor client does not play the audio data, in response to the audio play instruction, the anchor client transmits the audio data obtained through the PCM receiving device to the native player of the anchor terminal, so as to call the playback device (e.g., speaker 205) of the anchor terminal to play the audio data of the target audio through the native player.

[0065] After acquiring the audio data and voice data of the target audio, the live broadcast server will synthesize the live broadcast stream according to the acquired audio data and voice data through the mixing device, and send the synthesized live broadcast stream to the viewer client 206. The viewer client 206 can play the target audio according to the audio data in the live broadcast stream by playing the live broadcast stream.

[0066] By integrating the integrated player provided by the audio service provider into the anchor client, it is possible to obtain the audio data of the target audio through the integrated player. The anchor client provides the audio data of the target audio to the live broadcast server to synthesize the live broadcast stream and provide it to the audience client, so that the audience client can play the target audio according to the synthesized live broadcast stream. Since the process of obtaining the audio data of the target audio does not need to rely on the recording of the microphone, the anchor client can still obtain the audio data of the target audio even when the target audio is played through headphones. In the case where the target audio is background music, it is possible to enable the audience client to always play background music.

[0067] Figure 3 The figure is a flow chart of a method for synthesizing a live stream provided by an exemplary embodiment of the present application. The method can be used in a computer device or a client on a computer device, for example, in a host client.

[0068] like Figure 3 As shown, the method includes:

[0069] Step 302: In response to the audio acquisition instruction, audio data of the target audio is acquired through the integrated player.

[0070] The audio acquisition instruction is an instruction for acquiring the target audio, and the target audio is the audio indicated by the audio acquisition instruction. The target audio is the audio provided / played to the audience client corresponding to the anchor client during the live broadcast through the anchor client. Optionally, the type of the target audio includes at least one of music, songs, radio stations, and audiobooks, which is not limited in the embodiments of the present application. In some embodiments, the audio acquisition instruction is triggered by the anchor client, for example, by the anchor account during the live broadcast process. The audio acquisition instruction is triggered by the live broadcast server, and the live broadcast server can trigger the audio acquisition instruction and send it to the anchor client during the live broadcast of the anchor client. In the case where the live broadcast server triggers the audio acquisition instruction, the target audio is determined by the live broadcast server, and the live broadcast server and the anchor client establish a communication connection by wireless or wired means, and the live broadcast server is used to provide live broadcast related services to the anchor client.

[0071] In some embodiments, the anchor client broadcasts live through the live broadcast room. The anchor client is used to provide live broadcast content for the live broadcast room. The live broadcast room corresponding to the anchor client is created by the anchor account. The audience client corresponding to the anchor client is a client logged in / used by the audience account that joins the live broadcast room of the anchor client. Optionally, the live broadcast content provided by the live broadcast room includes at least one of game live broadcast, e-commerce live broadcast, music live broadcast, education live broadcast, and entertainment live broadcast. By providing functions such as video, audio and text chat in the live broadcast room, real-time interaction between the anchor user and the audience user can be achieved. For example, the audience user can send barrages, send comments, and give gifts in the live broadcast room, so as to achieve real-time communication with the anchor user.

[0072] The integrated player is integrated into the anchor client, and the integrated player is provided by an audio service provider other than the operator of the anchor client. The audio service provider provides the integrated player to the operator of the anchor client, so that the operator of the anchor client can integrate the integrated player into the anchor client to use the services provided by the audio service provider. In some embodiments, the integrated player supports integration in different anchor clients, for example, supports integration in anchor clients corresponding to different operating systems under different live broadcast platforms. In some embodiments, the integrated player is developed based on browser technology to be integrated in the anchor client. For example, the integrated player is implemented as a web application (web application) to be integrated in the anchor client.

[0073] The audio data of the target audio includes data for playing the target audio. In some embodiments, the audio data of the target audio is obtained by decoding the audio file corresponding to the target audio through the integrated player. Optionally, the integrated player is pre-set with audio files corresponding to multiple audios, and the audio file of the target audio is obtained from the pre-set audio file according to the audio acquisition instruction by the integrated player, and the audio file of the target audio is decoded by the integrated player, so that the host client can obtain the audio data of the target audio. Alternatively, the audio server stores audio files corresponding to multiple audios, and the audio file of the target audio is obtained from the audio server according to the audio acquisition instruction by the integrated player, and the audio file of the target audio is decoded by the integrated player, so that the host client can obtain the audio data of the target audio. The audio server and the integrated player establish a communication connection by wireless or wired means, and the audio server is maintained by the audio service party, that is, the audio file stored in the audio server is provided by the audio service party. In some embodiments, the audio service party has the copyright of the audio file stored in the audio server. In some embodiments, the audio data of the target audio is the PCM data corresponding to the audio file of the target audio.

[0074] Step 304: Acquire voice data.

[0075] The voice data includes data collected by a microphone connected to the anchor terminal, for example, data obtained by recording sound with the microphone connected to the anchor terminal. The anchor terminal is the terminal where the anchor client is located.

[0076] In some embodiments, when the live broadcast content provided by the anchor client includes data collected by the microphone, for example, when the anchor client turns on the microphone during the live broadcast, the anchor client will obtain voice data through the microphone. When the live broadcast content provided by the anchor client does not include data collected by the microphone, for example, when the anchor client turns off the microphone during the live broadcast, the anchor client will not obtain voice data through the microphone, nor will it send voice data to the live broadcast server.

[0077] Step 306: Send audio data and voice data to the live broadcast server.

[0078] The audio data and the voice data are used by the live broadcast server to synthesize the live broadcast stream. In some embodiments, after synthesizing the audio data and the voice data into the live broadcast stream, the live broadcast server sends the synthesized live broadcast stream to the audience client corresponding to the anchor client. By playing the live broadcast stream, the audience client can play the target audio according to the audio data in the live broadcast stream.

[0079] In some embodiments, the host client sends audio data and voice data to the live broadcast server at the same time, or the host client sends audio data and voice data to the live broadcast server at different times, which is not limited in this embodiment of the present application.

[0080] In some embodiments, when the live broadcast content provided by the anchor client includes a live broadcast screen, the anchor client will also send display data for displaying the live broadcast screen to the live broadcast server. When the display data is obtained, the live broadcast server will use the display data to synthesize the live broadcast stream, so that the audience client can display the corresponding live broadcast screen by playing the live broadcast stream. In some embodiments, the display data provided by the anchor client is collected by a camera connected to the anchor terminal, the display data can also be set by the anchor account, and the display data can also be generated based on the data collected by the above camera combined with the data set by the anchor account, and the embodiments of the present application do not limit this.

[0081] In summary, the method provided in this embodiment integrates an integrated player provided by an audio service provider into the anchor client, thereby enabling the audio data of the target audio to be acquired through the integrated player. The anchor client provides the audio data of the target audio to the live broadcast server to synthesize the live broadcast stream and provide it to the audience client, so that the audience client can play the target audio according to the synthesized live broadcast stream. Since the process of acquiring the audio data of the target audio does not need to rely on the recording of the microphone, the anchor client can still acquire the audio data of the target audio even when the target audio is played through headphones. In the case where the target audio is background music, it can be achieved that the audience client can always play the background music.

[0082] Figure 4 The figure is a flow chart of a method for synthesizing a live stream provided by an exemplary embodiment of the present application. The method can be used in a computer device or a client on a computer device, for example, in a host client.

[0083] like Figure 4 As shown, the method includes:

[0084] Step 402: In response to the audio acquisition instruction, audio data of the target audio is acquired through the integrated player.

[0085] The audio acquisition instruction is an instruction for acquiring target audio, and the target audio is the audio indicated by the audio acquisition instruction. The target audio is the audio provided / played to the viewer client corresponding to the anchor client during the live broadcast through the anchor client. Optionally, the type of the target audio includes at least one of music, songs, radio stations, and audiobooks, which is not limited in the embodiment of the present application.

[0086] In some embodiments, the audio acquisition instruction is triggered by the anchor client, for example, by the anchor account selecting the target audio during the live broadcast. In some embodiments, the audio acquisition instruction is triggered by the live broadcast server, which can trigger the audio acquisition instruction and send it to the anchor client during the live broadcast of the anchor client. In the case where the live broadcast server triggers the audio acquisition instruction, the target audio is determined by the live broadcast server, and the live broadcast server and the anchor client establish a communication connection by wireless or wired means, and the live broadcast server is used to provide live broadcast related services to the anchor client.

[0087] In some embodiments, the anchor client broadcasts live through the live broadcast room. The anchor client is used to provide live broadcast content for the live broadcast room. The live broadcast room corresponding to the anchor client is created by the anchor account. The audience client corresponding to the anchor client is a client logged in / used by the audience account that joins the live broadcast room of the anchor client. Optionally, the live broadcast content provided by the live broadcast room includes at least one of game live broadcast, e-commerce live broadcast, music live broadcast, education live broadcast, and entertainment live broadcast.

[0088] Optionally, at any time during the live broadcast of the anchor client, or at the time selected by the anchor account corresponding to the anchor client during the live broadcast, the live broadcast server will obtain the candidate audio corresponding to the anchor account, and determine the target audio from the candidate audio, for example, randomly determine the target audio from the candidate audio, thereby triggering an audio acquisition instruction to the anchor client. The candidate audio is determined by the live broadcast server based on the historical audio played in the live broadcast room by the anchor account during the historical live broadcast. In the process of determining the candidate audio based on the historical audio, the live broadcast server will determine the interactive information corresponding to the live broadcast room of the anchor account during the playback of each historical audio, thereby determining the score of the historical audio based on the interactive information, and sorting the historical audio according to the score of the historical audio, thereby determining the first n historical audios after sorting as candidate audio, where n is a positive integer. Optionally, the interactive information includes at least one of the number of people in the live broadcast room of the anchor account, the number of people entering the live broadcast room, the number of people exiting the live broadcast room, the number of bullet screens in the live broadcast room, and the number of likes in the live broadcast room during the playback of the historical audio. The number of people in the live broadcast room is the maximum number of people in the live broadcast room (the number of audience accounts) during the playback of historical audio, the number of people entering the live broadcast room is the cumulative number of people entering the live broadcast room during the playback of historical audio, the number of people exiting the live broadcast room is the cumulative number of people exiting the live broadcast room during the playback of historical audio, the number of barrages in the live broadcast room is the total number of barrages sent in the live broadcast room during the playback of historical audio, and the number of likes in the live broadcast room is the total number of likes for the live broadcast room or the anchor account during the playback of historical audio. Optionally, the live broadcast server determines the score of the historical audio by the following formula: (number of people in the live broadcast room + (number of people entering the live broadcast room - number of people exiting the live broadcast room))*(number of barrages in the live broadcast room / number of people in the live broadcast room)*(number of likes in the live broadcast room / number of people in the live broadcast room). Since the score of the historical audio determined by the above formula can accurately reflect the popularity of the historical audio in the live broadcast room, the historical audio is sorted according to the score of the historical audio, so as to determine that the target audio can achieve the audio with higher popularity in the live broadcast room, which helps to improve the live broadcast effect.

[0089] The integrated player is integrated into the anchor client, and the integrated player is provided by an audio service provider other than the operator of the anchor client. The audio service provider provides the integrated player to the operator of the anchor client, so that the operator of the anchor client can integrate the integrated player into the anchor client to use the services provided by the audio service provider. In some embodiments, the integrated player supports integration in different anchor clients, for example, supports integration in anchor clients corresponding to different operating systems under different live broadcast platforms. In some embodiments, the integrated player is developed based on browser technology to be integrated in the anchor client. For example, the integrated player is implemented as a web application (web application) to be integrated in the anchor client. In some embodiments, when the integrated player is implemented as a web application, the integrated player is integrated into the anchor client through an iframe (an embedded framework).

[0090] The audio data of the target audio includes data for playing the target audio. In some embodiments, the audio data of the target audio is obtained by decoding the audio file corresponding to the target audio through the integrated player. Optionally, the integrated player is pre-set with audio files corresponding to each of the multiple audios, and the integrated player obtains the audio file of the target audio from the pre-set audio file according to the audio acquisition instruction, and the audio file of the target audio is decoded by the integrated player, so that the host client can obtain the audio data of the target audio. Alternatively, the audio server stores audio files corresponding to each of the multiple audios, and the integrated player obtains the audio file of the target audio from the audio server according to the audio acquisition instruction, and the audio file of the target audio is decoded by the integrated player, so that the host client can obtain the audio data of the target audio. The audio server and the integrated player establish a communication connection by wireless or wired means, and the audio server is maintained by the audio service party, that is, the audio file stored in the audio server is provided by the audio service party. In some embodiments, the audio service party has the copyright of the audio file stored in the audio server.

[0091] In some embodiments, in the process of obtaining the audio data of the target audio, the anchor client obtains the audio file of the target audio in the audio server through the integrated player, and decodes the audio file of the target audio through the integrated player to obtain the audio data, and then the anchor client obtains the audio data of the target audio transmitted by the integrated player to the anchor client, thereby obtaining the audio data. Optionally, a decoder is pre-installed in the integrated player, and the integrated player decodes the audio file of the target audio through the decoder. In some embodiments, the decoder is implemented based on FFmpeg (an open source computer program for recording and converting digital audio or video and converting it into a stream), and the decoder is pre-installed with a decoding module implemented based on wasm (webassembly, a portable format that allows code to run in various browsers and operating systems). In the case where the audio file formats obtained by the integrated player are all mkv, by only turning on the decoding function for mkv files in the decoding module, a minimized decoding module can be compiled. In some embodiments, the audio data of the target audio is the PCM data corresponding to the audio file of the target audio. The audio file of the target audio is decoded by the integrated player, and the decoded data is processed, so that the PCM data corresponding to the audio file can be obtained. In some embodiments, during the decoding of the audio file of the target audio, a cache is set for the decoded data to improve the efficiency of decoding. When the integrated player transmits the audio data of the target audio to the host client, the integrated player can pass the binary data pointer corresponding to the audio data to the host client, thereby improving the efficiency of transmitting the audio data to the host client.

[0092] Step 404: When the audio data is not played, in response to the audio play instruction, the audio data is transmitted to the native player so that the native player calls a play device to play the audio data.

[0093] When the host client obtains the audio data of the target audio, the audio data of the target audio can be played, thereby playing the sound of the target audio. The host client may not play the audio data of the target audio, in which case the host client only obtains the audio data of the target audio and does not play the target audio.

[0094] The audio playback instruction is an instruction for triggering the playback of the target audio. The audio playback instruction is triggered by the anchor client, for example, by the anchor account's playback operation on the target audio. In some embodiments, the audio acquisition instruction and the audio playback instruction are implemented as the same instruction and are triggered by the same operation. In some embodiments, the audio acquisition instruction and the audio playback instruction are implemented as different instructions and are triggered by different operations.

[0095] The native player is a player pre-installed in the operating system of the anchor terminal where the anchor client is located, which can play the audio in the anchor terminal. The native player is different from the integrated player. In the process of calling the native player to play, the anchor client will transmit the audio data of the target audio obtained through the integrated player to the native player, so as to call the playback device connected to the anchor terminal through the native player to play the audio data, thereby playing the target audio through the playback device. Optionally, the playback device includes at least one of a speaker integrated in the anchor terminal, an external speaker connected to the anchor terminal wirelessly or wired, and a headset connected to the anchor terminal wirelessly or wired.

[0096] Step 406: Acquire voice data.

[0097] The voice data includes data collected by a microphone connected to the anchor terminal, for example, data obtained by recording sound with the microphone connected to the anchor terminal. The anchor terminal is the terminal where the anchor client is located.

[0098] In some embodiments, when the live broadcast content provided by the anchor client includes data collected by the microphone, for example, when the anchor client turns on the microphone during the live broadcast, the anchor client will obtain voice data through the microphone. When the live broadcast content provided by the anchor client does not include data collected by the microphone, for example, when the anchor client turns off the microphone during the live broadcast, the anchor client will not obtain voice data through the microphone, nor will it send voice data to the live broadcast server.

[0099] When the audio data of the target audio is played through headphones, the microphone connected to the host terminal is usually unable to collect the sound produced by the headphones when the target audio is played due to the tightness of the headphones when the sound is produced. When the audio data of the target audio is played through a speaker, the microphone connected to the host terminal can usually collect the sound produced by the speaker when the target audio is played due to the lack of tightness when the speaker is produced. At this time, the voice data collected by the microphone contains the data corresponding to the target audio, and the host client has obtained the audio data of the target audio. When the data collected by the microphone and the audio data obtained by the host client are played at the same time, audio overlap (echo) may occur.

[0100] In some embodiments, when playing audio data through a speaker, the anchor client can obtain identification data by setting a filter identification on the audio data of the acquired target audio. Later, in the process of acquiring voice data, the anchor client will filter out the above identification data in the acquired voice data, for example, filtering out data whose similarity with the above identification data is greater than a similarity threshold in the acquired voice data, thereby obtaining filtered voice data. In the case of playing the filtered voice data and the audio data acquired by the anchor client at the same time, audio overlap can be eliminated. In this case, the anchor client will send audio data and filtered voice data to the live broadcast server. The sending process can refer to the relevant content in the embodiments of the present application, which will not be described in detail here. It should be noted that the above process of filtering voice data can also be performed by the live broadcast server.

[0101] In some embodiments, when the identification data is obtained, the anchor client inputs the identification data and the voice data into the echo prediction model, thereby obtaining the echo data output by the echo prediction model, and filtering the echo data in the voice data to obtain the filtered voice data. Among them, the echo data is the data corresponding to the sound obtained after the sound corresponding to the identification data predicted by the echo prediction model propagates in the physical space in the voice data. Optionally, the echo prediction model is an artificial intelligence (AI) model, such as a machine learning model built based on a neural network (NN). The echo prediction model is obtained by training different sample audio data and the sound collection audio data corresponding to each of the different sample audio data. The sample audio data is the audio data corresponding to the first sound, and the sound collection audio data is the audio data corresponding to the second sound. The second sound is obtained by simulating the propagation of the first sound in the sample space, and the sample space is obtained by simulating the live broadcast scene corresponding to the sample anchor account, and the live broadcast scene is the real scene in the live broadcast screen corresponding to the sample anchor account.

[0102] In some embodiments, when the identification data is obtained, the anchor client will analyze the obtained voice data according to the voice data of the anchor voice corresponding to the anchor account, so as to obtain the reference audio data corresponding to the time period in which the anchor voice is not included in the voice data. Among them, the voice data of the anchor voice is recorded by the anchor client through a microphone. After that, by comparing the reference audio data with the identification data, the original audio data in the identification data can be obtained, and the original audio data is the audio data with the highest similarity to the reference audio data in the identification data. By analyzing the difference between the original audio data and the reference audio data, the anchor client can determine the propagation parameters corresponding to the original audio data and the reference audio data. The propagation parameters are parameters used to characterize the propagation path. The propagation path is the path of the sound from the speaker playing the target audio to the microphone. According to the propagation path, the change relationship between the sound signal played by the speaker and the sound signal collected by the microphone when the sound signal is propagated to the microphone can be determined. The anchor client processes the identification data according to the propagation parameters to obtain the echo data corresponding to the identification data. The echo data is the audio data corresponding to the sound signal collected by the microphone after the sound signal generated by the speaker playing the audio data is propagated through the above-mentioned propagation path. By filtering out the echo data from the voice data, filtered voice data can be obtained. By analyzing the reference audio data corresponding to the time period in the voice data that does not contain the host's voice, and determining the corresponding original audio data based on this, since in the process of determining the original audio data corresponding to the reference audio data, the interference of the host's voice can be avoided, which helps to improve the accuracy of determining the original audio data, thereby improving the filtering effect of the voice data.

[0103] For example, Figure 5 FIG. 1 is a schematic diagram of a process of filtering voice data provided by an exemplary embodiment of the present application. Figure 5As shown, the live broadcast platform 501 includes an anchor client and a live broadcast server, and an integrated player 502 is integrated in the anchor client. In response to the audio acquisition instruction, the anchor client will obtain the audio file of the target audio in the audio server through the integrated player 502. After that, the anchor client decodes the audio file of the target audio through the decoder in the integrated player 502, and processes it through the processing device, so as to obtain the audio data of the target audio, such as the PCM data of the target audio. The anchor client can obtain the audio data of the target audio provided by the integrated player 502 through the PCM receiving device. In the case of obtaining the audio data of the target audio, the anchor client will play the audio data of the target audio through the speaker. In the process of playing the audio data of the target audio through the speaker, the anchor client will also obtain the voice data collected by the microphone 503. In the process of obtaining the voice data, the anchor client will set a filtering mark on the obtained audio data, so as to obtain the mark data. And the above mark data is filtered out from the obtained voice data through the filtering device, so as to obtain the filtered voice data.

[0104] Step 408: Send audio data and voice data to the live broadcast server.

[0105] The audio data and the voice data are used by the live broadcast server to synthesize the live broadcast stream. In some embodiments, after synthesizing the audio data and the voice data into the live broadcast stream, the live broadcast server sends the synthesized live broadcast stream to the audience client corresponding to the anchor client. By playing the live broadcast stream, the audience client can play the target audio according to the audio data in the live broadcast stream.

[0106] In some embodiments, the host client sends audio data and voice data to the live broadcast server at the same time, or the host client sends audio data and voice data to the live broadcast server at different times, which is not limited in this embodiment of the present application.

[0107] In some embodiments, when the live broadcast content provided by the anchor client includes a live broadcast screen, the anchor client will also send display data for displaying the live broadcast screen to the live broadcast server. When the display data is obtained, the live broadcast server will use the display data to synthesize the live broadcast stream, so that the audience client can display the corresponding live broadcast screen by playing the live broadcast stream. In some embodiments, the display data provided by the anchor client is collected by a camera connected to the anchor terminal, the display data can also be set by the anchor account, and the display data can also be generated based on the data collected by the above camera combined with the data set by the anchor account, and the embodiments of the present application do not limit this.

[0108] In some embodiments, the integrated player also has the function of switching audio tracks, and the audio data is multi-track audio data. Optionally, the multi-track audio data at least includes audio data corresponding to the original sound track and audio data corresponding to the accompaniment track. In response to the audio track switching instruction, the anchor client will obtain the audio data under the target audio track in the multi-track audio data through the integrated player, thereby sending the audio data and voice data under the target audio track to the live broadcast server. Among them, the target audio track is the audio track indicated by the audio track switching instruction. Optionally, the audio track switching instruction is triggered by the anchor client, for example, by the audio track switching operation of the anchor account.

[0109] In some embodiments, the integrated player also has the function of processing audio. In response to the audio processing instruction, the anchor client can process the audio data of the target audio according to the audio processing instruction through the integrated player, thereby obtaining the processed audio data. Wherein, the audio processing instruction includes at least one of the instructions for changing the tone of the target audio, the instructions for adjusting the volume of the target audio, and the instructions for adjusting the sound effect of the target audio. Optionally, the instructions for changing the tone of the target audio, the instructions for adjusting the volume of the target audio, and the instructions for adjusting the sound effect of the target audio are triggered by the anchor client, for example, the instruction for changing the tone of the target audio is triggered by the tone changing operation of the anchor account, the instruction for adjusting the volume of the target audio is triggered by the volume adjustment operation of the anchor account, and the instruction for adjusting the sound effect of the target audio is triggered by the sound effect adjustment operation of the anchor account.

[0110] For example, Figure 6 FIG. 1 is a schematic diagram of a user interface of an integrated player provided by an exemplary embodiment of the present application. Figure 6 As shown, a song list is shown in the user interface 601 of the integrated player to provide the function of the song list. A setting control 602 is also shown in the user interface 601, and a lyrics control, a volume adjustment control and a tone change control are shown in the setting control 602. The integrated player can be controlled to obtain the lyrics file of the target audio from the audio server through the lyrics control, and is provided to the anchor client to realize displaying desktop lyrics. The instruction of triggering the volume of the target audio can be realized by the volume adjustment control, and the instruction of triggering the tone change of the target audio can be realized by the tone change control.

[0111] In an illustrative example, the application process of the integrated player provided in the embodiment of the present application is introduced as follows:

[0112] 1. Different live broadcast platforms (anchor clients) access the integrated player through iframe. The program files corresponding to the integrated player include Hyper Text Markup Language (HTML) files, Cascading Style Sheets (CSS) files, JS (JavaScript, a programming language) files, wasm files, etc.

[0113] The decoding module based on wasm is implemented based on the open source computer program FFmpeg. By only turning on the mkv decoding function, the minimized wasm decoding function file can be compiled. In addition, the integrated player supports different skins customized for different live platforms to provide different display styles. Transmitting PCM data through the integrated player can simply and safely transmit audio data.

[0114] 2. After the anchor user selects a song, the integrated player will obtain the MKV multi-track audio file and the corresponding lyrics file (such as the KRC file) from the music library (audio server). The multi-track audio file contains at least two tracks: the accompaniment and the original sound. The integrated player decodes the multi-track audio file into multiple single-track MP3 format files through the above decoding module and puts them into memory. The decoding module based on wasm can efficiently decode the MKV file to avoid jamming.

[0115] 3. The host user can switch the accompaniment or original sound track through the user interface of the integrated player. The integrated player can switch to play different single-track mp3 files to achieve accompaniment or original sound playback. AudioWorklet (a JavaScript library) can also be used to process audio data to achieve different sound effects.

[0116] 4. The integrated player can process lyrics files by creating a Worker process, thereby achieving lyrics and music synchronization.

[0117] 5. The integrated player passes the audio data and lyrics data to the live broadcast platform through PostMessage (a function used to put a message into the message queue), and then distributes it to the corresponding audience client of the live broadcast room through the corresponding live broadcast server.

[0118] The method provided in this embodiment realizes a music player that can be reused by multiple live broadcast platforms, and each live broadcast platform can be easily and efficiently accessed, thereby simply realizing the functions of playing the original music and the accompaniment during the live broadcast.

[0119] In summary, the method provided in this embodiment integrates an integrated player provided by an audio service provider into the anchor client, thereby enabling the audio data of the target audio to be acquired through the integrated player. The anchor client provides the audio data of the target audio to the live broadcast server to synthesize the live broadcast stream and provide it to the audience client, so that the audience client can play the target audio according to the synthesized live broadcast stream. Since the process of acquiring the audio data of the target audio does not need to rely on the recording of the microphone, the anchor client can still acquire the audio data of the target audio even when the target audio is played through headphones. In the case where the target audio is background music, it can be achieved that the audience client can always play the background music.

[0120] The method provided in this embodiment also obtains audio files from an audio server through an integrated player, thereby decoding the audio files so that the host client can obtain audio data, which can avoid copyright risks of the obtained audio files. By transmitting PCM data corresponding to the audio files to the host client, the efficiency of transmitting audio data can be improved. By developing an integrated player through browser technology, the scope of application of the integrated player can be improved. By filtering the acquired voice data according to the acquired audio data, audio echo can be avoided. By enabling the integrated player to provide audio track switching and audio processing functions, it is possible to avoid additional development of such functions for the host client, thereby reducing the development cost of the live broadcast platform.

[0121] It should be noted that before collecting relevant data of users and during the process of collecting relevant data of users, this application can display a prompt interface, pop-up window or output voice prompt information, and the prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining relevant data of users after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining relevant data of users are terminated, that is, the relevant data of users are not obtained. In other words, all user data collected by this application are collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0122] It should be noted that the sequence of the method steps provided in the embodiments of the present application can be appropriately adjusted, and the steps can be increased or decreased accordingly according to the circumstances. Any technician familiar with the technical field can easily think of the changed methods within the technical scope disclosed in the present application, and they should be included in the protection scope of the present application, so they will not be repeated here.

[0123] Figure 7It is a structural diagram of a live stream synthesis device provided by an exemplary embodiment of the present application.

[0124] like Figure 7 As shown, the device comprises:

[0125] An acquisition module 701 is used to respond to an audio acquisition instruction and acquire audio data of a target audio through an integrated player, wherein the target audio is the audio indicated by the audio acquisition instruction, and the integrated player is integrated in the anchor client, and the integrated player is provided by an audio service provider other than the operator of the anchor client;

[0126] The acquisition module 701 is further used to acquire voice data, where the voice data includes data collected by a microphone connected to an anchor terminal, where the anchor terminal is the terminal where the anchor client is located;

[0127] The sending module 702 is used to send the audio data and the voice data to the live broadcast server, and the audio data and the voice data are used by the live broadcast server to synthesize a live broadcast stream.

[0128] In an alternative design, such as Figure 8 As shown, the device also includes:

[0129] The acquisition module 701 is used to acquire the audio file of the target audio in the audio server through the integrated player;

[0130] A decoding module 703, configured to decode the audio file of the target audio through the integrated player to obtain the audio data;

[0131] The acquisition module 701 is further used to acquire the audio data transmitted by the integrated player to the anchor client;

[0132] Wherein, a communication connection is established between the audio server and the integrated player, and the audio server is maintained by the audio service provider.

[0133] In an optional design, the decoding module 703 is used to:

[0134] The audio file of the target audio is decoded by the integrated player to obtain PCM data corresponding to the audio file.

[0135] In an optional design, the integrated player is developed based on browser technology so as to be integrated into the anchor client.

[0136] In an alternative design, such as Fig. 9 As shown, the device also includes:

[0137] The transmission module 704 is used for transmitting the audio data to the native player in response to the audio playing instruction when the audio data is not played, so that the native player calls the playing device to play the audio data.

[0138] In an alternative design, such as Fig.10 As shown, the device also includes:

[0139] The identification module 705 is used to obtain identification data by setting a filtering identification on the audio data when the audio data is played through the speaker;

[0140] A filtering module 706, configured to filter out the identification data from the acquired voice data in the process of acquiring the voice data, so as to obtain filtered voice data;

[0141] The sending module 702 is used to send the audio data and the filtered voice data to the live broadcast server.

[0142] In an optional design, the audio data is multi-track audio data; the acquisition module 701 is used to:

[0143] In response to the track switching instruction, obtaining audio data under a target track from the multi-track audio data through the integrated player, wherein the target track is the track indicated by the track switching instruction;

[0144] The sending module 702 is used to:

[0145] The audio data and the voice data under the target audio track are sent to the live broadcast server.

[0146] In an alternative design, such as Fig.11 As shown, the device also includes:

[0147] The processing module 707 is used to respond to the audio processing instruction and process the audio data of the target audio according to the audio processing instruction through the integrated player to obtain processed audio data;

[0148] The audio processing instruction includes at least one of an instruction for changing the pitch of the target audio, an instruction for adjusting the volume of the target audio, and an instruction for adjusting the sound effect of the target audio.

[0149] It should be noted that the live stream synthesis device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the live stream synthesis device provided in the above embodiment and the live stream synthesis method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0150] An embodiment of the present application also provides a computer device, which includes: a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the live stream synthesis method provided by the above-mentioned method embodiments.

[0151] In some embodiments, the computer device is a terminal. Fig.12 It is a schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application.

[0152] Typically, the terminal 1200 includes: a processor 1201 and a memory 1202 .

[0153] The processor 1201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1201 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1201 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1201 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1201 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0154] The memory 1202 may include one or more computer-readable storage media, which may be non-transitory. The memory 1202 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1202 is used to store at least one instruction, which is used to be executed by the processor 1201 to implement the live stream synthesis method provided in the method embodiment of the present application.

[0155] In some embodiments, the terminal 1200 may further optionally include: a peripheral device interface 1203 and at least one peripheral device. The processor 1201, the memory 1202 and the peripheral device interface 1203 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1203 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1204, a display screen 1205, a camera assembly 1206, an audio circuit 1207 and a power supply 1208.

[0156] The peripheral device interface 1203 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1201 and the memory 1202. In some embodiments, the processor 1201, the memory 1202, and the peripheral device interface 1203 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1201, the memory 1202, and the peripheral device interface 1203 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0157] The radio frequency circuit 1204 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1204 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1204 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 1204 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1204 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1204 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0158] The display screen 1205 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1205 is a touch display screen, the display screen 1205 also has the ability to collect touch signals on the surface or above the surface of the display screen 1205. The touch signal can be input to the processor 1201 as a control signal for processing. At this time, the display screen 1205 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1205 can be one, and the front panel of the terminal 1200 is set; in other embodiments, the display screen 1205 can be at least two, which are respectively set on different surfaces of the terminal 1200 or are folded; in some other embodiments, the display screen 1205 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 1200. Even, the display screen 1205 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1205 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0159] The camera assembly 1206 is used to capture images or videos. Optionally, the camera assembly 1206 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal 1200, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1206 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0160] The audio circuit 1207 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1201 for processing, or input them into the radio frequency circuit 1204 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1200. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1201 or the radio frequency circuit 1204 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1207 may also include a headphone jack.

[0161] The power supply 1208 is used to power various components in the terminal 1200. The power supply 1208 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1208 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0162] In some embodiments, the terminal 1200 further includes one or more sensors 1209 . The one or more sensors 1209 include, but are not limited to: an acceleration sensor 1210 , a gyroscope sensor 1211 , a pressure sensor 1212 , an optical sensor 1213 , and a proximity sensor 1214 .

[0163] The acceleration sensor 1210 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 1200. For example, the acceleration sensor 1210 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1201 can control the touch display screen 1205 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1210. The acceleration sensor 1210 can also be used for collecting game or user motion data.

[0164] The gyro sensor 1211 can detect the body direction and rotation angle of the terminal 1200, and the gyro sensor 1211 can cooperate with the acceleration sensor 1210 to collect the user's 3D actions on the terminal 1200. The processor 1201 can implement the following functions based on the data collected by the gyro sensor 1211: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0165] The pressure sensor 1212 can be set on the side frame of the terminal 1200 and / or the lower layer of the touch display screen 1205. When the pressure sensor 1212 is set on the side frame of the terminal 1200, it can detect the user's holding signal of the terminal 1200, and the processor 1201 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 1212. When the pressure sensor 1212 is set on the lower layer of the touch display screen 1205, the processor 1201 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 1205. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0166] The optical sensor 1213 is used to collect the ambient light intensity. In one embodiment, the processor 1201 can control the display brightness of the touch display screen 1205 according to the ambient light intensity collected by the optical sensor 1213. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 1205 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 1205 is reduced. In another embodiment, the processor 1201 can also dynamically adjust the shooting parameters of the camera assembly 1206 according to the ambient light intensity collected by the optical sensor 1213.

[0167] The proximity sensor 1214, also called a distance sensor, is usually arranged on the front panel of the terminal 1200. The proximity sensor 1214 is used to collect the distance between the user and the front of the terminal 1200. In one embodiment, when the proximity sensor 1214 detects that the distance between the user and the front of the terminal 1200 is gradually decreasing, the processor 1201 controls the touch display screen 1205 to switch from the screen-on state to the screen-off state; when the proximity sensor 1214 detects that the distance between the user and the front of the terminal 1200 is gradually increasing, the processor 1201 controls the touch display screen 1205 to switch from the screen-off state to the screen-on state.

[0168] Those skilled in the art will understand that Fig.12 The structure shown in the figure does not constitute a limitation on the terminal 1200, and the terminal 1200 may include more or fewer components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0169] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores at least one instruction, at least one program, code set or instruction set. When the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor of a computer device, the live stream synthesis method provided by the above-mentioned method embodiments is implemented.

[0170] The present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the live stream synthesis method provided by each of the above method embodiments.

[0171] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned readable storage medium may be a read-only memory, a disk or an optical disk, etc.

[0172] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent switching, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for synthesizing a live stream, characterized in that: The method is applied to a host client, and the method includes: In response to the audio acquisition instruction, audio data of a target audio is acquired through an integrated player, wherein the target audio is the audio indicated by the audio acquisition instruction, the integrated player is integrated in the host client, and the integrated player is provided by an audio service provider other than the operator of the host client; Acquire voice data, where the voice data includes data collected by a microphone connected to an anchor terminal, where the anchor terminal is a terminal where the anchor client is located; The audio data and the voice data are sent to a live broadcast server, and the audio data and the voice data are used by the live broadcast server to synthesize a live broadcast stream.

2. The method according to claim 1, characterized in that The step of obtaining audio data of the target audio through the integrated player includes: Obtaining the audio file of the target audio in the audio server through the integrated player; Decoding the audio file of the target audio through the integrated player to obtain the audio data; Acquire the audio data transmitted by the integrated player to the anchor client; Wherein, a communication connection is established between the audio server and the integrated player, and the audio server is maintained by the audio service provider.

3. The method according to claim 2, characterized in that The step of decoding the audio file of the target audio through the integrated player to obtain the audio data comprises: The integrated player decodes the audio file of the target audio to obtain pulse code modulation (PCM) data corresponding to the audio file.

4. The method according to any one of claims 1 to 3, characterized in that: The integrated player is developed based on browser technology so as to be integrated into the anchor client.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: In the case where the audio data is not played, in response to the audio playing instruction, the audio data is transmitted to the native player so that the native player calls a playing device to play the audio data.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the audio data is played through a speaker, obtaining identification data by setting a filtering identification on the audio data; In the process of acquiring the voice data, filtering out the identification data from the acquired voice data to obtain filtered voice data; The sending the audio data and the voice data to the live broadcast server includes: The audio data and the filtered voice data are sent to the live broadcast server.

7. The method according to any one of claims 1 to 3, characterized in that: The audio data is multi-track audio data; the method further comprises: In response to the track switching instruction, obtaining audio data under a target track from the multi-track audio data through the integrated player, wherein the target track is the track indicated by the track switching instruction; The sending the audio data and the voice data to the live broadcast server includes: The audio data and the voice data under the target audio track are sent to the live broadcast server.

8. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: In response to the audio processing instruction, the integrated player processes the audio data of the target audio according to the audio processing instruction to obtain processed audio data; The audio processing instruction includes at least one of an instruction for changing the pitch of the target audio, an instruction for adjusting the volume of the target audio, and an instruction for adjusting the sound effect of the target audio.

9. A live stream synthesis device, characterized in that: The device comprises: An acquisition module, configured to acquire audio data of a target audio through an integrated player in response to an audio acquisition instruction, wherein the target audio is the audio indicated by the audio acquisition instruction, and the integrated player is integrated in the anchor client, and the integrated player is provided by an audio service provider other than the operator of the anchor client; The acquisition module is further used to acquire voice data, where the voice data includes data collected by a microphone connected to an anchor terminal, where the anchor terminal is a terminal where the anchor client is located; The sending module is used to send the audio data and the voice data to the live broadcast server, and the audio data and the voice data are used by the live broadcast server to synthesize the live broadcast stream.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the live stream synthesis method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the live stream synthesis method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the live stream synthesis method as described in any one of claims 1 to 8.

Images