Interaction method, device, storage medium and electronic device in live broadcast

Abstract

This disclosure provides an interactive method, interactive device, computer-readable storage medium, and electronic device for live streaming, relating to the field of computer technology. The interactive method for live streaming includes: acquiring audio information from the live streaming room; sending the audio information to a server, so that the server, when audience interaction behavior information within the live streaming room meets preset conditions, sends live streaming interaction effect information to the audience in the live streaming room; or constructing ambient sound field information of the live streaming room based on the audio information, generating live streaming interaction effects matching the audio information based on the ambient sound field information, sending live streaming interaction effect information to the server, so that the server, when audience interaction behavior information within the live streaming room meets preset conditions, sends the live streaming interaction effect information to the audience in the live streaming room. This disclosure enhances the viewer's immersion in the live streaming room and improves the viewing experience.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to interactive methods, interactive devices, computer-readable storage media, and electronic devices in live streaming. Background Technology

[0002] Live streaming has developed and been applied rapidly in people's daily work and life. By creating an atmosphere in the live streaming room, the audience's sense of immersion in the live streaming room can be enhanced.

[0003] In related technologies, background music is played in the live streaming environment to create a certain atmosphere in the live streaming room; however, simply creating an atmosphere in the live streaming room by playing music cannot enhance the fun of the live streaming room, affects the viewing atmosphere, and further reduces the audience's sense of immersion and viewing experience in the live streaming room. Summary of the Invention

[0004] This disclosure provides interactive methods, interactive devices, computer-readable storage media, and electronic devices for live streaming, thereby improving, at least to some extent, the problem of low viewer immersion in live streaming.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to a first aspect of this disclosure, an interactive method for live streaming is provided, comprising: acquiring audio information of a live streaming room; sending the audio information to a server, such that the server, when audience interaction behavior information in the live streaming room meets preset conditions, sends live streaming room interactive effect information to a viewer in the live streaming room; the live streaming room interactive effect is generated based on environmental sound field information constructed from the audio information and matches the audio information; or constructing environmental sound field information of the live streaming room based on the audio information and generating a live streaming room interactive effect matching the audio information based on the environmental sound field information; sending the live streaming room interactive effect information to the server, such that the server, when audience interaction behavior information in the live streaming room meets preset conditions, sends the live streaming room interactive effect information to a viewer in the live streaming room; the live streaming room interactive effect information is used to present the live streaming room interactive effect on the viewer's end.

[0007] Optionally, sending the sound information to the server, so that the server, when the audience interaction behavior information in the live broadcast room meets preset conditions, sends live broadcast room interaction effect information to the audience in the live broadcast room, includes: sending the sound information to the server, so that the server constructs the ambient sound field information of the live broadcast room based on the sound information; receiving the ambient sound field information returned by the server, generating live broadcast room interaction effects based on the ambient sound field information; and sending the live broadcast room interaction effect information to the server, so that the server, when the audience interaction behavior information in the live broadcast room meets preset conditions, sends the live broadcast room interaction effect information to the audience in the live broadcast room.

[0008] Optionally, generating a live-stream interactive effect that matches the sound information based on the ambient sound field information includes: when the current interactive mode of the live-stream is a virtual camera movement interactive mode, determining the camera movement trajectory based on the ambient sound field information, using the camera movement trajectory to control the virtual camera to move and capture the virtual scene of the live-stream, and generating a live-stream interactive effect based on the captured image; the virtual scene matches the real space of the live-stream.

[0009] Optionally, determining the camera motion trajectory based on the ambient sound field information includes: determining the coordinates of the strongest sound point in the ambient sound field information; and determining the camera coordinates and camera plane at the next moment based on the angle between the current camera coordinates and the coordinates of the strongest sound point, and the projection of the coordinates of the strongest sound point onto the camera plane at the current moment.

[0010] Optionally, determining the camera motion trajectory based on the ambient sound field information further includes: after determining the camera coordinates and camera plane at the next moment, adding a perturbation in a preset direction to the camera coordinates at the next moment based on the overall sound field intensity in the ambient sound field information, so as to update the camera coordinates and camera plane at the next moment.

[0011] Optionally, generating a live-stream interactive effect that matches the sound information based on the ambient sound field information includes: when the current interactive mode of the live-stream is a virtual element interactive mode, determining the sound intensity statistics in each unit space of the virtual scene of the live-stream based on the ambient sound field information, and rendering virtual elements with sizes matching the sound intensity statistics in each unit space to generate a live-stream interactive effect; the virtual scene matches the real space of the live-stream.

[0012] Optionally, determining the sound intensity statistics in each unit space of the virtual scene of the live broadcast room based on the environmental sound field information includes: dividing the virtual scene of the live broadcast room into multiple unit spaces along the first axis, calculating the average sound intensity along the second and third axes in each unit space, and obtaining the sound intensity statistics in each unit space.

[0013] Optionally, before generating the live room interactive effect based on the ambient sound field information, the method further includes: receiving interactive mode indication information sent by the server, wherein the interactive mode indication information is determined by the server based on the audience interactive behavior information; and setting the current interactive mode of the live room based on the interactive mode indication information.

[0014] Optionally, if the sound information is two-dimensional sound data, then constructing the environmental sound field information of the live broadcast room based on the sound information includes: converting the two-dimensional sound data into three-dimensional sound data by performing third-dimensional interpolation on the two-dimensional sound data; and determining the sound intensity data of multiple coordinate points in the real space of the live broadcast room based on the three-dimensional sound data, so as to construct the environmental sound field information of the live broadcast room.

[0015] Optionally, the step of constructing the ambient sound field information of the live broadcast room based on the sound information and generating a live broadcast room interactive effect that matches the sound information based on the ambient sound field information includes: responding to receiving an interactive trigger instruction information issued by the server when the interactive behavior information meets preset conditions, constructing the ambient sound field information of the live broadcast room based on the sound information, and generating a live broadcast room interactive effect that matches the sound information based on the ambient sound field information.

[0016] According to a second aspect of this disclosure, an interactive method for live streaming is provided, comprising: acquiring audience interaction behavior information in a live streaming room; and, when the audience interaction behavior information meets preset conditions, sending information about live streaming room interaction effects to the audience terminal in the live streaming room; wherein the live streaming room interaction effects are generated based on environmental sound field information constructed from the sound information of the broadcaster terminal and are matched with the sound information of the broadcaster terminal; and wherein the information about the live streaming room interaction effects is used to present the live streaming room interaction effects on the audience terminal.

[0017] Optionally, the method further includes: receiving the sound information sent by the broadcaster; constructing the ambient sound field information of the live broadcast room based on the sound information; returning the ambient sound field information to the broadcaster, so that the broadcaster generates interactive effects in the live broadcast room based on the ambient sound field information; and receiving information about the interactive effects in the live broadcast room sent by the broadcaster.

[0018] Optionally, returning the ambient sound field information to the broadcaster terminal, so that the broadcaster terminal generates live room interactive effects based on the ambient sound field information, includes: when the audience interactive behavior information meets preset conditions, sending the ambient sound field information and interaction trigger indication information to the broadcaster terminal, so that the broadcaster terminal responds to the interaction trigger indication information and generates live room interactive effects based on the ambient sound field information.

[0019] Optionally, the method further includes: receiving the sound information sent by the broadcaster; constructing the ambient sound field information of the live broadcast room based on the sound information; and generating the interactive effects of the live broadcast room based on the ambient sound field information.

[0020] Optionally, the method further includes: determining interaction mode indication information based on the audience interaction behavior information; sending the interaction mode indication information to the broadcaster terminal, so that the broadcaster terminal sets the current interaction mode of the live broadcast room according to the interaction mode indication information; and matching the type of the live broadcast room interaction effect with the current interaction mode.

[0021] According to a third aspect of this disclosure, an interactive method for live streaming is provided, comprising: sending audience interaction behavior information of the audience terminal in the live streaming room to a server; receiving information on the live streaming room interaction effect issued by the server when the audience interaction behavior information meets preset conditions; the live streaming room interaction effect is generated based on environmental sound field information constructed from the sound information of the broadcaster terminal and is matched with the sound information of the broadcaster terminal in the live streaming room; and presenting the live streaming room interaction effect based on the information of the live streaming room interaction effect.

[0022] According to a fourth aspect of this disclosure, an interactive device for live streaming is provided, comprising: a sound information acquisition module configured to acquire sound information of a live streaming room; and a data transmission module configured to send the sound information to a server, such that the server, when the audience interaction behavior information in the live streaming room meets preset conditions, sends live streaming room interaction effect information to a viewer in the live streaming room; the live streaming room interaction effect is generated based on environmental sound field information constructed from the sound information and matches the sound information; or the environmental sound field information of the live streaming room is constructed based on the sound information, and a live streaming room interaction effect matching the sound information is generated based on the environmental sound field information; the live streaming room interaction effect information is sent to the server, such that the server, when the audience interaction behavior information in the live streaming room meets preset conditions, sends the live streaming room interaction effect information to a viewer in the live streaming room; and the live streaming room interaction effect information is used to present the live streaming room interaction effect on the viewer's end.

[0023] According to a fifth aspect of this disclosure, an interactive device for live streaming is provided, comprising: a viewer interaction behavior information acquisition module configured to acquire viewer interaction behavior information within a live streaming room; and an interaction effect sending module configured to send live streaming room interaction effect information to a viewer in the live streaming room when the viewer interaction behavior information meets preset conditions; the live streaming room interaction effect is generated based on environmental sound field information constructed from the voice information of the broadcaster, and matches the voice information of the broadcaster; the live streaming room interaction effect information is used to present the live streaming room interaction effect on the viewer's end.

[0024] According to a sixth aspect of this disclosure, an interactive device for live streaming is provided, comprising: a viewer interaction behavior information sending module configured to send viewer interaction behavior information of the viewer terminal in the live streaming room to a server; an interaction effect receiving module configured to receive live streaming room interaction effect information sent by the server when the viewer interaction behavior information meets preset conditions; the live streaming room interaction effect is generated based on environmental sound field information constructed from the voice information of the broadcaster terminal and matches the voice information of the broadcaster terminal in the live streaming room; and an interaction effect presentation module configured to present the live streaming room interaction effect based on the information of the live streaming room interaction effect.

[0025] According to a seventh aspect of this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the interactive method in the live broadcast described in the first, second, or third aspect, and its possible implementations.

[0026] According to an eighth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor. The processor is configured to execute the interactive methods and possible implementations of the live streaming described in the first, second, or third aspects above by executing the executable instructions.

[0027] The technical solution disclosed herein has the following beneficial effects:

[0028] On the one hand, the broadcaster sends the acquired audio information from the live stream to the server. When the viewer's interactive behavior information meets preset conditions, the server sends the viewer information matching the audio information, providing feedback on the interactive effect. This strengthens the correlation between viewer behavior and live stream interaction, creating a positive viewing atmosphere and enhancing viewer immersion. On the other hand, the broadcaster can generate live stream interaction effects that match the audio information, further strengthening the correlation between the interaction effect and the audio information, increasing the live stream's fun and diversity, and effectively improving the viewer's viewing experience.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0031] Figure 1 This illustrates the system architecture of the operating environment for this exemplary embodiment;

[0032] Figure 2 This diagram illustrates the process of an interactive method applied to a broadcaster's end in a live stream according to this exemplary embodiment.

[0033] Figure 3 This diagram illustrates a process in which a broadcaster generates interactive effects based on sound information in an exemplary embodiment of the present invention.

[0034] Figure 4 This diagram illustrates the process of setting the current interactive mode of the live broadcast room according to the interactive mode indication information in this exemplary embodiment.

[0035] Figure 5 This diagram illustrates the process of determining the camera's motion trajectory based on ambient sound field information in this exemplary embodiment.

[0036] Figure 6 This diagram illustrates the process of constructing the environmental sound field information of a live streaming room in this exemplary embodiment.

[0037] Figure 7 This diagram illustrates how the sound loudness of a coordinate point is determined using the three-point positioning method in this exemplary embodiment.

[0038] Figure 8 This diagram illustrates the process of an interactive method applied to a live stream on a server in this exemplary embodiment.

[0039] Figure 9 This diagram illustrates the process by which the server receives information about the interactive effects in the live stream sent by the broadcaster in this exemplary embodiment.

[0040] Figure 10 This diagram illustrates the process by which the server generates interactive effects in the live streaming room based on sound information in this exemplary embodiment.

[0041] Figure 11 This diagram illustrates the process of an interactive method applied to a viewer's end in a live broadcast according to this exemplary embodiment;

[0042] Figure 12 This example illustrates an interaction flowchart of an interaction method in live streaming according to this exemplary embodiment;

[0043] Figure 13 This illustration shows a structural diagram of an interactive device applied to a broadcaster's terminal in a live broadcast according to an exemplary embodiment of the present invention;

[0044] Figure 14 This illustration shows a structural diagram of an interactive device applied to a live broadcast on a server in this exemplary embodiment.

[0045] Figure 15 This illustration shows a structural diagram of an interactive device used in live streaming on the viewer's end in this exemplary embodiment;

[0046] Figure 16 A schematic diagram of the structure of an electronic device in this exemplary embodiment is shown. Detailed Implementation

[0047] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0048] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0049] In related technologies, background music is played in the live streaming room to create an atmosphere. However, the inventors have found that playing music in the live streaming room fails to enhance the connection between the music and the audience's interactive behavior, resulting in poor interactivity between the live stream and the audience; relying solely on music to create atmosphere in the live streaming room leads to a low correlation between the music and the live streaming visuals, resulting in poor user immersion; and the special effects in related technologies are limited to fixed, concentrated effects, resulting in low entertainment value, making the live streaming experience monotonous for viewers, easily causing aesthetic fatigue, thus affecting the viewing atmosphere and further reducing the viewer's viewing experience in the live streaming room.

[0050] In view of one or more of the above-mentioned problems, this disclosure first provides an exemplary implementation of a method for interaction in live streaming. The following is in conjunction with... Figure 1 The system architecture of the operating environment for this exemplary embodiment will be described.

[0051] refer to Figure 1 As shown, the system architecture 100 may include a broadcast client 110, a server 120, and a viewer client 130. The broadcast client 110 and the viewer client 130 can be electronic devices such as laptops, desktop computers, smartphones, and tablets. The broadcast client 110 can be used to receive audio information from the live broadcast room, and the viewer client 130 can be used to receive viewer interaction behavior information. The server 120 generally refers to the backend system that provides interaction-related services in the live broadcast in this exemplary embodiment, such as a server that implements the interaction methods in the live broadcast. The server 120 can be a single server or a cluster of multiple servers; this disclosure does not limit this. The broadcast client 110, the server 120, and the viewer client 130 can be connected via wired or wireless communication links to exchange data.

[0052] In one implementation, the aforementioned live streaming room can be a virtual live streaming room where the streamer is located. The viewer terminal 130 can send the acquired viewer interaction behavior information to the server terminal 120. The streamer terminal 110 can acquire the audio information of the live streaming room and send the audio information to the server terminal 120. If the viewer interaction behavior information sent by the viewer terminal 130 meets the preset conditions, the server terminal 120 can send the viewer terminal 130 information on the interactive effect that matches the audio information. After receiving the interactive effect information sent by the server terminal 120, the viewer terminal 130 displays the corresponding interactive effect to the viewer.

[0053] In one implementation, the broadcaster terminal 110 can also construct the ambient sound field information of the live broadcast room based on the sound information, generate live broadcast room interactive effects that match the sound information based on the ambient sound field information, and send the information of the live broadcast room interactive effects to the server terminal 120. The server terminal 120 can send the information of the live broadcast room interactive effects to the viewer terminal 130 when the viewer interactive behavior information sent by the viewer terminal 130 meets the preset conditions. The viewer terminal 130 can render the information of the live broadcast room interactive effects to present the interactive effects to the viewer. Through the interaction of the broadcaster terminal 110, the server terminal 120 and the viewer terminal 130, an interactive atmosphere is created in the live broadcast room, thereby enhancing the viewer's immersion in the live broadcast room.

[0054] The following describes the interactive method in live streaming in this exemplary embodiment from three aspects: the broadcaster's end 110, the server end 120, and the viewer's end 130.

[0055] Figure 2 An exemplary flow of an interaction method applied to a live stream on the broadcaster's terminal 110 is shown, including the following steps S210 to S220:

[0056] Step S210: Obtain the audio information from the live stream;

[0057] Step S220: The sound information is sent to the server, so that the server sends the live room interaction effect information to the audience in the live room when the audience interaction behavior information in the live room meets the preset conditions; the live room interaction effect is generated based on the environmental sound field information constructed by the sound information and matches the sound information.

[0058] Step S230: Construct ambient sound field information for the live broadcast room based on the sound information; generate live broadcast room interactive effects that match the sound information based on the ambient sound field information; send the live broadcast room interactive effect information to the server; and when the audience interaction behavior information in the live broadcast room meets the preset conditions, the server sends the live broadcast room interactive effect information to the audience in the live broadcast room; the live broadcast room interactive effect information is used to present the live broadcast room interactive effects on the audience's end.

[0059] Based on the above method, on the one hand, the broadcaster sends the acquired audio information of the live broadcast room to the server. When the audience interaction behavior information sent by the audience meets preset conditions, the server sends information about the live broadcast room's interactive effects that match the audio information to the audience. That is, when the audience interaction behavior information meets preset conditions, the server provides feedback on the interactive effects to the audience, strengthening the correlation between audience behavior and live broadcast room interactive effects, creating a positive viewing atmosphere, and further enhancing the audience's immersion in the live broadcast room. On the other hand, the broadcaster can generate live broadcast room interactive effects that match the audio information, enhancing the correlation between the interactive effects and the audio information of the live broadcast room, increasing the fun and diversity of the live broadcast room, and effectively improving the audience's viewing experience.

[0060] The following is about Figure 2 Each step in the process will be explained in detail.

[0061] refer to Figure 2 In step S210, the audio information of the live broadcast room is obtained.

[0062] The aforementioned live streaming room may include a virtual live streaming room opened when the streamer switches to a virtual scene, or it may include a live streaming room in the streamer's real environment. This disclosure does not impose any special limitations on this. For example, the virtual live streaming room in which the streamer is located may include a virtual live streaming scene rendered by UE (Unreal Engine). The aforementioned sound information may include sound information from the streamer's real environment. For example, the sound information may include ambient sound from the live streaming scene collected by the microphone or other audio recording equipment used by the streamer, and may also include sound intensity information obtained based on the ambient sound.

[0063] In one implementation, when the broadcaster switches to a virtual live streaming scene on the broadcaster's end, the broadcaster's end can collect ambient sound from the live streaming scene in real time through the broadcaster's microphone, and send the sound intensity information obtained based on the ambient sound to the server in real time. After receiving the sound intensity information, the server can send the interactive effect information of the live streaming room that matches the sound intensity information to the audience end if the audience's interactive behavior information meets the preset conditions.

[0064] Continue to refer to Figure 2 In step S220, the sound information is sent to the server so that the server sends the live room interaction effect information to the audience in the live room when the audience interaction behavior information in the live room meets the preset conditions; the live room interaction effect is generated based on the environmental sound field information constructed by the sound information and matches the sound information.

[0065] The audience interaction behavior information can be a specified action taken by viewers on the viewer's end during the live stream. This disclosure does not specifically limit the content of the audience interaction behavior information; for example, it could be the viewer clicking the "send gift" button to send a gift to the streamer during the live stream, or it could be the viewer continuously "liking" the live stream. The preset conditions can be evaluation conditions for the audience's interaction behavior on the viewer's end during the live stream. For example, preset conditions could be the amount of the gift sent to the streamer, or the number of consecutive likes given by the viewer in the live stream. The information on the live stream interaction effect can be visual information related to the live stream interaction effect matched with the sound information, such as information on changes in interactive elements. For example, the interaction effect information could include visual information of a virtual camera automatically moving in sync with the music rhythm in the live stream, or information on the coordinates and colors of interactive elements such as sheet music and lighting changing with the music intensity. Ambient sound field information can be a virtual space constructed based on sound information that can simulate the real sound environment in which the broadcaster is located. Ambient sound field information can include coordinate information of the virtual space, such as the coordinates of the sound source, the coordinates of the listener (such as the broadcaster), and the boundary coordinates of the virtual space. The broadcaster can further generate interactive effects in the live broadcast room based on the ambient sound field information.

[0066] In one implementation, the server can collect the ID of the virtual scene used by the live streaming room, determine whether the live streaming room supports "interactive mode" based on the virtual scene ID, and send the determination result to the viewer's end. If the viewer's end receives a message indicating that the current virtual scene supports "interactive mode," then the server displays an "interactive mode entry" to the viewers in the live streaming room. Viewers can click on the "interactive mode entry" to view the gameplay instructions, the activation status, and the activation progress of the interactive mode on the interactive mode description interface. Viewers can continue to send gifts to the streamer on the interactive mode description interface, and the viewer's end can receive the gifts in real time and upload information such as the amount of the gifts to the server. If the server determines... If the total value of gifts given by viewer A in the live stream reaches a preset threshold, an "interaction mode selection interface" will be displayed on viewer A's device. For example, interaction modes may include "virtual camera movement" and "interactive elements." The viewer's device will upload the "virtual camera movement" interaction mode information selected by viewer A to the server. The server will then send the corresponding live stream interaction effect information to the viewer's device based on the interaction mode information selected by viewer A. The viewer's device will then present a virtual camera that moves according to the rhythm of the music in the live stream, creating a new 3D effect of "nearer objects appearing larger and farther objects appearing smaller" in the current live stream scene. This will enhance the connection between the live stream and the viewer, thereby increasing the viewer's immersion in the live stream and further improving the user's viewing experience.

[0067] In step S220, the interactive effects in the live broadcast room can be generated jointly by the server and the broadcaster based on the audio information, or the interactive effects can be generated solely by the server based on the audio information. The two scenarios are explained below.

[0068] First, the specific process of generating live stream interaction effect information jointly by the server and the broadcaster is explained. In one implementation, the aforementioned audio information is sent to the server, and when the viewer's interactive behavior information in the live stream meets preset conditions, the server sends live stream interaction effect information to the viewer in the live stream, such as... Figure 3 As shown, steps S310 to S330 may be included:

[0069] Step S310: Send the sound information to the server so that the server can construct the ambient sound field information of the live broadcast room based on the sound information;

[0070] Step S320: Receive the ambient sound field information returned by the server, and generate interactive effects for the live broadcast room based on the ambient sound field information;

[0071] Step S330: Send the information on the interactive effect of the live room to the server, so that the server sends the information on the interactive effect of the live room to the viewers in the live room when the audience's interactive behavior information in the live room meets the preset conditions.

[0072] In step S310, the sound information can be sent to the server so that the server can construct the ambient sound field information of the live broadcast room based on the sound information. After the server constructs the ambient sound field information of the live broadcast room, it can send the ambient sound field information to the broadcaster so that the broadcaster can receive the ambient sound field information returned by the server in step S320 and generate interactive effects in the live broadcast room based on the ambient sound field information.

[0073] In one implementation, after receiving the ambient sound field information returned by the server, the broadcaster can automatically distinguish the interaction mode based on the ambient sound field information. For example, the broadcaster can play music from a pre-set music library in the live broadcast room; the server can generate an ambient sound field information library based on the ambient sound field information corresponding to each piece of music in the music library; after the broadcaster obtains the sound information of the live broadcast room, it uploads the sound information to the server, the server can construct the ambient sound field information of the live broadcast room based on the sound information, and then match the ambient sound field information with the ambient sound field information library to obtain the ambient sound field information ID, and send the ambient sound field information ID to the broadcaster; after receiving the ambient sound field information ID, the broadcaster can select the corresponding interaction mode based on the ambient sound field information ID to set the current interaction mode of the live broadcast room.

[0074] In one implementation, the current interaction mode of the live stream can also be set according to the interaction mode selected by the audience. Before generating the live stream interaction effects based on the ambient sound field information, such as... Figure 4 As shown, the interactive methods in the above live streaming can also include the following steps:

[0075] Step S410: Receive interaction mode indication information sent by the server. The interaction mode indication information is determined by the server based on the audience's interaction behavior information.

[0076] Step S420: Set the current interaction mode of the live broadcast room according to the interaction mode instruction information.

[0077] The interaction mode can be information about the type of interactive effect, such as "virtual camera movement" or "change of interactive elements." The interaction mode indication information can be information used to indicate the interaction mode selected by the viewer on the viewer's end; for example, the interaction mode indication information can include the ID of the interaction mode.

[0078] In step S410, the broadcaster can receive the interaction mode indication information sent by the server, and then in step S420, the current interaction mode of the live broadcast room is set according to the interaction mode indication information, so that the live broadcast room displays the interaction effect corresponding to the interaction mode information.

[0079] For example, viewers can select the desired interactive mode in the "Interactive Mode Selection Interface" displayed on the viewer's end. After receiving the interactive mode ID selected by the viewer, the viewer's end can upload the interactive mode ID to the server. The server can generate interactive mode instruction information based on the interactive mode ID and send the interactive mode instruction information to the broadcaster's end. After receiving the interactive mode instruction information, the broadcaster's end can set the current interactive mode of the live broadcast room according to the interactive mode instruction information.

[0080] based on Figure 4 The method sets the current interaction mode of the live broadcast room according to the interaction mode instruction information to generate interactive effects that viewers like, thereby enhancing the fun and diversity of the live broadcast room and improving the user's viewing experience.

[0081] The following describes the specific process of generating interactive effects for the live stream based on ambient sound field information in step S320. In one embodiment, generating interactive effects for the live stream that match the sound information based on ambient sound field information may include the following steps:

[0082] When the current interactive mode in the live broadcast room is virtual camera movement interactive mode, the camera movement trajectory is determined based on the ambient sound field information. The camera movement trajectory is used to control the virtual camera to move and shoot the virtual scene of the live broadcast room. The live broadcast room interactive effect is generated based on the captured images; the virtual scene matches the real space of the live broadcast room.

[0083] The virtual camera movement interaction mode allows for adjustments to the movement trajectory of a virtual camera within the live stream, creating a dynamic and interactive virtual scene. The camera movement trajectory can be the movement trajectory of a virtual camera within the virtual scene. The virtual scene of the live stream can include a scene obtained by virtualizing the real space of the live stream, or a virtual scene constructed based on preset virtual elements. This disclosure does not impose any specific limitations on this. For example, the virtual scene can be a virtual live stream scene constructed based on virtual smoke elements, virtual lighting elements, and virtual background wall elements.

[0084] In one implementation, since it's impossible to directly render virtual live-stream interactive effects in real space, a virtual scene matching the real space of the live-stream can be created. The three-dimensional coordinate system of this virtual scene can be consistent with the three-dimensional coordinate system of the real space, allowing objects in the virtual scene to correspond to objects in the real space, thus achieving spatial matching between the virtual scene and the live-stream. For example, the ambient sound field information can be ambient sound field information from the real space. Since the virtual scene matches the real space, the ambient sound field information can be used as the ambient sound field information in the virtual scene, thereby obtaining information such as sound distribution in the virtual scene, and subsequently determining the camera motion trajectory within the virtual scene.

[0085] Furthermore, from the perspective of protecting the personal privacy of the streamer, a virtual scene can be created to present the superimposed effect of the streamer being in a virtual scene, thereby achieving the effect of the streamer broadcasting in a virtual scene, which not only protects the personal privacy of the streamer, but also makes the live broadcasting method more interesting.

[0086] The method described above, which sets the current interactive mode of the live broadcast room to virtual camera movement interactive mode based on the ambient sound field information, adds dynamic effects to the live broadcast room and enhances its fun.

[0087] In one implementation, such as Figure 5 As shown, determining the camera motion trajectory based on the ambient sound field information may include steps S510 to S520:

[0088] Step S510: Determine the coordinates of the strongest sound point in the ambient sound field information;

[0089] Step S520: Determine the camera coordinates and camera plane at the next moment based on the angle between the current camera coordinates and the coordinates of the strongest sound point, and the projection of the strongest sound point coordinates onto the camera plane at the current moment.

[0090] The coordinates of the strongest sound point and the camera coordinates at the current moment can be three-dimensional coordinates within the virtual scene of the current live stream. The coordinates of the strongest sound point are the location with the loudest sound in the virtual scene of the current live stream. The camera plane can be a plane parallel to the virtual camera lens, or a plane on which the virtual camera can move. The camera plane can be adjusted by modifying the camera's shooting angle and shooting trajectory.

[0091] For example, the ambient sound field information of the current live broadcast room can be a three-dimensional coordinate system corresponding to the virtual scene of the live broadcast room, and each coordinate point of the three-dimensional coordinate system carries the sound intensity information of that coordinate point; the coordinates of the strongest sound point can be determined in the ambient sound field information through step S510; the camera plane at the current moment can be the XY plane in the three-dimensional coordinate system, that is, the plane parallel to the anchor's image in the virtual scene of the live broadcast room (a plane located in front of the anchor); through step S520, the angle between the camera coordinates at the current moment and the coordinates of the strongest sound point can be calculated, and this angle can be determined as the Z-axis orientation of the camera at the next moment, that is, the orientation of the camera's camera at the next moment. At this time, the orientation of the virtual camera's camera is aligned with the coordinates of the strongest sound point; based on the projection of the strongest sound point coordinates onto the camera plane at the current moment, the camera coordinates at the next moment can be determined, that is, the coordinates of the camera at the next moment on the XY plane at the next moment; based on the plane where the camera coordinates at the next moment are located, the camera plane at the next moment can be determined.

[0092] based on Figure 5 The method determines the camera coordinates and camera plane at the next moment based on the coordinates of the strongest sound point in the ambient sound field information. This allows the virtual camera's shooting angle to be positioned at the location with the loudest sound in the virtual scene of the live broadcast room, thereby enhancing the interactive effect of the live broadcast room.

[0093] In one embodiment, the above-mentioned determination of the camera motion trajectory based on ambient sound field information may further include the following steps:

[0094] After determining the camera coordinates and camera plane at the next moment, based on the overall sound field intensity in the ambient sound field information, a perturbation in a preset direction is added to the camera coordinates at the next moment to update the camera coordinates and camera plane at the next moment.

[0095] For example, after obtaining the camera coordinates for the next moment, the camera coordinates for the next moment can be perturbed in the Z-axis direction according to the changes in the music rhythm in the current live broadcast room. That is, the camera can be randomly displaced on the Z-axis to increase the randomness and sense of motion of the virtual camera movement interaction mode, making the interactive effect of the live broadcast room more vivid.

[0096] In one implementation, several camera coordinates can be generated based on the changes in the music rhythm during the live stream. A relatively smooth camera motion curve is then obtained from these coordinates, which can be used to determine the camera's trajectory. This allows for an interactive effect where the camera's movement follows the music rhythm during the interactive mode's activation period. The activation period of the interactive mode can be determined based on viewer interaction information; for example, the higher the amount of money viewers spend on virtual gifts or rewards during the live stream, the longer the interactive mode will be active.

[0097] The following describes the specific process of generating the interactive effect corresponding to the "virtual element interaction mode" based on the ambient sound field information. In one embodiment, the above-mentioned generation of live room interactive effects that match the sound information based on the ambient sound field information may include the following steps:

[0098] When the current interactive mode of the live broadcast room is virtual element interactive mode, the sound intensity statistics of each unit space in the virtual scene of the live broadcast room are determined according to the ambient sound field information. Virtual elements with sizes matching the sound intensity statistics are rendered in each unit space to generate interactive effects in the live broadcast room.

[0099] In one implementation, determining the sound intensity statistics within each unit space of the virtual scene in the live streaming room based on ambient sound field information may include the following steps:

[0100] The virtual scene of the live broadcast room is divided into multiple unit spaces along the first axis. The average sound intensity along the second and third axes in each unit space is calculated to obtain the sound intensity statistics for each unit space.

[0101] The first, second, and third axes can be coordinate axes of a virtual scene. For example, the first axis can be the X-axis in three-dimensional space, the second axis can be the Y-axis in three-dimensional space, and the third axis can be the Z-axis in three-dimensional space. The X-axis can be divided into multiple unit spaces, and the average sound intensity of the Y-axis and Z-axis in each unit space can be statistically analyzed to obtain the sound intensity statistics for each unit space.

[0102] In one implementation, the broadcaster can obtain data such as the model, material, and texture of the virtual element to be added based on a preset interactive effect template. In the virtual scene of the live broadcast room, the X-axis is divided into multiple unit spaces. The sound intensity of the Y-axis and Z-axis within any unit space is weighted and averaged to obtain the height and depth of the audio histogram within that unit space. This audio histogram within any unit space is then defined as the sound intensity statistics within that unit space. This process of obtaining sound intensity statistics is repeated continuously to obtain sound intensity statistics for all unit spaces along the X-axis. Based on the sound intensity statistics, the sound spectrum data within that space is determined. Combined with the data of the virtual element to be added, a model of the virtual element to be added can be generated. During the interactive mode's active period, the process of generating the virtual element model is repeated to obtain a 3D effect where the virtual element changes with the music rhythm during the interactive mode's active period. This makes the live broadcast more diverse and engaging, increases its appeal to viewers, and further enhances the viewers' immersion in the live broadcast room.

[0103] Based on the method in step S320, the broadcaster can generate interactive effects in the live broadcast room according to the ambient sound field information, realizing the linkage between the interactive effects in the live broadcast room and the broadcaster's ambient sound, enhancing the linkage between the audience's interactive behavior and the interactive effects in the live broadcast room, improving the fun of the live broadcast and enhancing the user's viewing experience.

[0104] Continue to refer to Figure 3 After the host generates the interactive effect, in step S330, the host can send the information of the interactive effect of the live room to the server, so that the server can send the information of the interactive effect of the live room to the audience in the live room when the audience's interactive behavior information in the live room meets the preset conditions.

[0105] based on Figure 2 One method involves the broadcaster acquiring audio information and sending it to the server. The server then generates ambient sound field information, and the broadcaster, upon receiving this information, generates interactive effects for the live stream. Alternatively, to reduce the number of interactions between the broadcaster and server, the broadcaster can construct ambient sound field information for the live stream based on the acquired audio information, generate interactive effects, and then send this information to the server. This allows the server to send live stream interactive effects matching the audio information to the viewers when their interaction behavior meets preset conditions.

[0106] In one implementation, if the aforementioned sound information is two-dimensional sound data, then the aforementioned construction of the environmental sound field information of the live broadcast room based on the sound information, such as... Figure 6 As shown, steps S610 to S620 may be included:

[0107] Step S610: The two-dimensional sound data is converted into three-dimensional sound data by performing third-dimensional interpolation on the two-dimensional sound data.

[0108] Among them, the three-dimensional sound data can be obtained by the stereo sound collected by the sound sensor, that is, the sound data belongs to stereo sound; since some sound sensors do not support the collection of stereo sound, the two-dimensional sound data collected can be linearly interpolated by the method in step S610, and the third-dimensional sound data can be determined according to the result of the linear interpolation, so as to obtain the three-dimensional sound data corresponding to the two-dimensional sound data.

[0109] After obtaining the three-dimensional sound data based on the sound information, in step S620, the sound intensity data of multiple coordinate points within the real space of the live broadcast room can be determined based on the three-dimensional sound data to construct the environmental sound field information of the live broadcast room. For example, such as Figure 7 As shown, the sound intensity data of each coordinate point in three-dimensional space can be calculated using the cosine theorem, and the environmental sound field information of the live broadcast room can be determined based on the sound intensity data of each coordinate point.

[0110] In one implementation, the broadcaster's microphone is used as a sound sensor in the live streaming room to collect sound information. If the microphone supports stereo capture, the sound information acquired by the broadcaster is three-dimensional sound data. The sound intensity data at multiple coordinate points within the live streaming room can be directly determined based on this three-dimensional sound data to construct the environmental sound field information of the live streaming room. If the microphone does not support stereo capture, the sound information collected by the microphone is two-dimensional sound data. Interpolation can be performed on the two-dimensional sound data along the X and Y axes, and an approximate value for the three-dimensional sound data along the Z axis can be obtained based on the interpolation result, thus obtaining the three-dimensional sound data corresponding to the two-dimensional sound data. The sound intensity data at multiple coordinate points within the live streaming room can then be further determined based on this three-dimensional sound data to construct the environmental sound field information of the live streaming room.

[0111] The following explains the process by which the server generates interactive effects for the live stream based on audio information.

[0112] In one implementation, the aforementioned sending of audio information to the server, so that the server, when the audience interaction behavior information in the live broadcast room meets preset conditions, sends live broadcast room interaction effect information to the audience in the live broadcast room, may include the server, after receiving the audio information sent by the broadcaster, constructing the ambient sound field information of the live broadcast room based on the audio information, generating live broadcast room interaction effect information based on the ambient sound field information, and sending the live broadcast room interaction effect information to the audience in the live broadcast room when the audience interaction behavior information in the live broadcast room meets preset conditions, so that the audience can watch the live broadcast room interaction effect on their viewing end. In this exemplary implementation, the server can directly generate the live broadcast room interaction effect based on the audio information, reducing the number of interactions between the broadcaster and the server, which is beneficial to improving the overall efficiency of the interaction method in the live broadcast.

[0113] In addition, the broadcaster can directly generate interactive effects in the live stream based on the audio information, and continue to refer to... Figure 2 In step S230, the broadcaster can construct the ambient sound field information of the live broadcast room based on the sound information, generate live broadcast room interactive effects that match the sound information based on the ambient sound field information, and send the live broadcast room interactive effect information to the server. When the audience interaction behavior information in the live broadcast room meets preset conditions, the server sends the live broadcast room interactive effect information to the audience in the live broadcast room. The live broadcast room interactive effect information is used to present the live broadcast room interactive effects on the audience's end.

[0114] In one implementation, the above-mentioned process of constructing the ambient sound field information of the live broadcast room based on the sound information, and generating interactive effects in the live broadcast room that match the sound information based on the ambient sound field information, may include the following steps:

[0115] In response to receiving an interaction trigger instruction from the server when the viewer's interactive behavior information meets preset conditions, the system generates a live-stream interactive effect that matches the sound information and sends the live-stream interactive effect information to the server. In other words, the broadcaster can receive ambient sound field information and wait for the server to send the interaction trigger instruction; upon receiving the instruction, the broadcaster then generates the live-stream interactive effect based on the ambient sound field information.

[0116] In one implementation, the broadcaster can also generate interactive effects in the live room immediately after receiving the ambient sound field information, and cache the interactive effects in the live room. After receiving the interactive trigger instruction information sent by the server, the broadcaster can send the information of the cached interactive effects in the live room to the server.

[0117] For example, when a viewer's donation in the live stream reaches a preset donation threshold, the server can send a "High-Energy Mode Activation Command" to both the streamer and the viewer, triggering an interactive interaction. Upon receiving the "High-Energy Mode Activation Command," the viewer is redirected to the interactive mode loading screen, where they can see several ambient UI elements and a countdown to the activation of interactive effects. After receiving the interactive interaction trigger command, the streamer can generate live stream interactive effects that match the audio information based on the ambient sound field. When the countdown ends on the viewer's end, the viewer can send real-time change notifications to the streamer. Upon receiving the real-time change notifications, the streamer sends the live stream interactive effect information to the server, which then sends it to the viewer, allowing the viewer to watch the interactive effects that match the audio information in their live stream.

[0118] In one implementation, after acquiring the ambient sound field information, the broadcaster can immediately generate live room interactive effects that match the sound information based on the ambient sound field information, and cache the live room interactive effect information; when the viewer's interactive behavior information meets preset conditions, the server sends interactive trigger instruction information to the viewer; after receiving the interactive trigger instruction information, the viewer jumps to the interactive mode loading interface, and after the interactive mode is loaded, sends real-time change instruction information to the broadcaster; after receiving the real-time change instruction information, the broadcaster sends the cached live room interactive effect information to the viewer through the server; after receiving the live room interactive effect information sent by the broadcaster, the viewer renders the live room interactive effect information according to the live room interactive effect information, so that the viewer can watch the interactive effect in the live room on the viewer's end.

[0119] For example, the broadcaster can generate information on virtual camera movement effects or virtual element live room interaction effects based on ambient sound field information and preset lighting effects, such as light flashing frequency, and cache this information locally. When the viewer's donation amount reaches a preset donation threshold, the server sends a "high-energy mode activation command" (interaction trigger instruction) to the broadcaster. Upon receiving this instruction, the broadcaster sends the locally cached live room interaction effect information to the viewer via the server. After receiving the live room interaction effect information, the viewer can use the UI / UX to render the content based on this information to display the interaction effects. Viewers can then observe the broadcaster's constantly shifting movements and the virtual elements moving rhythmically with the music, thus achieving a linkage between viewer interaction and the live room scene, enhancing the user's immersion in the live stream.

[0120] The following describes the interaction method in live streaming in this exemplary embodiment from the perspective of the server. Figure 8 This illustrates an exemplary flow of interaction methods applied in server-side live streaming, with reference to... Figure 8 As shown, the interaction method in live streaming may include steps S810 to S820:

[0121] Step S810: Obtain audience interaction behavior information in the live broadcast room;

[0122] Step S820: When the audience interaction behavior information meets the preset conditions, send the live room interaction effect information to the audience terminal in the live room; the live room interaction effect is generated based on the environmental sound field information constructed by the voice information of the anchor terminal, and matches the voice information of the anchor terminal; the live room interaction effect information is used to present the live room interaction effect on the audience terminal.

[0123] In step S810, the server can receive the audience interaction behavior information sent by the audience terminal, and make further judgments in step S820. If the audience interaction behavior information meets the preset conditions, the server sends the live room interaction effect information generated by the anchor terminal that matches the sound information to the audience terminal in the live room, so that the audience terminal can present the live room interaction effect on the audience terminal according to the live room interaction effect information.

[0124] The server can determine whether the audience's interactive behavior information meets the preset conditions and respond in a timely manner based on the judgment result to send information about the interactive effect of the live broadcast to the audience, which helps to improve the interaction efficiency in the live broadcast.

[0125] In one implementation, such as Figure 9 As shown, the interaction method in the above live broadcast may also include the following steps S910 to S940:

[0126] Step S910: Receive audio information sent by the broadcaster.

[0127] Step S920: Construct the ambient sound field information of the live broadcast room based on the sound information;

[0128] Step S930: Return the ambient sound field information to the broadcaster, so that the broadcaster can generate interactive effects in the live broadcast room based on the ambient sound field information.

[0129] Step S940: Receive information on the interactive effects of the live broadcast room sent by the broadcaster.

[0130] In step S910, the server can receive the audio information sent by the broadcaster, and in step S920, construct the ambient sound field information of the live broadcast room based on the audio information; in step S930, the constructed ambient sound field information is sent to the broadcaster, so that the broadcaster can generate interactive effects in the live broadcast room based on the ambient sound field information; and in step S940, the server can receive the interactive effect information of the live broadcast room sent by the broadcaster, and then send the interactive effect information of the live broadcast room to the audience, so that the audience can see the interactive effect of the live broadcast room.

[0131] In one implementation, after receiving audience interaction behavior information sent by the audience, the server can determine whether the audience interaction behavior information meets preset conditions. If the audience interaction behavior information meets the preset conditions, the server sends an indication message to the audience indicating that the audience interaction behavior meets the preset conditions. After receiving the indication message indicating that the audience interaction behavior meets the preset conditions, the audience can jump to the interaction mode display interface and send the interaction mode selected by the audience, the audience's basic information, and the interaction mode activation duration obtained based on the audience interaction behavior information to the server. After receiving the above information, the server can send an interaction trigger indication message to the broadcaster.

[0132] Upon receiving an interaction trigger instruction, the broadcaster sends the acquired sound information to the server. The server, upon receiving the sound information, constructs the ambient sound field information of the live stream based on whether the sound information is 3D sound data. If the sound information is 2D sound data, an approximate value for the third-dimensional sound data can be determined using linear interpolation of the 2D sound data, thus obtaining the corresponding 3D sound data. Then, based on the 3D sound data, the sound intensity data of multiple coordinate points within the real space of the live stream is determined to construct the ambient sound field information. After constructing the ambient sound field information, the server sends this information, the interaction mode selected by the audience, the audience's ID information, and the duration of the interaction mode based on the audience's interaction behavior to the broadcaster. The broadcaster then generates the live stream interaction effect based on this information and sends the information of this interaction effect to the server. Upon receiving the information of the live stream interaction effect, the server sends this information to the audience, allowing the audience to view the live stream interaction effect.

[0133] In this exemplary embodiment, the interactive effects of the live broadcast room are presented to users through the interaction between the broadcaster, the server, and the viewer, which further improves the efficiency of generating interactive effects and enhances the user experience.

[0134] In one implementation, the above-mentioned process of returning ambient sound field information to the broadcaster, enabling the broadcaster to generate interactive effects in the live broadcast room based on the ambient sound field information, may include the following steps:

[0135] When audience interaction information meets preset conditions, the server sends ambient sound field information and interaction trigger indication information to the broadcaster. The broadcaster then responds to the interaction trigger indication information and generates interactive effects based on the ambient sound field information. By sending interaction trigger indication information to the broadcaster and generating interactive effects on the broadcaster's end, the server's workload is reduced, which helps improve the overall interaction efficiency during the live stream.

[0136] In one implementation, when the audience's interactive behavior information meets preset conditions, environmental sound field information and interactive trigger indication information are sent to the broadcaster's terminal. The broadcaster's terminal responds to the interactive trigger indication information and sends the information of the pre-generated cached live room interactive effects to the server. The broadcaster's terminal can generate live room interactive effects in advance based on the environmental sound field information when memory usage is low, and send the live room interactive effects to the server when it receives the interactive trigger indication information sent by the server. This improves the memory usage efficiency of the broadcaster's terminal and further enhances the generation efficiency of interactive effects.

[0137] In one implementation, such as Figure 10 As shown, the interaction method in the above live broadcast may also include the following steps S1010 to S1030:

[0138] Step S1010: Receive audio information sent by the broadcaster.

[0139] Step S1020: Construct the ambient sound field information of the live broadcast room based on the sound information;

[0140] Step S1030: Generate interactive effects for the live streaming room based on the ambient sound field information.

[0141] Different from Figure 9 The method shown generates interactive effects in the live stream on the broadcaster's end. Figure 10 The interactive method shown in the live stream receives the audio information sent by the broadcaster on the server side, constructs the ambient sound field information of the live stream room based on the audio information, and further generates interactive effects in the live stream room on the server side based on the ambient sound field information. This can reduce the number of interactions between the broadcaster and the server side and effectively improve the interaction efficiency in the live stream.

[0142] In addition, when setting the interactive mode, the server can also send interactive mode instruction information to the broadcaster, so that the broadcaster can set the interactive mode. In one implementation, the server can determine the interactive mode instruction information based on the audience's interactive behavior information. After receiving the interactive mode instruction information, the server can send the interactive mode instruction information to the broadcaster, so that the broadcaster can set the current interactive mode of the live broadcast room according to the interactive mode instruction information. The type of interactive effect in the live broadcast room matches the current interactive mode.

[0143] Interactive modes can also be set via the server. In one implementation, the server can determine the interactive mode instruction information based on the audience's interactive behavior information, and set the current interactive mode of the live broadcast room according to the interactive mode instruction information; then send the current interactive mode of the live broadcast room to the broadcaster, so that the broadcaster generates the live broadcast room interactive effect corresponding to the current interactive mode, thereby reducing the interactive logic of the broadcaster and further improving the overall operating efficiency of the interactive methods in the live broadcast.

[0144] The following description explains the interactive method in live streaming in this exemplary embodiment from the viewer's perspective. Figure 11 An exemplary flow of an interaction method applied to a live stream on the viewer's end is shown, with reference to... Figure 11 As shown, the interaction method in live streaming may include steps S1110 to S1130:

[0145] Step S1110: Send the audience interaction behavior information in the live broadcast room to the server.

[0146] Step S1120: Receive information on the live room interaction effect sent by the server when the audience interaction behavior information meets preset conditions; the live room interaction effect is generated based on the environmental sound field information constructed from sound information, and matches the sound information of the host in the live room.

[0147] Step S1130: Present the live room interaction effect based on the information of the live room interaction effect.

[0148] In step S1110, the audience interaction behavior information in the live broadcast room can be sent to the server first, so that the server can determine whether to send the live broadcast room interaction effect to the audience based on the audience interaction behavior information. If the server determines that it needs to send the live broadcast room interaction effect to the audience, the audience can receive the live broadcast room interaction effect information sent by the server in step S1120 when the audience interaction behavior information meets the preset conditions, and the live broadcast room interaction effect information matches the audio information of the broadcaster. In step S1130, the audience presents the live broadcast room interaction effect based on the live broadcast room interaction effect information. This disclosure does not make special limitations on the specific steps of the audience presenting the live broadcast room interaction effect. For example, the UE can render the live broadcast room of the audience based on the live broadcast room interaction effect information to present the live broadcast room interaction effect to the audience.

[0149] Based on the above method, through the interaction between the broadcaster, the server, and the viewer during the live broadcast, various interactive effects are presented to the viewer when the viewer's interactive behavior meets the preset conditions. This enhances the connection between the viewer and the live broadcast room, makes the live broadcast more interesting and vivid, improves the viewer's immersion in the live broadcast room, and further improves the viewer's viewing experience.

[0150] In one implementation, an interactive flow of the live streaming interaction method disclosed herein is as follows: Figure 12 As shown, the broadcaster can generate interactive effects in the live room according to the following steps S1201 to S1207, the server can perform data scheduling in the interactive process during the live broadcast according to the following steps S1208 to S1215, and the viewer can display the interactive effects in the live room to the viewer according to the following steps S1216 to S1223.

[0151] Step S1201: Select a virtual live streaming scene and start the virtual live streaming;

[0152] Step S1202: Determine whether the currently used virtual live streaming scene supports enabling "interactive mode" based on the scene ID;

[0153] Step S1203: If the current live streaming scenario supports enabling "interactive mode", then send "instruction message to enable interactive mode" to the server.

[0154] Step S1204: In response to the "high-energy mode pre-activation instruction", the sound information of the live broadcast room is obtained through the microphone device used by the broadcaster and the sound information is sent to the server.

[0155] Step S1205: In response to the interactive mode information selected by the audience, generate interactive effects for the live broadcast room based on the ambient sound field information;

[0156] Step S1206: Send the information on the interactive effects of the live stream to the server.

[0157] In step S1207, in response to the "stop acquiring sound information instruction", the acquired sound information is no longer sent to the server;

[0158] Step S1208: Monitor the broadcaster's terminal that is conducting virtual live streaming in real time, and return the virtual live streaming scene ID used by the virtual live streaming to the broadcaster's terminal;

[0159] Step S1209: Upon receiving the "Support to enable interactive mode instruction message", send the "Show interactive mode entry instruction message" to the viewer's end;

[0160] Step S1210: Based on the audience interaction behavior information reaching the preset conditions, send an "interaction mode selection interface display instruction" to the audience terminal;

[0161] Step S1211: In response to the interactive mode information selected by the viewer, the viewer's basic information and the duration of the interactive mode sent by the viewer's terminal, send a "high-energy mode pre-activation instruction" to the broadcaster's terminal.

[0162] Step S1212: Receive the audio information sent by the broadcaster and construct the ambient sound field information based on the audio information;

[0163] Step S1213: Send the ambient sound field information and the interactive mode information selected by the audience to the broadcaster.

[0164] Step S1214: Receive information on the interactive effects in the live stream and send the information on the interactive effects in the live stream to the viewer's end;

[0165] In step S1215, in response to the "exit high-energy mode command", a "stop acquiring audio information command" is sent to the broadcaster.

[0166] Step S1216: Obtain business master data from multiple business systems;

[0167] Step S1217: Present the "Interactive Mode Display Entrance" to the audience;

[0168] Step S1218: In response to the user clicking the "Interactive Mode Display Entry", display the gameplay instructions and the progress of the interactive mode activation to the user.

[0169] Step S1219: Obtain audience interaction behavior information and send it to the server.

[0170] Step S1220: In response to the "interactive mode selection interface display instruction", the interactive mode selection interface is displayed to the audience;

[0171] Step S1221: Send the interaction mode information selected by the audience, the audience's basic information, and the interaction mode activation duration determined based on the audience's interaction behavior information to the server.

[0172] Step S1222: Render the received live room interaction effect information using the UE instance, and present the live room interaction effect to the audience during the interactive mode activation period.

[0173] Step S1223: When the viewer clicks "Exit Interactive Mode" or the "Interactive Mode Start Time" ends, send an "Exit High Energy Mode Command" to the server.

[0174] Exemplary embodiments of this disclosure also provide an interactive device applied in live streaming on a broadcaster's end. For example... Figure 13 As shown, the interactive device 1300 in this live broadcast may include:

[0175] The audio information acquisition module 1310 is configured to acquire audio information from the live broadcast room.

[0176] The data sending module 1320 is configured to send sound information to the server, so that the server, when the audience interaction behavior information in the live room meets preset conditions, sends live room interaction effect information to the audience in the live room; the live room interaction effect is generated based on the environmental sound field information constructed from the sound information and matches the sound information; or the environmental sound field information of the live room is constructed based on the sound information, and a live room interaction effect matching the sound information is generated based on the environmental sound field information, and the live room interaction effect information is sent to the server, so that the server, when the audience interaction behavior information in the live room meets preset conditions, sends the live room interaction effect information to the audience in the live room; the live room interaction effect information is used to present the live room interaction effect on the audience's end.

[0177] In one implementation, the aforementioned sending of sound information to the server, so that the server, upon meeting preset conditions regarding audience interaction behavior information within the live stream, sends information about the live stream interaction effect to the audience in the live stream, may include:

[0178] The audio information is sent to the server, which then constructs the ambient sound field information of the live broadcast room based on the audio information.

[0179] Receive ambient sound field information returned by the server and generate interactive effects for the live broadcast room based on the ambient sound field information;

[0180] The information on the interactive effects in the live stream is sent to the server, so that when the audience's interactive behavior in the live stream meets the preset conditions, the server sends the information on the interactive effects in the live stream to the audience in the live stream.

[0181] In one implementation, the above-mentioned generation of live-stream interactive effects matching the sound information based on the ambient sound field information may include:

[0182] When the current interactive mode in the live broadcast room is virtual camera movement interactive mode, the camera movement trajectory is determined based on the ambient sound field information. The camera movement trajectory is used to control the virtual camera to move and shoot the virtual scene of the live broadcast room. The live broadcast room interactive effect is generated based on the captured images; the virtual scene matches the real space of the live broadcast room.

[0183] In one implementation, determining the camera motion trajectory based on ambient sound field information may include:

[0184] Determine the coordinates of the strongest sound point from the ambient sound field information;

[0185] The camera coordinates and camera plane at the next moment are determined based on the angle between the current camera coordinates and the coordinates of the strongest sound point, as well as the projection of the strongest sound point coordinates onto the camera plane at the current moment.

[0186] In one embodiment, the above-mentioned determination of the camera motion trajectory based on ambient sound field information may further include:

[0187] After determining the camera coordinates and camera plane at the next moment, based on the overall sound field intensity in the ambient sound field information, a perturbation in a preset direction is added to the camera coordinates at the next moment to update the camera coordinates and camera plane at the next moment.

[0188] In one implementation, the above-mentioned generation of live-stream interactive effects that match the sound information based on ambient sound field information may include:

[0189] When the current interactive mode of the live broadcast room is virtual element interactive mode, the sound intensity statistics of each unit space in the virtual scene of the live broadcast room are determined according to the ambient sound field information. Virtual elements with sizes matching the sound intensity statistics are rendered in each unit space to generate interactive effects in the live broadcast room; the virtual scene matches the real space of the live broadcast room.

[0190] In one implementation, the above-mentioned determination of sound intensity statistics within each unit space of the virtual scene of the live broadcast room based on ambient sound field information may include:

[0191] The virtual scene of the live broadcast room is divided into multiple unit spaces along the first axis. The average sound intensity along the second and third axes in each unit space is calculated to obtain the sound intensity statistics for each unit space.

[0192] In one implementation, before generating interactive effects for the live streaming room based on ambient sound field information, the above method may further include:

[0193] Receive interactive mode indication information sent by the server. The interactive mode indication information is determined by the server based on the audience's interactive behavior information.

[0194] The current interaction mode of the live stream is set according to the interaction mode indication information. In one embodiment, if the above-mentioned sound information is two-dimensional sound data, then the above-mentioned construction of the environmental sound field information of the live stream based on the sound information may include:

[0195] By performing third-dimensional interpolation on the two-dimensional sound data, the two-dimensional sound data is converted into three-dimensional sound data;

[0196] Based on the 3D sound data, the sound intensity data of multiple coordinate points in the real space of the live broadcast room are determined in order to construct the environmental sound field information of the live broadcast room.

[0197] In one implementation, the above-mentioned construction of ambient sound field information for the live streaming room based on sound information, and the generation of interactive effects for the live streaming room that match the sound information based on the ambient sound field information, includes:

[0198] In response to receiving an interaction trigger instruction from the server when the audience's interactive behavior information meets preset conditions, the system generates a live room interactive effect that matches the audio information and sends the live room interactive effect information to the server.

[0199] Exemplary embodiments of this disclosure also provide an interactive device applied in live streaming on a server. For example... Figure 14 As shown, the interactive device 1400 in this live broadcast may include:

[0200] The audience interaction behavior information acquisition module 1410 is configured to acquire audience interaction behavior information in the live broadcast room.

[0201] The interactive effect sending module 1420 is configured to send live room interactive effect information to the audience terminal in the live room when the audience interactive behavior information meets the preset conditions; the live room interactive effect is generated based on the environmental sound field information constructed by the broadcaster's sound information and matches the broadcaster's sound information; the live room interactive effect information is used to present the live room interactive effect on the audience terminal.

[0202] In one embodiment, the interactive device in the live broadcast may further include:

[0203] Receive audio information sent by the broadcaster;

[0204] Construct the ambient sound field information of the live broadcast room based on the sound information;

[0205] The ambient sound field information is returned to the broadcaster, enabling the broadcaster to generate interactive effects in the live broadcast room based on the ambient sound field information.

[0206] Receive information about the interactive effects in the live stream sent by the broadcaster.

[0207] In one implementation, the above-mentioned method of returning ambient sound field information to the broadcaster, enabling the broadcaster to generate interactive effects in the live broadcast room based on the ambient sound field information, may include:

[0208] When the audience's interactive behavior information meets the preset conditions, the system sends environmental sound field information and interactive trigger instruction information to the broadcaster, so that the broadcaster responds to the interactive trigger instruction information and generates interactive effects in the live broadcast room based on the environmental sound field information.

[0209] In one embodiment, the interactive device in the live broadcast may further include:

[0210] Receive audio information sent by the broadcaster;

[0211] Construct the ambient sound field information of the live broadcast room based on the sound information;

[0212] Interactive effects in the live stream are generated based on ambient sound field information.

[0213] In one embodiment, the interactive device in the live broadcast may further include:

[0214] Determine the interaction mode instruction information based on audience interaction behavior information;

[0215] The interactive mode instruction information is sent to the broadcaster's end, enabling the broadcaster to set the current interactive mode of the live broadcast room according to the interactive mode instruction information; the type of interactive effect in the live broadcast room matches the current interactive mode.

[0216] Exemplary embodiments of this disclosure also provide an interactive device for use in live streaming on a viewer's end. For example... Figure 15 As shown, the interactive device 1500 in this live broadcast may include:

[0217] The audience interaction behavior information sending module 1510 is configured to receive audio information sent by the broadcaster.

[0218] The interactive effect receiving module 1520 is configured to receive the live room interactive effect information sent by the server when the audience's interactive behavior information meets the preset conditions; the live room interactive effect is generated based on the environmental sound field information constructed from the broadcaster's sound information and matches the broadcaster's sound information in the live room.

[0219] The interactive effect presentation module 1530 is configured to present the interactive effects of the live room based on the information of the interactive effects in the live room.

[0220] The specific details of each part of the above-mentioned device have been described in detail in the method section of the implementation, and therefore will not be repeated here.

[0221] Exemplary embodiments of this disclosure also provide a computer-readable storage medium that can be implemented as a program product including program code, which, when run on an electronic device, causes the electronic device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. In an alternative embodiment, the program product can be implemented as a portable compact disc read-only memory (CD-ROM) including program code and can run on an electronic device, such as a personal computer. However, the program product of this disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0222] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0223] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0224] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0225] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0226] Exemplary embodiments of this disclosure also provide an electronic device. The electronic device may include a processor and a memory. The memory stores executable instructions for the processor, such as program code. The processor executes the executable instructions to perform the methods of this exemplary embodiment.

[0227] The following is for reference. Figure 16The electronic device is illustrated by way of a general-purpose computing device. It should be understood that... Figure 16 The electronic device 1600 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0228] like Figure 16 As shown, the electronic device 1600 may include: a processor 1610, a memory 1620, a bus 1630, an I / O (input / output) interface 1640, and a network adapter 1650.

[0229] Processor 1610 may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a display processing unit (DPU), a graphics processing unit (GPU), an image signal processor (ISP), a controller, an encoder, a decoder, a digital signal processor (DSP), a baseband processor, an artificial intelligence processor, etc. In one embodiment, the interactive method for live streaming can be implemented in the CPU. This can involve first acquiring the audio information of the live stream; then sending the audio information to the server, so that the server, when the audience interaction behavior information in the live stream meets preset conditions, sends information about the live stream interaction effect to the audience in the live stream; or generating a live stream interaction effect that matches the audio information, and then sending the information about the live stream interaction effect to the server, so that the server, when the audience interaction behavior information in the live stream meets preset conditions, sends the information about the live stream interaction effect to the audience in the live stream.

[0230] Memory 1620 may include volatile memory, such as RAM 1621 and cache unit 1622, and may also include non-volatile memory, such as ROM 1623. Memory 1620 may also include one or more program modules 1624, such program modules 1624 including, but not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, program module 1624 may include the modules in the above-described devices 1300, 1400, or 1500.

[0231] Bus 1630 is used to connect different components of electronic device 1600, and may include data bus, address bus and control bus.

[0232] Electronic device 1600 can communicate with one or more external devices 1700 (such as keyboard, mouse, external controller, etc.) through I / O interface 1640.

[0233] Electronic device 1600 can communicate with one or more networks via network adapter 1650. For example, network adapter 1650 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. Network adapter 1650 can communicate with other modules of electronic device 1600 via bus 1630.

[0234] although Figure 16 As not shown in the diagram, other hardware and / or software modules may also be configured in the electronic device 1600, including but not limited to: a display, microcode, device driver, redundant processor, external disk drive array, RAID system, tape drive, and data backup storage system.

[0235] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0236] Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be embodied in entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuit,” “module,” or “system.” Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0237] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is defined only by the appended claims.

Claims

1. An interactive method for live streaming, characterized in that, Applied to the broadcaster's end, the method includes: Obtain the audio information from the live stream; The sound information is sent to the server, and the server, when the audience interaction behavior information in the live broadcast room meets preset conditions, sends live broadcast room interaction effect information to the audience in the live broadcast room; the live broadcast room interaction effect is generated based on the environmental sound field information constructed from the sound information and matches the sound information; the environmental sound field information is constructed based on the sound intensity data of multiple coordinate points in the real space of the live broadcast room; or The ambient sound field information of the live broadcast room is constructed based on the sound information, and interactive effects matching the sound information are generated based on the ambient sound field information. The interactive effects are sent to the server, so that the server sends the interactive effects to the viewers in the live broadcast room when the viewers' interactive behavior information in the live broadcast room meets preset conditions. The step of generating interactive effects matching the sound information based on the ambient sound field information includes: when the current interactive mode of the live broadcast room is a virtual element interactive mode, determining the sound intensity statistics in each unit space of the virtual scene of the live broadcast room based on the ambient sound field information, and rendering virtual elements with sizes matching the sound intensity statistics in each unit space to generate interactive effects; the virtual scene matches the real space of the live broadcast room. The information about the interactive effects in the live stream is used to present the interactive effects in the live stream on the viewer's device.

2. The method according to claim 1, characterized in that, The step of sending the audio information to the server, so that the server, when the audience interaction behavior information in the live broadcast room meets preset conditions, sends information about the live broadcast room interaction effect to the audience in the live broadcast room, includes: The sound information is sent to the server, so that the server can construct the environmental sound field information of the live broadcast room based on the sound information; Receive the ambient sound field information returned by the server, and generate interactive effects for the live broadcast room based on the ambient sound field information; The information on the interactive effects in the live stream is sent to the server, so that when the interactive behavior information of the audience in the live stream meets the preset conditions, the server sends the information on the interactive effects in the live stream to the audience in the live stream.

3. The method according to claim 1, characterized in that, The step of generating live-stream interactive effects that match the sound information based on the ambient sound field information includes: When the current interactive mode of the live broadcast room is virtual camera movement interactive mode, the camera movement trajectory is determined according to the ambient sound field information, and the camera movement trajectory is used to control the virtual camera to move and shoot the virtual scene of the live broadcast room. The live broadcast room interactive effect is generated based on the shot. The virtual scene matches the real space of the live broadcast room.

4. The method according to claim 3, characterized in that, Determining the camera motion trajectory based on the ambient sound field information includes: Determine the coordinates of the strongest sound point from the environmental sound field information; The camera coordinates and camera plane at the next moment are determined based on the angle between the current camera coordinates and the coordinates of the strongest sound point, and the projection of the coordinates of the strongest sound point onto the camera plane at the current moment.

5. The method according to claim 4, characterized in that, The step of determining the camera motion trajectory based on the ambient sound field information further includes: After determining the camera coordinates and camera plane at the next moment, based on the overall sound field intensity in the ambient sound field information, a perturbation in a preset direction is added to the camera coordinates at the next moment to update the camera coordinates and camera plane at the next moment.

6. The method according to claim 1, characterized in that, The step of determining the sound intensity statistics within each unit space of the virtual scene in the live streaming room based on the environmental sound field information includes: The virtual scene of the live broadcast room is divided into multiple unit spaces along the first axis, and the average sound intensity along the second and third axes in each unit space is calculated to obtain the sound intensity statistics in each unit space.

7. The method according to any one of claims 3 to 6, characterized in that, Before generating live-stream interactive effects based on the ambient sound field information, the method further includes: Receive interactive mode indication information sent by the server, wherein the interactive mode indication information is determined by the server based on the audience interactive behavior information; The current interaction mode of the live broadcast room is set according to the interaction mode instruction information.

8. The method according to claim 1, characterized in that, If the sound information is two-dimensional sound data, then constructing the environmental sound field information of the live broadcast room based on the sound information includes: By performing third-dimensional interpolation on the two-dimensional sound data, the two-dimensional sound data is converted into three-dimensional sound data; Based on the three-dimensional sound data, the sound intensity data of multiple coordinate points in the real space of the live broadcast room are determined to construct the environmental sound field information of the live broadcast room.

9. The method according to claim 1, characterized in that, The step of constructing the ambient sound field information of the live broadcast room based on the sound information, and generating interactive effects in the live broadcast room that match the sound information based on the ambient sound field information, includes: In response to receiving an interaction trigger instruction from the server when the audience's interactive behavior information meets preset conditions, the system constructs the ambient sound field information of the live broadcast room based on the sound information, and generates live broadcast room interactive effects that match the sound information based on the ambient sound field information.

10. An interactive method in live streaming, characterized in that, Applied to the server side, the method includes: To obtain information on audience interaction behavior within the live stream; When the audience interaction behavior information meets preset conditions, information about the live room interaction effect is sent to the audience terminal in the live room; the live room interaction effect is generated based on the environmental sound field information constructed from the voice information of the anchor terminal, and matches the voice information of the anchor terminal; the environmental sound field information is constructed based on the sound intensity data of multiple coordinate points in the real space of the live room; the information about the live room interaction effect is used to present the live room interaction effect on the audience terminal. Wherein, when the current interactive mode of the live broadcast room is the virtual element interactive mode, the interactive effect of the live broadcast room is generated by determining the sound intensity statistics in each unit space of the virtual scene of the live broadcast room based on the environmental sound field information, and rendering virtual elements with sizes matching the sound intensity statistics in each unit space; the virtual scene matches the real space of the live broadcast room.

11. The method according to claim 10, characterized in that, The method further includes: Receive the audio information sent by the broadcaster; The ambient sound field information of the live broadcast room is constructed based on the sound information; The ambient sound field information is returned to the broadcaster's terminal, enabling the broadcaster's terminal to generate interactive effects in the live broadcast room based on the ambient sound field information; Receive information about the interactive effects in the live stream sent by the broadcaster.

12. The method according to claim 11, characterized in that, The step of returning the ambient sound field information to the broadcaster's terminal, so that the broadcaster's terminal can generate interactive effects in the live broadcast room based on the ambient sound field information, includes: When the audience interaction behavior information meets the preset conditions, the ambient sound field information and interaction trigger indication information are sent to the broadcaster terminal, so that the broadcaster terminal responds to the interaction trigger indication information and generates live room interaction effects based on the ambient sound field information.

13. The method according to claim 10, characterized in that, The method further includes: Receive the audio information sent by the broadcaster; The ambient sound field information of the live broadcast room is constructed based on the sound information; The interactive effects in the live streaming room are generated based on the ambient sound field information.

14. The method according to claim 10, characterized in that, The method further includes: The interaction mode instruction information is determined based on the audience interaction behavior information; The interactive mode instruction information is sent to the broadcaster's terminal, so that the broadcaster's terminal sets the current interactive mode of the live broadcast room according to the interactive mode instruction information; the type of interactive effect in the live broadcast room matches the current interactive mode.

15. An interactive method in live streaming, characterized in that, Applied to the audience, the method includes: The viewer's interactive behavior information in the live broadcast room is sent to the server. The system receives information from the server regarding the interactive effect in the live stream when the viewer's interactive behavior information meets preset conditions. This interactive effect is generated based on environmental sound field information constructed from sound information, and matches the sound information of the broadcaster in the live stream. The environmental sound field information is constructed based on sound intensity data from multiple coordinate points within the real space of the live stream. Specifically, when the current interactive mode of the live stream is a virtual element interactive mode, the interactive effect is generated by determining the sound intensity statistics within each unit space of the virtual scene in the live stream based on the environmental sound field information, and rendering virtual elements with sizes matching the sound intensity statistics within each unit space. The virtual scene matches the real space of the live stream. The live stream interaction effect is presented based on the information about the live stream interaction effect.

16. An interactive device for live streaming, characterized in that, Applied to the broadcaster's end, including: The audio information acquisition module is configured to acquire audio information from the live broadcast room; The data sending module is configured to send the sound information to the server, so that the server, when the audience interaction behavior information in the live broadcast room meets preset conditions, sends information about the live broadcast room interaction effect to the audience in the live broadcast room; the live broadcast room interaction effect is generated based on the environmental sound field information constructed from the sound information and matches the sound information; the environmental sound field information is constructed based on the sound intensity data of multiple coordinate points in the real space of the live broadcast room; or The ambient sound field information of the live broadcast room is constructed based on the sound information, and interactive effects matching the sound information are generated based on the ambient sound field information. The interactive effects are sent to the server, so that the server sends the interactive effects to the viewers in the live broadcast room when the viewers' interactive behavior information in the live broadcast room meets preset conditions. The step of generating interactive effects matching the sound information based on the ambient sound field information includes: when the current interactive mode of the live broadcast room is a virtual element interactive mode, determining the sound intensity statistics in each unit space of the virtual scene of the live broadcast room based on the ambient sound field information, and rendering virtual elements with sizes matching the sound intensity statistics in each unit space to generate interactive effects; the virtual scene matches the real space of the live broadcast room. The information about the interactive effects in the live stream is used to present the interactive effects in the live stream on the viewer's device.

17. An interactive device for live streaming, characterized in that, Applied to the server side, including: The audience interaction behavior information acquisition module is configured to acquire audience interaction behavior information in the live broadcast room; An interactive effect sending module is configured to send live room interactive effect information to the audience terminal in the live room when the audience interactive behavior information meets preset conditions. The live room interactive effect is generated based on the environmental sound field information constructed from the broadcaster's sound information and matches the broadcaster's sound information. The environmental sound field information is constructed based on the sound intensity data of multiple coordinate points in the real space of the live room. The live room interactive effect information is used to present the live room interactive effect on the audience terminal. Wherein, when the current interactive mode of the live room is the virtual element interactive mode, the live room interactive effect is generated by determining the sound intensity statistics in each unit space of the virtual scene of the live room based on the environmental sound field information, and rendering virtual elements with sizes matching the sound intensity statistics in each unit space. The virtual scene matches the real space of the live room.

18. An interactive device for live streaming, characterized in that, Applied to the audience, including: The audience interaction behavior information sending module is configured to send the audience interaction behavior information of the audience terminal in the live broadcast room to the server. The interactive effect receiving module is configured to receive live room interactive effect information sent by the server when the audience's interactive behavior information meets preset conditions; the live room interactive effect is generated based on environmental sound field information constructed from sound information, and matches the sound information of the anchor in the live room; the environmental sound field information is constructed based on the sound intensity data of multiple coordinate points in the real space of the live room; wherein, when the current interactive mode of the live room is virtual element interactive mode, the live room interactive effect is generated by determining the sound intensity statistics in each unit space of the virtual scene of the live room based on the environmental sound field information, and rendering virtual elements with sizes matching the sound intensity statistics in each unit space; the virtual scene matches the real space of the live room. The interactive effect presentation module is configured to present the interactive effect of the live room based on the information of the interactive effect of the live room.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 15.

20. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 15 by executing the executable instructions.

Images