Live broadcast system, sound card equipment, computer program product and storage medium

Abstract

The embodiments of the present application provide a live broadcast system, a sound card device, an audio acquisition device, a monitoring device, a computer program product, and a storage medium. In order to realize the wireless connection between the sound card device and other devices in the live broadcast system, a first wireless transmission channel and a second wireless transmission channel can be set in the sound card device. The two wireless transmission channels can operate based on different wireless transmission protocols. The first wireless transmission channel can be used to transmit voice data with high real-time requirements between the audio acquisition device and the monitoring device, and the second wireless transmission channel can be used to transmit data with low real-time requirements such as accompaniment audio between the accompaniment device. In this way, the audio acquisition device, monitoring device, accompaniment device, etc. in the live broadcast system can be connected to the sound card device through a wireless connection, simplifying the construction process of the live broadcast system.

Description

Technical Field

[0001] The present application relates to the field of live broadcast technology, and more specifically, to a live broadcast system, a sound card device, an audio acquisition device, a monitoring device, a computer program product, and a storage medium. Background Art

[0002] Traditional personal live streaming systems typically include an audio capture device (e.g., a live broadcast microphone), an accompaniment device (e.g., a mobile phone that plays the accompaniment audio), a sound card, a monitoring device (e.g., headphones), and a live broadcast device (e.g., a mobile phone with live broadcast software installed). The audio capture device captures the host's voice data and transmits it to the sound card. The accompaniment device provides the accompaniment audio and transmits it to the sound card. The sound card mixes the host's voice data and the accompaniment audio, processing the audio effects. The resulting mixed data is then output in two channels: one channel is sent to the live broadcast device as the audio data in the live video stream, and the other channel is sent to the monitoring device so that the live broadcast user can monitor whether the final processed mixed data meets the expected sound effects. Currently, the audio capture device, accompaniment device, monitoring device, and sound card in personal live broadcast systems are all connected via data cables. Before each live broadcast, these devices must be connected using data cables, which is cumbersome and unsightly. The cables also limit the distance between the connected devices, making them inconvenient. Summary of the Invention

[0003] In view of this, the present application provides a live broadcast system, a sound card device, an audio acquisition device, a monitoring device, a computer program product and a storage medium.

[0004] According to a first aspect of the present application, a live broadcast system is provided, comprising an audio capture device, a sound card device, and a monitoring device, wherein the audio capture device and the monitoring device are respectively wirelessly connected to the sound card device, and the sound card device is also wirelessly connected to an accompaniment device; the sound card device comprises a first wireless transmission channel and a second wireless transmission channel; the first wireless transmission channel and the second wireless transmission channel operate based on different wireless transmission protocols, and the transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel;

[0005] The audio acquisition device is used to collect voice data of live broadcast users and send it to the sound card device through the first wireless transmission channel;

[0006] The sound card device is used to receive the accompaniment audio sent by the accompaniment device through the second wireless transmission channel, mix the voice data and the accompaniment audio to obtain mixed data, and send the mixed data to the monitoring device through the first wireless transmission channel; and output the mixed data to the live broadcast device;

[0007] The monitoring device is used to play the mixed audio data so that a live broadcast user can monitor the mixed audio data.

[0008] According to a second aspect of the present application, a sound card device is provided, which is wirelessly connected to an audio acquisition device, a monitoring device, and an accompaniment device, respectively. The sound card device includes a first wireless transmission channel and a second wireless transmission channel, the first wireless transmission channel and the second wireless transmission channel operate based on different wireless transmission protocols, and the transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel; the sound card device is used to receive voice data of a live broadcast user collected by the audio acquisition device through the first wireless transmission channel, receive accompaniment audio sent by the accompaniment device through the second wireless transmission channel, mix the voice data and the accompaniment audio to obtain mixed data, and send the mixed data to the monitoring device through the first wireless transmission channel so that the monitoring device plays the mixed data; and output the mixed data to the live broadcast device.

[0009] According to a third aspect of the present application, an audio acquisition device is provided, which is used to implement the functions of the audio acquisition device in the live broadcast system mentioned in the first aspect above.

[0010] According to a fourth aspect of the present application, a monitoring device is provided, which is used to implement the functions of the monitoring device in the live broadcast system mentioned in the first aspect above.

[0011] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program. When executed, the computer program can implement the functions of a client, wherein the client is wirelessly connected to the sound card device in the live broadcast system mentioned in the first aspect, and is configured to send the sound effect adjustment parameters set by the user to the sound card device via the second wireless transmission channel;

[0012] After mixing the voice data and the accompaniment audio to obtain mixed data, the sound card device is further configured to perform sound effect adjustment processing on the mixed data based on the sound effect adjustment parameter, and then transmit the mixed data after the sound effect adjustment processing to the monitoring device through the first wireless transmission channel, and output the mixed data to the live broadcast device; or

[0013] Before the sound card device mixes the voice data and the accompaniment audio to obtain mixed data, it is also used to perform sound effect adjustment processing on the voice data and / or accompaniment audio based on the sound effect adjustment parameters, and then perform mixing processing to obtain mixed data, and send it to the monitoring device through the first wireless transmission channel, and output it to the live broadcast device.

[0014] According to a sixth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the computer program can implement the functions of a client, wherein the client is wirelessly connected to the sound card device in the live broadcast system mentioned in the first aspect, and is configured to send the sound effect adjustment parameters set by the user to the sound card device via the second wireless transmission channel;

[0015] After mixing the voice data and the accompaniment audio to obtain mixed data, the sound card device is further configured to perform sound effect adjustment processing on the mixed data based on the sound effect adjustment parameter, and then transmit the mixed data after the sound effect adjustment processing to the monitoring device through the first wireless transmission channel, and output the mixed data to the live broadcast device; or

[0016] Before the sound card device mixes the voice data and the accompaniment audio to obtain mixed data, it is also used to perform sound effect adjustment processing on the voice data and / or accompaniment audio based on the sound effect adjustment parameters, and then perform mixing processing to obtain mixed data, and send it to the monitoring device through the first wireless transmission channel, and output it to the live broadcast device.

[0017] By applying the solution provided in this application, in order to realize the wireless connection between the sound card device and other devices in the live broadcast system, two wireless transmission channels can be set in the sound card device, namely the first wireless transmission channel and the second wireless transmission channel. The two wireless transmission channels can operate based on different wireless transmission protocols to meet the transmission delay requirements of different data. Among them, the transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel. The first wireless transmission channel can be used to transmit voice data with high real-time requirements between the audio acquisition device and the monitoring device, and the second wireless transmission channel can be used to transmit data with low real-time requirements such as accompaniment audio between the accompaniment device. In this way, the audio acquisition device, monitoring device, accompaniment device, etc. in the live broadcast system can be connected to the sound card device through a wireless connection, simplifying the construction process of the live broadcast system. Moreover, by setting two wireless transmission channels with different delays to transmit different types of data, the efficiency of wireless transmission of various data in the live broadcast system and the real-time transmission can be guaranteed.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 It is a schematic diagram of a live broadcast system in the prior art.

[0021] Figure 2 、 Figure 3 、 Figure 4 It is a schematic diagram of a live broadcast system according to an embodiment of the present application.

[0022] Figure 5 It is a schematic diagram of a sound card device according to an embodiment of the present application.

[0023] Figure 6 It is a schematic diagram of a sound card device according to another embodiment of the present application.

[0024] Figure 7 It is a schematic diagram of a live broadcast system according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] like Figure 1As shown, it is a schematic diagram of a traditional personal live broadcast system, which includes an audio acquisition device 11 (for example, a live broadcast microphone), a sound card device 12, a monitoring device 13 (for example, a monitoring headset), an accompaniment device 14 (for example, a mobile phone 1), and a live broadcast device 15 (for example, a mobile phone 2). The audio acquisition device 11 collects the host's voice data and transmits it to the sound card device 12. The accompaniment device 14 is used to transmit the accompaniment audio to the sound card device 12. The sound card device 12 is used to mix the host's voice data and the accompaniment audio and perform sound effect processing. The processed mixed data is then output in two channels, one channel is output to the live broadcast device 15 as the voice data in the live video stream, and the other channel is output to the monitoring device 13, so that the live broadcast user can monitor whether there is any abnormality in the voice during the live broadcast and whether the sound effect of the processed mixed data meets expectations. At present, the audio acquisition device 11, accompaniment device 14, monitoring device 13 and sound card device 12 in the personal live broadcast system are all connected through data cables. It takes time to connect these devices before each live broadcast, which is cumbersome. In addition, the wires are not only unsightly, but also have certain restrictions on the distance between the connected devices, which is not very convenient.

[0027] Considering that in the existing live broadcast system, the sound card device 12 serves as the center of audio data processing, different devices need to be connected to the sound card device 12 through a wired connection, which will make the connection of the devices in the entire live broadcast system very cumbersome and not conducive to the rapid construction of the live broadcast system. Based on this, the embodiment of the present application has thought of converting the wired connection into a wireless connection, thereby simplifying the construction of the live broadcast system. Considering that during the live broadcast process, the sound card device 12 needs to establish connections with multiple devices and transmit data with these multiple devices. The amount of data to be transmitted is large, and the data types transmitted between these multiple devices are different. These data also have different requirements for transmission delay. For example, the live broadcast process has higher real-time requirements for the host's voice data, while for the accompaniment music, its real-time requirements are lower. Therefore, in the embodiment of the present application, two wireless transmission channels can be set in the sound card device 12, namely the first wireless transmission channel and the second wireless transmission channel. The two wireless transmission channels can operate based on different wireless transmission protocols to meet the transmission delay requirements of different data, wherein the transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel. The first wireless transmission channel can be used to transmit data with high real-time requirements such as voice data, and the second wireless transmission channel can be used to transmit data with low real-time requirements such as accompaniment audio. In this way, the audio acquisition device 11, monitoring device 13, accompaniment device 14, etc. in the live broadcast system can be connected to the sound card device 12 through a wireless connection, simplifying the construction process of the live broadcast system, and by setting two wireless transmission channels with different delays to transmit different types of data, the efficiency of wireless transmission of various data in the live broadcast system and the real-time transmission can be guaranteed.

[0028] Based on the above invention concept, firstly, the embodiment of the present application provides a live broadcast system, such as Figure 2 As shown, the live broadcast system includes an audio capture device 11, a sound card device 12, and a monitoring device 13. The audio capture device 11 is a device that can be used to capture the host's voice data, such as a microphone. The sound card device 12 is a device that can be used to perform various processing on the audio data, such as mixing, adding effects, dynamic range control, and adjusting sound effect volume. The monitoring device 13 is a device that can be used to play audio data so that the user can hear the audio data effects. For example, it can be a headset or other device with audio playback capabilities.

[0029] The sound card device 12 serves as an audio data processing center and is wirelessly connected to the audio acquisition device 11 and the monitoring device 13. The sound card device 12 is also wirelessly connected to the accompaniment device 14. The accompaniment device 14 can be any device for providing accompaniment audio during the live broadcast, such as a user's mobile phone, tablet, laptop, etc.

[0030] The sound card device 12 includes a first wireless transmission channel and a second wireless transmission channel, wherein the first wireless transmission channel and the second wireless transmission channel can be implemented by two wireless modules, and the two wireless modules can be integrated on the same chip or on two different chips.

[0031] To meet the transmission delay requirements of different types of data during live broadcasts, the first and second wireless transmission channels can operate based on different wireless transmission protocols, and the transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel. For example, the first and second wireless transmission channels can operate based on existing standard wireless transmission protocols, such as Bluetooth, WiFi, Zigbee, LoRa, etc., or based on custom proprietary protocols.

[0032] Considering that during live broadcasting, in order to ensure the synchronization of the host's voice and the picture, the real-time performance of the host's voice data is highly required. In addition, the host also needs to promptly monitor whether there are any abnormalities in the voice processed by the sound card device 12. Therefore, the audio data between the sound card device 12 and the audio acquisition device 11 and the monitoring device 13 can be transmitted via a first wireless transmission channel with a shorter transmission delay, so that the transmission delay of such data is as low as possible. As for the accompaniment audio, since it does not strictly require synchronization with the picture, the real-time performance requirement is not high. Therefore, the sound card device 12 and the accompaniment device 14 can be transmitted via a second wireless transmission channel with a longer transmission delay.

[0033] During the live broadcast process, the audio acquisition device 11 can collect the voice data of the live broadcast user and send it to the sound card device 12 via the first wireless transmission channel. The accompaniment device 14 can send the accompaniment audio to the sound card device 12 via the second wireless transmission channel. After receiving the above two types of data, the sound card device 12 can mix the voice data and the accompaniment audio to obtain mixed data, and output the mixed data in two ways, one of which is sent to the monitoring device 13 via the first wireless transmission channel. The monitoring device 13 can play the mixed data so that the user can monitor whether there is any abnormality in the mixed data or whether the sound effect of the mixed data meets the expected effect. The other mixed data can be output to the live broadcast device 15 as audio in the live video stream. Generally, considering that the live broadcast device 15 is a device produced by a third-party manufacturer, if the mixed data is transmitted to the live broadcast device based on the existing WiFi or Bluetooth protocol, it may not meet the delay requirements of the live broadcast scene. Therefore, a wired connection can be used between the live broadcast device 15 and the sound card device to ensure the real-time performance of the data transmission.

[0034] In the past, the sound card device 12 in the live broadcast system usually includes buttons, and users can adjust the sound effect adjustment parameters by adjusting these buttons to adjust the sound effect of the audio data. However, the setting of the sound effect parameters often requires a professional tuner to operate, which requires the tuner to go to the live broadcast site to adjust the sound effect, which is relatively cumbersome. In order to facilitate the adjustment of the sound effect during the live broadcast, Figure 3 As shown, the live broadcast system may further include a client 16 wirelessly connected to the sound card device 12. The client 16 may be an APP installed on a personal terminal device such as a mobile phone or a computer. The user may set the sound effect adjustment parameters through the APP, and the client 16 may send the sound effect adjustment parameters to the sound card device 12 through the second wireless transmission channel. In some scenarios, the sound card device 12 may first perform a mixing process on the voice data and the accompaniment audio to obtain mixed data, and then perform a sound effect adjustment on the mixed data based on the sound effect adjustment parameters before outputting the mixed data to the live broadcast device 15 and the monitoring device 13. In some scenarios, the sound card device 12 may also first perform a sound effect adjustment process on one or both of the voice data and the accompaniment audio, and then perform a mixing process on the two to obtain mixed data.

[0035] Considering that the sound effect data is usually pre-stored in the sound card device 12, the sound effect adjustment parameters only need to control and adjust the sound effects, and thus can be transmitted using the second wireless transmission channel with higher latency. In this way, the tuner does not need to go to the live broadcast site, but can set the sound effect adjustment parameters remotely.

[0036] Of course, in some scenarios, in order to allow live broadcast users to adjust the sound effects without the help of a tuner, some debugging pages can be set up in the APP to facilitate live broadcast users to adjust the sound effects. For example, multiple adjustment controls can be set in the debugging interface, and users can touch these adjustment controls to set the sound effect adjustment parameters. At the same time, in order to let users know whether the currently set sound effect adjustment parameters meet expectations, you can also set an audition button in the debugging interface. When the user clicks this button, he can audition the sound effects of the audio adjusted using the currently set sound effect adjustment parameters.

[0037] In some embodiments, it is considered that the audio acquisition device 11, the sound card device 12, and the monitoring device 13 in the live broadcast system are usually used in conjunction with each other and are generally produced by the same manufacturer, while the accompaniment device 14 and the personal terminal equipped with the client 16 for setting the sound effect adjustment parameters are usually produced by other manufacturers. In order to enable the sound card device 12 to achieve wireless connection with the accompaniment device 14 and the personal terminal produced by other manufacturers, the second wireless transmission channel can achieve data transmission based on an existing standard wireless transmission protocol. For example, it can work based on some common wireless transmission protocols such as WiFi and Bluetooth, so that the sound card device 12 can achieve wireless connection with devices produced by other manufacturers. Taking into account that the audio acquisition device 11 and the monitoring device 13 need to ensure low latency and high-quality audio quality for recording and monitoring, while also needing to take into account low power consumption, transmission distance and anti-interference performance, therefore, for the first wireless transmission channel, it can achieve wireless transmission through a customized private protocol. For example, the private protocol can be a customized 2.4G private protocol, which can flexibly select the codec protocol according to the audio delay and quality requirements of the audio acquisition device 11 and the monitoring device 13, and follow the basic wireless communication rules (3 broadcast channels and 37 data communication channels) and anti-interference frequency hopping mechanism. Before the connection is established, the audio acquisition device 11, the monitoring device 13 and the sound card device 12 can exchange relevant information through the broadcast channel and establish a connection for subsequent audio data transmission.

[0038] In some embodiments, considering that the Bluetooth protocol has low power consumption, the second wireless transmission channel can operate based on the Bluetooth protocol to reduce the overall power consumption of the sound card device 12.

[0039] In some embodiments, the customized private protocol can be a private 2.4G protocol. Due to the high real-time requirements in scenarios such as live broadcasts, it is necessary to ensure low latency and high-quality audio quality for recording, while also taking into account low power consumption, transmission distance, and anti-interference performance. The A2DP (Advanced Audio Distribution Profile) used in the standard Bluetooth protocol still has a relatively large delay, and the HFP (Hands-free Profile) protocol is limited by codecs and supports poor audio quality. Therefore, users often choose to support private 2.4G protocols for wireless transmission of audio data. Compared to the standard Bluetooth protocol, the private 2.4G protocol can be deeply customized according to the needs of specific applications, providing more flexible transmission rates, lower latency, and more streamlined system costs. In addition, the private 2.4G protocol offers stronger and more stable connection performance than 2.4G wireless connection technology, with advantages such as millisecond-level response technology, plug-and-play, low power consumption, longer transmission distance, and suitability for high-speed data transmission. These advantages make the private 2.4G protocol suitable for specific applications such as high-performance wireless mice and remote-controlled toys. This application uses a customized private protocol to achieve the transmission of audio data in live broadcast scenarios, ensuring real-time transmission.

[0040] In some embodiments, to simplify the live broadcast system, the client 16 can be installed in the accompaniment device 14. For example, the accompaniment device 14 can be a personal terminal such as a mobile phone or tablet, and the client 16 can be installed on the personal terminal. Thus, the accompaniment audio can be provided through a single device, and the audio adjustment parameters can be remotely set.

[0041] In some embodiments, in order to facilitate users to remotely adjust the sound effects of the mixed data, such as Figure 4As shown, the live broadcast system may also include a wireless controller 17 wirelessly connected to the sound card device 12. The wireless controller 17 may be a hardware device that facilitates the user to adjust the sound effects. For example, the wireless controller 17 may include control components such as joysticks and buttons. The user may touch these control components to give sound effect adjustment instructions to the sound card device 12 to adjust the sound effects of the mixed data. The wireless controller 17 may obtain the user's sound effect control instructions and send the sound effect control instructions to the sound card device 12 through the second wireless transmission channel. The sound card device 12 may adjust the sound effects of the mixed data based on the sound effect control instructions, and then send it to the monitoring device 13 through the first wireless transmission channel, or output it to the live broadcast device 15. Considering that the sound effect data is usually pre-stored in the sound card device 12, the sound effect control only needs to control and adjust the sound effects, and thus the sound effect control instructions may be sent to the sound card device 12 through the second wireless transmission channel. In some embodiments, considering that the sound card device 12 acts as an audio data processing center, it needs to establish wireless connections with multiple devices and process multiple channels of data. In order to improve its processing performance, such as Figure 5 As shown, the sound card device 12 may include two main control chips, with the first and second wireless transmission channels integrated into the two main control chips, respectively. Considering that the two wireless transmission channels can operate simultaneously, to minimize interference between the two wireless transmission channels and the other wireless transmission channel during operation, either main control chip can determine the operating channel of its own wireless transmission channel based on the operating channel of the wireless transmission channel integrated into the other main control chip. For example, assume that the sound card device 12 includes main control chip 1 and main control chip 2, with the first wireless transmission channel integrated into main control chip 1 and the second wireless transmission channel integrated into main control chip 2. To minimize interference between the first wireless transmission channel and the second wireless transmission channel, when determining the current operating channel of the second wireless transmission channel, main control chip 2 can first obtain the channel identifier of the first wireless transmission channel's current operating channel (assuming it is channel A) from main control chip 1. Then, based on channel A, main control chip 2 can determine the operating channel of the second wireless transmission channel, minimizing interference with channel A.

[0042] The entire wireless communication frequency band can be roughly divided into three frequency ranges: the 2.40-2.44 GHz frequency range, where the primary interference source resides; the 2.44-2.46 GHz frequency range, where the first wireless transmission channel operates, with a total of 10 selectable operating channels; and the 2.46-2.48 GHz frequency range, where the second transmission channel operates. Because the two wireless transmission channels operate in essentially the same environment, they are subject to similar interference sources. Therefore, the operating channel ranges for the two wireless transmission channels are also essentially the same. Since wireless communication protocols typically specify how devices scan and process surrounding signal interference through mechanisms such as adaptive frequency hopping, interference detection, continuous channel monitoring, and signal strength detection to ensure that wireless devices can maintain stable connections and communication quality in complex wireless environments, the sound card device can estimate the channel where the main interference source is located by scanning the surrounding signals based on the scanning mechanisms specified in these protocols. Then, based on the frequency band of the interference source, the sound card device can reallocate the working channels of the two wireless transmission channels to be as far away from the interference source as possible, and the interval between the working channels of the two wireless transmission channels is not likely to cause adjacent channel interference, so that the first wireless transmission channel and the second wireless transmission channel can operate simultaneously without interfering with each other. In some embodiments, to reduce channel interference between the first wireless transmission channel and the second wireless transmission channel, the number of channels between the working channels of the first wireless transmission channel and the working channels of the second wireless transmission channel is greater than a preset threshold. That is, the working channels of the two wireless transmission channels should not be adjacent to each other as much as possible, but should have a certain degree of isolation to minimize interference between the two.

[0043] In some embodiments, when the working channel of the first wireless transmission channel is switched to a new channel, the working channel of the second wireless transmission channel is also switched to another new channel. When the frequency hopping process is triggered in either the first wireless transmission channel or the second wireless transmission channel, the main control chip 1 or 2 can identify the distribution of the main interference channels based on the strength of the interference signal, identify the available idle channels, and divide the idle channels into two relatively isolated available channel intervals, and respectively allocate them to the first wireless transmission channel and the second wireless transmission channel as the preferred frequency bands for frequency hopping, so as to achieve the purpose of working independently of each other.

[0044] In addition, the embodiment of the present application also provides a sound card device 12, such as Figure 6As shown, the sound card device 12 is wirelessly connected to the audio capture device 11, the monitoring device 13, and the accompaniment device 14, respectively. The sound card device 12 includes a first wireless transmission channel and a second wireless transmission channel. The sound card device 12 may include two wireless modules (such as wireless module 1 and wireless module 2 in the figure), and the first wireless transmission channel and the second wireless transmission channel may be implemented by two wireless modules respectively. The first wireless transmission channel and the second wireless transmission channel operate based on different wireless transmission protocols, and the transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel. The sound card device 12 is used to receive the voice data of the live broadcast user collected by the audio capture device 11 through the first wireless transmission channel, receive the accompaniment audio sent by the accompaniment device 14 through the second wireless transmission channel, mix the voice data and the accompaniment audio to obtain mixed data, and send the mixed data to the monitoring device 13 through the first wireless transmission channel so that the monitoring device 13 can play the mixed data; and output the mixed data to the live broadcast device 15.

[0045] In some embodiments, the first wireless transmission channel implements data transmission based on a user-defined private protocol, and the second wireless transmission channel implements data transmission based on an existing standard wireless transmission protocol.

[0046] In some embodiments, the existing standard wireless transmission protocol includes a Bluetooth protocol.

[0047] In some embodiments, the sound card device 12 includes two main control chips, and the first wireless transmission channel and the second wireless transmission channel are respectively integrated on the two main control chips. Any of the two main control chips can determine the working channel of the wireless transmission channel integrated in itself based on the working channel of the wireless transmission channel integrated on the other main chip.

[0048] In some embodiments, the number of channels between the working channel of the first wireless transmission channel and the working channel of the second wireless transmission channel is greater than a preset number threshold.

[0049] Among them, the specific structure and function implementation of the sound card device 12 can refer to the introduction of the sound card device 12 in the above-mentioned live broadcast system, and will not be repeated here.

[0050] In addition, an embodiment of the present application also provides an audio acquisition device 11, which can realize the functions of the audio acquisition device 11 in the live broadcast system introduced in the above embodiment.

[0051] In addition, an embodiment of the present application further provides a monitoring device 13, which can implement the functions of the monitoring device 13 in the live broadcast system introduced in the above embodiment.

[0052] In some embodiments, the monitoring device may be a Bluetooth headset that supports a customized private 2.4G protocol and a Bluetooth protocol.

[0053] Live streaming and other scenarios demand high-quality audio recording with low latency, low power consumption, and robust transmission range. However, the standard Bluetooth protocol, A2DP (Advanced Audio Distribution Profile), still exhibits relatively high latency, while the Hands-free Profile (HFP) suffers from codec limitations and offers poor audio quality. Therefore, users often opt for proprietary 2.4G protocols for wireless audio transmission. Compared to standard Bluetooth, proprietary 2.4G protocols offer enhanced customization for specific applications, providing more flexible transmission rates, lower latency, and streamlined system costs. Furthermore, proprietary 2.4G protocols offer more robust and stable connection performance than 2.4G wireless technology, with advantages such as millisecond-level response time, plug-and-play functionality, low power consumption, and extended transmission distance, making them suitable for high-speed data transmission. These advantages make proprietary 2.4G protocols particularly suitable for applications such as high-performance wireless mice and remote-controlled toys. This application pioneered the proposal to apply Bluetooth headsets that support private 2.4G protocols and Bluetooth protocols to monitoring devices. In live broadcast scenarios, the headset can work based on a customized 2.4G protocol and be used as a monitoring headset. In ordinary scenarios, it can be used as an ordinary Bluetooth headset.

[0054] In order to further introduce the live broadcast system provided by the embodiment of the present application, a specific embodiment is introduced below.

[0055] In traditional personal live streaming systems, microphones, monitoring headphones, mobile phones used to provide accompaniment audio, and other devices all need to be connected to the sound card device via wires, making the live streaming system very cumbersome to set up, especially in scenarios where outdoor live streaming is not possible. Based on this, this embodiment replaces the wired connection between the microphone, monitoring headphones, mobile phones used to provide accompaniment audio, and the sound card device with a wireless connection, which can greatly reduce the complexity of carrying equipment when traveling and the complexity of setting up the live streaming system.

[0056] like Figure 7As shown, this is a schematic diagram of the live broadcast system of this embodiment. Considering that the devices such as mobile phones that provide accompaniment audio and install APPs for implementing sound effect adjustment parameter settings are manufactured by third-party manufacturers, in order to establish wireless connections with these devices, universal protocols such as WiFi and Bluetooth can be used. In order to save power, the Bluetooth protocol is selected in this embodiment. Since the delay of the classic Bluetooth protocol A2DP is too large, the audio quality supported by the HFP protocol is poor due to codec restrictions. However, the wireless microphone 21 and the monitoring headphones 23 need to ensure low delay and high-quality audio quality for recording and monitoring, while also taking into account low power consumption, transmission distance and anti-interference performance. The audio data transmission between the wireless microphone 21, the monitoring headphones 23 and the sound card device 22 is not suitable for the use of the Bluetooth protocol. Therefore, a custom 2.4G private protocol can be used between the wireless microphone 21, the monitoring headphones 23 and the sound card device 22 for wireless transmission of audio data. The transmission delay of this custom 2.4G private protocol is lower than the transmission delay of the Bluetooth protocol.

[0057] To enable wireless connection between the sound card device 22 and the aforementioned multiple devices, it includes two wireless transmission modules, corresponding to two wireless transmission channels, RF1 and RF2. RF1 operates on a proprietary 2.4G wireless protocol, while RF2 operates on a standard wireless Bluetooth protocol. The sound card device 22 can transmit data via RF1 to the monitoring headphones 23 and wireless microphone 21, and via RF2 to the wireless controller and mobile phone A24 (which provides accompaniment audio and has an app installed for setting sound effect adjustment parameters). Since the two wireless transmission channels operate simultaneously, to prevent mutual interference between the two channels, in addition to ensuring maximum wireless isolation in the hardware design, a related wireless frequency hopping strategy has also been designed at the software level. The two wireless transmission channels are integrated on different main control chips. When RF1 needs to hop to address external interference, RF1 can obtain RF2's current operating frequency band and stagger it with RF2's operating frequency band, thereby minimizing the possibility of mutual interference and ensuring the reliability of wireless communication.

[0058] The workflow of the entire live broadcast system is as follows:

[0059] (1) RF1 of the sound card device 22 is paired with the wireless microphone 21 and the monitoring headset 23 using the 2.4G private protocol, and RF2 of the sound card device 22 is paired with the mobile phone and the wireless controller using the Bluetooth standard protocol.

[0060] (2) The wireless microphone 21 collects voice data, encodes it and sends it.

[0061] The wireless microphone 21 collects the host's voice data. In order to ensure the reliability of wireless transmission and take into account low latency and high-quality audio quality, the voice data can be encoded using the LC3+ encoding method and sent to the sound card device 22 through RF1.

[0062] (3) The sound card device 22 receives the accompaniment audio and the sound effect adjustment parameters sent by the mobile phone A24, mixes the voice data and the accompaniment audio, and processes the mixed data based on the sound effect adjustment parameters.

[0063] To facilitate remote sound adjustment, mobile phone A24 includes an app that allows users to set sound effect parameters. Sound card device 22 receives the accompaniment audio and sound effect adjustment parameters sent by mobile phone A24 via RF2. It decodes the accompaniment audio and mixes it with the voice data. It then adjusts the sound effects of the mixed data based on the received sound effect adjustment parameters.

[0064] (4) The sound card device 22 receives the sound effect control instruction sent by the wireless controller 26 to adjust the sound effect of the mixed data.

[0065] In order to facilitate users to adjust the sound effects, the live broadcast system may also include a wireless controller 26, and users can issue sound effect control instructions through the wireless controller 26. The sound card device 22 receives the sound effect control instructions sent by the wireless controller 26 via RF2 to adjust the sound effects and volume of the mixed data.

[0066] (5) The sound card device 22 outputs the processed mixed audio data in two ways, one way is sent to the monitoring earphone 23 via RF1, and the other way is output to the mobile phone B25 as the audio of the live video stream.

[0067] (6) The monitoring earphone 23 decodes the mixed audio data.

[0068] After receiving the mixed audio data sent by the sound card device 22, the monitoring earphone 23 decodes and plays the mixed audio data so that the user can monitor whether there is any audio abnormality during the live broadcast process.

[0069] Among them, the solutions of the above embodiments can be freely combined to obtain new solutions when there is no conflict. Due to space constraints, they will not be listed one by one here.

[0070] In addition, an embodiment of the present application also provides an audio acquisition device, which is used to perform the functions of the audio acquisition device in the live broadcast system introduced in the above embodiment.

[0071] In addition, an embodiment of the present application also provides a monitoring device, which is used to implement the functions of the monitoring device in the live broadcast system introduced in the above embodiment.

[0072] In addition, an embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed, it realizes the functions of the client in the live broadcast system introduced in the embodiment.

[0073] Correspondingly, an embodiment of the present application also provides a computer storage medium, in which a program is stored. When the program is executed by a processor, the functions of the client in the live broadcast system introduced in any of the above embodiments are realized.

[0074] The embodiments of the present application may take the form of a computer program product implemented on one or more storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0075] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0076] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0077] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0078] The above is a detailed introduction to the methods and devices provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of the present application should not be understood as a limitation on the present application.

Claims

1. A live broadcast system, characterized in that: The live broadcast system includes an audio acquisition device, a sound card device, and a monitoring device. The audio acquisition device and the monitoring device are respectively wirelessly connected to the sound card device, and the sound card device is also wirelessly connected to the accompaniment device. The sound card device includes a first wireless transmission channel and a second wireless transmission channel. The first wireless transmission channel and the second wireless transmission channel operate based on different wireless transmission protocols. The transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel. The first wireless transmission channel and the second wireless transmission channel are respectively implemented by two wireless modules. The audio acquisition device is used to collect voice data of live broadcast users and send it to the sound card device through the first wireless transmission channel; The sound card device is used to receive the accompaniment audio sent by the accompaniment device through the second wireless transmission channel, mix the voice data and the accompaniment audio to obtain mixed data, and send the mixed data to the monitoring device through the first wireless transmission channel; and for outputting the mixed audio data to a live broadcast device; The monitoring device is used to play the mixed audio data so that the live broadcast user can monitor the mixed audio data; The sound card device includes two main control chips, the first wireless transmission channel and the second wireless transmission channel are respectively integrated on the two main control chips, and either of the two main chips determines the working channel of the wireless transmission channel integrated in the main chip based on the working channel of the wireless transmission channel integrated in the other main chip; The number of channels between the working channel of the first wireless transmission channel and the working channel of the second wireless transmission channel is greater than a preset number threshold.

2. The live broadcast system according to claim 1, characterized in that The live broadcast system further includes a client wirelessly connected to the sound card device, and the client is used to send the sound effect adjustment parameters set by the user to the sound card device through the second wireless transmission channel; After mixing the voice data and the accompaniment audio to obtain mixed data, the sound card device is further configured to perform sound effect adjustment processing on the mixed data based on the sound effect adjustment parameter, and then transmit the mixed data after the sound effect adjustment processing to the monitoring device through the first wireless transmission channel, and output the mixed data to the live broadcast device; or Before the sound card device mixes the voice data and the accompaniment audio to obtain mixed data, it is also used to perform sound effect adjustment processing on the voice data and / or accompaniment audio based on the sound effect adjustment parameters, and then perform mixing processing to obtain mixed data, and send it to the monitoring device through the first wireless transmission channel, and output it to the live broadcast device.

3. The live broadcast system according to claim 1 or 2, characterized in that: The first wireless transmission channel implements data transmission based on a user-defined private protocol, and the second wireless transmission channel implements data transmission based on an existing standard wireless transmission protocol.

4. The live broadcast system according to claim 3, characterized in that The existing standard wireless transmission protocol includes the Bluetooth protocol, and the customized private protocol is a private 2.4G protocol.

5. The live broadcast system according to claim 1, characterized in that: The live broadcast system also includes a wireless controller wirelessly connected to the sound card device, and the wireless controller is used to obtain the user's sound effect control instructions and send the sound effect control instructions to the sound card device through the second wireless transmission channel; the sound card device is used to perform sound effect adjustment processing on the mixed data based on the sound effect control instructions, and then send it to the monitoring device through the first wireless transmission channel, and output it to the live broadcast device.

6. A sound card device, characterized in that: The sound card device is wirelessly connected to the audio acquisition device, the monitoring device, and the accompaniment device respectively, and the sound card device includes a first wireless transmission channel and a second wireless transmission channel. The first wireless transmission channel and the second wireless transmission channel operate based on different wireless transmission protocols, and the transmission delay of the first wireless transmission channel is lower than the transmission delay of the second wireless transmission channel. The sound card device is configured to receive voice data of a live broadcast user collected by the audio collection device through the first wireless transmission channel, receive accompaniment audio sent by the accompaniment device through the second wireless transmission channel, mix the voice data and the accompaniment audio to obtain mixed data, and send the mixed data to the monitoring device through the first wireless transmission channel so that the monitoring device plays the mixed data; and for outputting the mixed audio data to a live broadcast device; The first wireless transmission channel and the second wireless transmission channel are respectively implemented by two wireless modules; the sound card device includes two main control chips, the first wireless transmission channel and the second wireless transmission channel are respectively integrated on the two main control chips, and either of the two main chips determines the working channel of the wireless transmission channel integrated into itself based on the working channel of the wireless transmission channel integrated on the other main chip; The number of channels between the working channel of the first wireless transmission channel and the working channel of the second wireless transmission channel is greater than a preset number threshold.

7. The sound card device according to claim 6, characterized in that: The first wireless transmission channel implements data transmission based on a user-defined private protocol, and the second wireless transmission channel implements data transmission based on an existing standard wireless transmission protocol.

8. The device according to claim 6, characterized in that When the working channel of the first wireless transmission channel is switched to a new channel, the working channel of the second wireless transmission channel is also switched to another new channel.

Images