Wireless audio-video system, mobile terminal and wireless acquisition device

Abstract

This application discloses a wireless audio-visual system, a mobile terminal, and a wireless acquisition device. The wireless audio-visual system includes: a wireless acquisition device, a mobile terminal with video recording function, and a wireless monitoring device. The wireless acquisition device is used to acquire first audio data and transmit the first audio data to the mobile terminal via a wireless communication connection. The mobile terminal is used to transmit the received first audio data, second audio data obtained based on the first audio data, or audio-visual data obtained based on the first audio data to the wireless monitoring device via a wireless communication connection.

Description

Technical Field

[0001] This application belongs to the field of wireless communication technology, specifically relating to a wireless audio and video system, a mobile terminal, and a wireless acquisition device. Background Technology

[0002] Current wireless audio systems are undergoing continuous iteration and optimization, rapidly evolving towards lighter designs, clearer sound quality, lower latency, and stronger anti-interference capabilities. Current wireless audio systems include wireless acquisition devices (e.g., wireless microphones), wireless audio receivers, wireless monitoring devices (e.g., Bluetooth headsets), and audio-visual systems (e.g., mobile phones) physically connected to the wireless audio receiver. The wireless audio receiver must receive audio from the wireless microphone, send the processed audio to the Bluetooth headset, and transmit audio to the mobile phone via a physical interface. Since the transmitting and receiving antennas of the wireless audio receiver are extremely close together, self-interference occurs between the transmitted and received signals. This causes strong transmitted signals to suppress weak received signals, hindering efficient and high-quality transmission of the audio captured by the wireless microphone. Summary of the Invention

[0003] The purpose of this application is to provide a wireless audio and video system, a mobile terminal, and a wireless acquisition device, which can solve the problem in related technologies that wireless audio receiving devices cannot efficiently and with high quality transmit the audio acquired by the wireless microphone.

[0004] In a first aspect, embodiments of this application provide a wireless audio and video system, which includes: a wireless acquisition device, a mobile terminal with video recording function, and a wireless monitoring device; the wireless acquisition device is used to acquire first audio data and transmit the first audio data to the mobile terminal via a wireless communication connection; the mobile terminal is used to transmit the received first audio data, second audio data obtained based on the first audio data, or audio and video data obtained based on the first audio data to the wireless monitoring device via a wireless communication connection.

[0005] Secondly, embodiments of this application provide a mobile terminal, which includes: a camera module for recording video data; a first wireless communication module for establishing a first wireless communication connection with an external wireless acquisition device to obtain first audio data; a second wireless communication module for establishing a second wireless communication connection with a wireless monitoring device to transmit the received first audio data, second audio data obtained based on the first audio data, or audio-visual data obtained based on the first audio data; and a processor for processing the first audio data to obtain the second audio data or the audio-visual data.

[0006] Thirdly, embodiments of this application provide a wireless acquisition device, which includes: an audio acquisition module for acquiring first audio data; and a wireless transmission module for transmitting the first audio data to a mobile terminal with video recording capabilities via a wireless communication connection.

[0007] Fourthly, embodiments of this application provide a wireless audio and video component, which includes: at least one wireless monitoring device and a wireless acquisition device as described in the second aspect, wherein the wireless acquisition device is independently configured from the at least one wireless monitoring device.

[0008] The wireless audio-visual system provided in this application embodiment includes a wireless acquisition device, a mobile terminal with video recording function, and a wireless monitoring device. The wireless acquisition device acquires first audio data and transmits the first audio data to the mobile terminal via a wireless communication connection. The mobile terminal transmits the received first audio data, second audio data obtained based on the first audio data, or audio-visual data obtained based on the first audio data to the wireless monitoring device via a wireless communication connection. This application embodiment's wireless audio-visual system eliminates the wireless audio receiving device in related technologies' wireless audio systems; the wireless acquisition device directly connects wirelessly to the mobile terminal and receives the first audio data transmitted by the wireless acquisition device. The mobile terminal also connects wirelessly to the wireless monitoring device, transmitting the received first audio data, second audio data obtained based on the first audio data, or audio-visual data obtained based on the first audio data to the wireless monitoring device. The mobile terminal acts as the core hub, transmitting the audio data acquired by the wireless acquisition device and the audio-visual data obtained based on the audio data. By eliminating the wireless audio receiving device, the self-interference problem caused by the wireless audio receiving device is avoided, which prevents the efficient and high-quality transmission of audio and video data. The transmission of audio and video data is realized by the mobile terminal, and the complete communication performance of the mobile terminal enables the efficient and high-quality transmission of audio and video data. Attached Figure Description

[0009] Figure 1a A schematic diagram of the structure of a wireless audio system in the related art is shown; Figure 1b A flowchart illustrating a signaling interaction method for a wireless audio system in the related art is shown. Figure 2 This paper shows a schematic diagram of the structure of a wireless audio and video system provided in an embodiment of this application; Figure 3 This paper shows a schematic diagram of the structure of another wireless audio and video system provided in an embodiment of this application; Figure 4 This illustration shows a structural diagram of a mobile terminal provided in an embodiment of this application; Figure 5 This paper shows a schematic diagram of the structure of a wireless data acquisition device provided in an embodiment of this application; Figure 6 This illustration shows a structural schematic diagram of a wireless audio / video component provided in an embodiment of this application; Figure 7 This illustration shows a structural schematic diagram of yet another wireless audio-visual system provided in an embodiment of this application; Figure 8 A schematic flowchart of a signaling interaction method for a wireless audio and video system provided in an embodiment of this application is shown. Figure 9a This illustration shows a schematic diagram of the time difference between audio data and video data provided in an embodiment of this application; Figure 9b This illustration shows another time difference between audio and video data provided in an embodiment of this application; Figure 9c A schematic diagram comparing the related technologies provided in the embodiments of this application with the delays generated in the embodiments of this application is shown; Figure 10 A flowchart illustrating an audio / video data transmission method provided in an embodiment of this application is shown. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0011] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0012] See Figure 1a , Figure 1a A schematic diagram of a wireless audio system 1000 in the related art is shown. The wireless audio system 1000 includes a wireless acquisition device 11 (e.g., a wireless microphone), a wireless audio receiving device 12, and a wireless monitoring device 13 (e.g., a Bluetooth headset). The wireless acquisition device 11 includes two receiving microphone devices 111 and 112, a main control module 113, and two wireless modules 114 and 115. The wireless audio receiving device 12 includes two wireless modules 121 and 122, and a main control module 123. The wireless monitoring device 13 includes a wireless module 131 and a main control module 132.

[0013] In wireless audio systems within related technologies, see [link / reference]. Figure 1b , Figure 1b The diagram illustrates a signaling interaction method for a wireless audio system in the related art. Wireless acquisition device 11 and wireless audio receiving device 12 establish a wireless communication connection via a proprietary 2.4G protocol, while wireless audio receiving device 12 and wireless monitoring device 13 establish a wireless communication connection via a standard Bluetooth protocol. The microphones 111 and 112 of wireless acquisition device 11 acquire audio. The main control module 113 preprocesses the acquired audio, such as ambient noise reduction, backup storage of the bass track, analog-to-digital conversion, digital signal processing, and encoding compression. Then, wireless modules 114 or 115 adjust and transmit the audio to wireless module 121 of wireless audio receiving device 12. Its main control module 123 demodulates, decompresses, decodes, and performs digital-to-analog conversion on the audio before transmitting it to wireless module 131 of wireless monitoring device 13 via wireless module 122. Its main control module 132 processes the received audio and outputs it for user monitoring via the wireless monitoring device. Furthermore, the wireless audio receiving device 12 establishes a communication connection with the audio-visual system 14 through a data cable and a physical interface. The main control module 123 of the wireless audio receiving device 12 also transmits the processed audio to the audio-visual system 14 (e.g., a mobile phone) through a data cable and a physical interface. The audio-visual system 14 then performs personalized processing on the audio.

[0014] During the aforementioned audio transmission process, both the wireless microphone and the Bluetooth headset operate on the same frequency band. However, the total bandwidth of this frequency band is limited. Communication between the wireless microphone and the wireless audio receiving device, and between the wireless audio receiving device and the Bluetooth headset, will compete for resources. Furthermore, the wireless audio receiving device both receives and transmits signals, and due to the extremely close physical distance between its receiving and transmitting antennas, it will also experience self-interference. The communication between the wireless microphone and the wireless audio receiving device is based on a proprietary 2.4G protocol, while the communication between the wireless audio receiving device and the Bluetooth headset is based on the standard Bluetooth protocol. There is a protocol conflict between the two: the proprietary 2.4G protocol typically uses frequency-hopping spread spectrum (FHSS) to avoid interference, while the standard Bluetooth protocol also has its own frequency-hopping mechanism. This can lead to channel overlap, increasing the retransmission rate and causing audio quality degradation or disconnection. Multiple communication links between the wireless microphone and the wireless audio receiving device, and between the wireless audio receiving device and the Bluetooth headset, all experience latency. Moreover, third-party manufacturers cannot modify the underlying protocol of the mobile phone, making it impossible to perform audio-visual synchronization adaptation, thus exacerbating the audio-visual desynchronization problem. In addition, the number of wireless monitoring devices that can be connected to the wireless audio receiving device in existing wireless audio systems is limited to one or two, which restricts the number of users that can be monitored and is not convenient for team or multi-person collaboration.

[0015] The wireless audio-visual system, mobile terminal, and wireless acquisition device provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios, so as to at least solve the above-mentioned problems.

[0016] Figure 2 A schematic diagram of the structure of a wireless audio and video system provided in an embodiment of this application is shown below. Figure 2 The wireless audio and video system 200 includes a wireless acquisition device 21, a mobile terminal 22 with video recording function, and a wireless monitoring device 23.

[0017] The wireless acquisition device 21 is used to acquire first audio data and transmit the first audio data to the mobile terminal 22 via a wireless communication connection; the mobile terminal 22 is used to transmit the received first audio data, second audio data obtained based on the first audio data, or audio and video data obtained based on the first audio data to the wireless monitoring device 23 via a wireless communication connection.

[0018] The wireless acquisition device 21 may include a wireless microphone, lavalier microphone, etc. The mobile terminal 22 may include a mobile phone, a camera, a handheld gimbal, an action camera, a drone, etc. The wireless listening device 23 may include a Bluetooth headset, etc.

[0019] The wireless audio-visual system 200 of this application embodiment eliminates the wireless audio receiving device 12 in the wireless audio system 1000 of the related art. The wireless acquisition device 21 is directly wirelessly connected to the mobile terminal 22 and receives the first audio data transmitted by the wireless acquisition device 21. The mobile terminal 22 is also wirelessly connected to the wireless monitoring device 23, and transmits the received first audio data, second audio data obtained based on the first audio data, or audio-visual data obtained based on the first audio data to the wireless monitoring device 23. The mobile terminal 22 serves as the core hub, transmitting the audio data acquired by the wireless acquisition device 21 and the audio-visual data obtained based on the audio data to the wireless monitoring device 23. Thus, by eliminating the wireless audio receiving device, the self-interference problem of the wireless audio receiving device is avoided, which would prevent the efficient and high-quality transmission of audio and audio-visual data. The transmission of audio and audio-visual data is achieved by the mobile terminal 22, and the complete communication performance of the mobile terminal 22 enables efficient and high-quality transmission of audio and audio-visual data.

[0020] In some embodiments, the wireless acquisition device 21 and the mobile terminal 22 establish a first wireless communication connection for transmitting the first audio data based on the LE Audio protocol.

[0021] Among them, the LE Audio (Low Energy Audio) protocol mainly provides low-latency, low-power, efficient, and high-quality wireless communication connections for wireless acquisition device 21 and mobile terminal 22 at the LC3 encoder or LC3plus encoder, enhanced audio stream (CIS (Connected Isochronous Stream)) and broadcast audio function levels.

[0022] In some embodiments, the first wireless communication connection satisfies at least one of the following: (1) The wireless acquisition device 21 sends the first audio data to the mobile terminal 22 through the CIS mode or low latency LL mode based on the LE Audio protocol.

[0023] The LE Audio protocol uses Bluetooth Low Energy radio instead of the traditional Basic Rate / Enhanced Data Rate (BR / EDR) and introduces a synchronous isochronous channel (CIS) for transmission. The CIS mode supports precise synchronization of audio data between the wireless acquisition device 21 and the mobile terminal 22, ensuring timely transmission of audio data to the mobile terminal 22. Based on the LL (Low Latency) mode setting, the wireless acquisition device 21 transmits the first audio data to the mobile terminal 22 according to the low latency mode configuration to reduce the latency of transmitting the first audio data.

[0024] (2) The wireless acquisition device 21 uses an LC3 encoder or an LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms.

[0025] Among them, the LC3 encoder (Low Complexity Communication Codec) or LC3plus encoder has the advantages of high efficiency and low power consumption. At the same bit rate, the sound quality of the first audio data encoded by the LC3 encoder or LC3plus encoder is significantly better than that of the traditional SBC (Subband Coding). In other words, at the same sound quality, the bit rate required by the LC3 encoder or LC3plus encoder is half that of SBC. Furthermore, encoding with a frame duration of less than or equal to 5ms allows the wireless acquisition device 21 to perform encoding in a shorter waiting time, reducing latency and improving retransmission efficiency. Therefore, using the LC3 encoder or LC3plus encoder can bring lower latency and minimal storage space requirements, and can reduce the bit rate to a lower level without sacrificing too much sound quality, achieving a balance between high sound quality and low power consumption.

[0026] For example, the LC3 encoder supports a flexible bitrate range of 16-345kbps and a maximum sampling rate of 48kHz, maintaining excellent sound quality even at low bitrates. The LC3plus encoder supports 192kbps, achieving subjective sound quality close to aptX HD at 192kbps, and features efficient compression and strong packet loss resistance, reducing bandwidth requirements while maintaining sound quality. In terms of power consumption, the LE Audio protocol achieves up to 40% power reduction compared to traditional Bluetooth protocols, resulting in 30-40% longer playback time with the same battery capacity, achieving a practical design life of 10-20 hours. Regarding latency performance, test data shows that the LC3 encoder has a latency of approximately 50ms, and the LC3plus encoder has a latency of approximately 25ms, a significant improvement over the 220ms latency of the traditional SBC encoder.

[0027] In some optional embodiments, the first wireless communication connection satisfies both the requirement that the wireless acquisition device 21 and the mobile terminal 22 establish a first wireless communication connection for transmitting the first audio data based on the LE Audio protocol, and the requirement that when the wireless acquisition device 21 uses an LC3 encoder or LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms, it can ensure that the wireless acquisition device 21 transmits the first audio data to the mobile terminal 22 in a way that has lower latency and lower power consumption. This allows the wireless acquisition device 21 and the mobile terminal 22 to reduce latency and power consumption and improve the efficiency and quality of transmitting the first audio data without using a proprietary protocol.

[0028] (3) When transmitting the first audio data through the first wireless communication connection, turn off the sound enhancement function or noise reduction function of the mobile terminal 22.

[0029] When the wireless acquisition device 21 transmits the first audio data to the mobile terminal 22 through the first wireless communication connection, it turns off the sound enhancement function or noise reduction function of the mobile terminal 22, prioritizes ensuring that the first audio data is completely received by the mobile terminal 22, turns off the sound enhancement function or noise reduction function that is not currently necessary for the mobile terminal 22, and performs necessary processing on the received first audio data, such as decoding and encoding, so that the mobile terminal 22 can efficiently receive and transmit the first audio data.

[0030] In this embodiment, the wireless acquisition device 21 and the mobile terminal 22 establish a first wireless communication connection for transmitting the first audio data based on the LE Audio protocol, which can satisfy at least one of the following: (1) The wireless acquisition device 21 sends the first audio data to the mobile terminal 22 through the CIS mode or low-latency LL mode based on the LE Audio protocol, so as to ensure that the common LE Audio protocol is used and the CIS mode or low-latency LL mode is set, thereby reducing the latency of transmitting the first audio data and maximizing the synchronization of the first audio data collected by the wireless acquisition device 21 to the mobile terminal 22 on time. (2) The wireless acquisition device 21 uses an LC3 encoder or an LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms, so as to ensure that the sound quality of the first audio data is improved while reducing power consumption and latency. (3) When transmitting the first audio data through the first wireless communication connection, the sound enhancement function or noise reduction function of the mobile terminal 22 is turned off, so as to improve the processing efficiency of the mobile terminal 22 on the first audio data, thereby efficiently transmitting the first audio data. The above three factors can be combined to improve the transmission efficiency and quality of the first audio data from multiple aspects, including reducing latency, reducing power consumption, improving sound quality, and improving the processing efficiency of the mobile terminal 22.

[0031] In some embodiments, transmitting the first audio data to the mobile terminal 22 via the above-described wireless communication connection transmission method may include the following:

[0032] The wireless acquisition device 21 and the mobile terminal 22 establish a first wireless communication connection for transmitting the first audio data; the transmission power of the wireless acquisition device 21 and / or the channel quality of the first wireless communication connection are obtained, and the operating frequency band is determined according to the transmission power and / or the channel quality of the first wireless communication connection; the first wireless communication connection is switched to the operating frequency band using an adaptive frequency hopping (AFH) method.

[0033] In this embodiment, the quality of the received signal is evaluated by obtaining the transmission power of the wireless acquisition device 21. For example, if the wireless acquisition device 21 has used its maximum transmission power to transmit a signal but the signal quality received by the mobile terminal 22 is poor, the current operating frequency band is determined to be unsuitable based on the transmission power, requiring a switch to the operating frequency band and triggering a reselection of the operating frequency band. Alternatively, the operating frequency band suitable for the first wireless communication connection can be determined by obtaining the channel quality of the first wireless communication connection. Or, the operating frequency band with less signal interference and high channel quality can be determined by obtaining the transmission power of the wireless acquisition device 21 and the channel quality of the first wireless communication connection, and then an adaptive frequency hopping (AFH) method is used to switch the first wireless communication connection to the determined operating frequency band. The wireless acquisition device 21 transmits signals to transmit the first audio data in this operating frequency band, and the mobile terminal 22 receives the first audio data in this operating frequency band, thereby improving the efficiency and quality of transmitting the first audio data.

[0034] In some embodiments, the operating frequency band of the first wireless communication connection is from 2.4 GHz to 5 GHz.

[0035] In some optional embodiments, the operating frequency band of the first wireless communication connection can be a high-frequency band of 2.4 GHz, such as 2.470-2.483 GHz. In this operating frequency band, the first wireless communication connection can avoid frequency bands where other networks operate densely, such as Wi-Fi (Wireless Fidelity), to avoid interference with other networks, thereby reducing interference to the first wireless communication connection and improving the efficiency and quality of the wireless acquisition device 21 transmitting the first audio data to the mobile terminal 22.

[0036] In some embodiments, the number of wireless acquisition devices 21 that simultaneously transmit audio data to the mobile terminal 22 via the wireless communication connection is less than or equal to 4.

[0037] In this embodiment, multiple wireless acquisition devices 21 can simultaneously transmit audio data to the mobile terminal 22 via wireless communication. Due to bandwidth limitations, the simultaneous transmission of audio data to the mobile terminal 22 by a limit of 4 or fewer wireless acquisition devices 21 is avoided to prevent bandwidth overload from reducing the efficiency and quality of the mobile terminal 22 in receiving audio data.

[0038] In some embodiments, the wireless acquisition device 21 and the mobile terminal 22 establish a first wireless communication connection for transmitting the first audio data; the mobile terminal 22 and the wireless monitoring device 23 establish a second wireless communication connection.

[0039] Wherein, the channel priority of the first wireless communication connection is higher than that of the second wireless communication connection; when the channel bandwidth resources cannot simultaneously meet the channel bandwidth requirements of the first wireless communication connection and the second wireless communication connection, the mobile terminal 22 shall prioritize allocating bandwidth resources for the channel occupied by the first wireless communication connection.

[0040] In this embodiment, the wireless acquisition device 21 transmits first audio and video data to the mobile terminal 22 via a first wireless communication connection. The mobile terminal 22 transmits first audio data, second audio data obtained based on the first audio data, or audio and video data obtained based on the first audio data via a second wireless communication connection and a wireless monitoring device 23. The data transmission includes the channels of the first and second wireless communication connections. Due to bandwidth limitations, the communication quality and latency requirements of the first wireless communication connection are prioritized, meaning the channel of the first wireless communication connection has a higher priority than the channel of the second wireless communication connection. When the channel bandwidth resources cannot simultaneously meet the channel bandwidth requirements of both the first and second wireless communication connections, bandwidth resources are allocated to the channel occupied by the first wireless communication connection. This ensures that the mobile terminal 22 can efficiently and effectively receive the first audio and video data acquired by the wireless acquisition device 21. After the mobile terminal 22 receives the first audio and video data, the bandwidth resources required by the second wireless communication connection are further considered. This ensures the communication efficiency and quality of both stages, preventing the efficiency and quality of the first wireless communication connection from being compromised and thus affecting the efficiency and quality of the second wireless communication connection.

[0041] In one embodiment, Figure 3 A schematic diagram of another wireless audio / video system is shown; see [link / reference]. Figure 3The mobile terminal 22 provided in this application embodiment is used to transmit the received first audio data, the second audio data obtained based on the first audio data, or the audio and video data obtained based on the first audio data to at least one of the wireless monitoring devices 23 in a broadcast manner.

[0042] In this embodiment, the broadcast method enables the mobile terminal 22 to perform one-to-many data transmission. The mobile terminal 22 can synchronously transmit first audio data, second audio data obtained based on the first audio data, or audio and video data obtained based on the first audio data to multiple wireless monitoring devices 23. This eliminates the need for the mobile terminal 22 to establish a one-to-one pairing connection with each wireless monitoring device 23, thereby reducing data transmission latency and meeting the needs of teams or multiple users to monitor through the wireless monitoring devices 23.

[0043] In other embodiments, the wireless acquisition device 21 and the mobile terminal 22 of the wireless audio-visual system 200 provided in this application establish a first wireless communication connection for transmitting the first audio data. This first wireless communication connection is established based on the public LE Audio protocol. The mobile terminal 22 and the wireless monitoring device 23 establish a second wireless communication connection, which is also implemented based on a public broadcast method. Thus, the entire link of the wireless audio-visual system 200 adopts a public standard wireless communication protocol, avoiding protocol conflicts between multiple communication links.

[0044] In some embodiments, the second audio data is obtained by the mobile terminal 22 through audio processing of the first audio data; and / or, the audio and video data is obtained by the mobile terminal 22 by merging the first audio data and the video data stored or captured by the mobile terminal 22; and / or, the audio and video data is obtained by the mobile terminal 22 by merging the second audio data and the video data stored or captured by the mobile terminal 22.

[0045] The audio processing of the first audio data by the mobile terminal 22 may include: artificial intelligence (AI) noise reduction, equalizer (EQ) adjustment, mixing, sound quality enhancement, sound effects addition, separation of human voice and ambient sound, and extraction or synthesis of ambient sound or human voice.

[0046] In this embodiment, after receiving the first audio data transmitted by the wireless acquisition device 21, the mobile terminal 22 uses its own audio processing and video processing functions to process the first audio data to obtain second audio data, thereby improving the sound quality of the second audio data or meeting personalized needs for the first audio data. Alternatively, the first audio data can be merged with video data stored or captured by the mobile terminal 22 to obtain audio-visual data, and / or the second audio data can be merged with video data stored or captured by the mobile terminal 22 to obtain audio-visual data, thus achieving the merging of the first audio data acquired by the wireless acquisition device 21 with the video data stored or captured by the mobile terminal 22, satisfying personalized needs for acquiring audio data for listening and acquiring audio-visual data for sharing.

[0047] In some embodiments, the wireless acquisition device 21 and the mobile terminal 22 establish a first wireless communication connection for transmitting the first audio data; the mobile terminal 22 is further configured to determine the Quality of Service (QoS) parameters based on the network status information of the first wireless communication connection, and configure the communication parameters of the first wireless communication connection based on the QoS parameters.

[0048] The communication parameters may include the operating frequency band, bandwidth resources, and the number of wireless acquisition devices 21.

[0049] In this embodiment, the mobile terminal 22 obtains network status information of the first wireless communication connection established with the wireless acquisition device 21, determines the Quality of Service (QoS) parameters of the first wireless communication connection based on the network status information, and configures the communication parameters of the first wireless communication connection based on the QoS parameters. That is, the transmission strategy between the wireless acquisition device 21 and the mobile terminal 22 is dynamically adjusted based on the real-time network status of the first wireless communication connection during communication, so that the transmission quality and efficiency of audio data can reach the optimal level under the constraints of real-time network status.

[0050] In some embodiments, the mobile terminal 22 is further configured to acquire video data, and the mobile terminal 22 is configured to synchronize the first audio data with the video data, and obtain the audio and video data based on the first audio data and the video data.

[0051] In this embodiment, the mobile terminal 22 can also acquire video data, and based on its own audio and video processing functions, it can perform synchronous processing on the first audio data and video data to optimize the audio-visual asynchrony problem between the received first audio data and the acquired video data. After solving the audio-visual asynchrony problem between the first audio data and video data, audio and video data are then obtained based on the first audio data and video data, so that the audio and video of the audio and video data achieve a synchronized effect.

[0052] In some embodiments, the mobile terminal 22 synchronously processes the first audio data and the video data, and obtains the audio and video data based on the first audio data and the video data, including the following.

[0053] The mobile terminal 22 processes the first audio data to obtain the second audio data.

[0054] The audio processing of the first audio data by the mobile terminal 22 may include: performing AI noise reduction, EQ adjustment, mixing, sound quality enhancement, adding sound effects, separating human voice and ambient sound, and extracting or synthesizing ambient sound or human voice, etc., to optimize the quality and effect of the first audio data.

[0055] The mobile terminal 22 sets a first timestamp corresponding to the second timestamp of the video data for the second audio data, determines the time difference between the second audio data and the video data based on the first timestamp and the second timestamp, and adjusts the time difference to within a preset threshold range to complete the synchronization processing of the second audio data and the video data.

[0056] Specifically, by adjusting the time difference between the first timestamp of the second audio data and the second timestamp of the video data, the time difference is reduced to within a preset threshold range, such as 20ms. When the time difference is within the preset threshold range, the user cannot perceive the audio-visual asynchrony, thus completing the synchronization processing of the second audio data and video data.

[0057] The mobile terminal 22 synthesizes the second audio data and the video data to obtain the audio and video data.

[0058] In this embodiment, the mobile terminal 22 obtains the second audio data by processing the first audio data. First, it processes the first audio data transmitted by the wireless acquisition device 21 to obtain second audio data that meets the user's needs and requirements. Then, it sets a first timestamp on the second audio data corresponding to the second timestamp of the video data. Based on the first and second timestamps, it determines the time difference between the second audio data and the video data, adjusting the time difference to within a preset threshold range to complete the synchronization processing of the second audio data and the video data, thus resolving the audio-visual asynchrony problem between the second audio data and the video data. Finally, it synthesizes the second audio data and the video data to obtain the audio-visual data, thereby obtaining synchronized audio-visual data for subsequent rapid sharing or creation.

[0059] Figure 4This paper shows a schematic diagram of the structure of a mobile terminal according to an embodiment of this application. See also: Figure 4 The mobile terminal 22 includes: a camera module 221, a first wireless communication module 222, a second wireless communication module 223, and a processor 224.

[0060] The camera module 221 is used to record video data; the first wireless communication module 222 is used to establish a first wireless communication connection with an external wireless acquisition device 21 to obtain first audio data; the second wireless communication module 223 is used to establish a second wireless communication connection with a wireless monitoring device 23 to transmit the received first audio data, second audio data obtained based on the first audio data, or audio and video data obtained based on the first audio data; and the processor 224 is used to process the first audio data to obtain the second audio data or the audio and video data.

[0061] In the mobile terminal 22 provided in this application embodiment, the first wireless communication module 222 establishes a first wireless communication connection with the external wireless acquisition device 21 to obtain first audio data. The processor 224 processes the first audio data to obtain the second audio data or the audio-visual data. Then, the second wireless communication module 223, which establishes a second wireless communication connection with the wireless monitoring device 23, transmits the received first audio data, the second audio data obtained based on the first audio data, or the audio-visual data obtained based on the first audio data to the wireless monitoring device 23. The mobile terminal 22 acts as the core hub, transmitting the audio data collected by the wireless acquisition device 21 and the audio-visual data obtained based on the audio data and video data. The complete communication performance of the mobile terminal 22 enables efficient and high-quality transmission of audio data and audio-visual data, avoiding the self-interference problem of the original wireless audio receiving device, which prevents efficient and high-quality transmission of audio data and audio-visual data.

[0062] In some embodiments, the first wireless communication module 222 and the wireless acquisition device 21 establish the first wireless communication connection based on the LEAudio protocol.

[0063] Among them, the LE Audio protocol mainly provides low-latency, low-power, efficient, and high-quality wireless communication connections for wireless acquisition device 21 and mobile terminal 22 at the LC3 encoder or LC3plus encoder, enhanced audio stream (CIS) and broadcast audio function levels.

[0064] In some embodiments, the first wireless communication module 222 satisfies at least one of the following: (1) The mobile terminal 22 receives the first audio data through CIS mode or low latency LL mode based on the LE Audio protocol.

[0065] Among them, the CIS mode or low-latency LL mode of the LE Audio protocol introduces a synchronous isochronous channel and a low-latency configuration, which enables the mobile terminal 22 and the wireless acquisition device 21 to accurately synchronize the first audio data, ensuring that the mobile terminal 22 receives the first audio data on time.

[0066] (2) The first audio data is generated based on LC3 encoding or LC3plus encoding.

[0067] Among them, the LC3 encoding method or LC3plus encoding method provides an efficient and low-power encoding method for the first audio data, which can reduce the power consumption of the wireless acquisition device 21 and the latency of the first wireless communication connection, thereby reducing the overall power consumption of the wireless audio and video system and the latency of the entire link.

[0068] In some alternative embodiments, the mobile terminal 22 uses a digital-to-analog converter to decode the first audio data.

[0069] (3) When transmitting the first audio data through the first wireless communication connection, turn off the sound enhancement function or noise reduction function of the mobile terminal 22.

[0070] When the mobile terminal 22 receives the first audio data transmitted by the wireless acquisition device 21 through the first wireless communication connection, it turns off the sound enhancement function or noise reduction function of the mobile terminal 22, prioritizes ensuring that the mobile terminal 22 receives the first audio data completely, turns off the sound enhancement function or noise reduction function that is not currently necessary for the mobile terminal 22, and performs necessary processing on the received first audio data, such as decoding and encoding, so that the mobile terminal 22 can efficiently receive and transmit the first audio data.

[0071] In this embodiment, the first wireless communication connection established between the mobile terminal 22 and the wireless acquisition device 21 satisfies at least one of the following: (1) The mobile terminal 22 receives the first audio data through the CIS mode or low-latency LL mode based on the LE Audio protocol to reduce the latency of transmitting the first audio data and maximize the synchronization of the mobile terminal 22 with the first audio data collected by the wireless acquisition device 21. (2) The first audio data is generated based on the LC3 encoding method or the LC3plus encoding method to ensure that the sound quality of the received first audio data is improved while reducing power consumption and latency. (3) When transmitting the first audio data through the first wireless communication connection, the sound effect enhancement function or noise reduction function of the mobile terminal 22 is turned off to improve the processing efficiency of the mobile terminal 22 on the first audio data, thereby efficiently transmitting the first audio data. The above three can be combined to improve the transmission efficiency and quality of the first audio data from multiple aspects, such as reducing latency, reducing power consumption and improving sound quality, and improving the processing efficiency of the mobile terminal 22.

[0072] In some embodiments, the second wireless communication module 223 is used to broadcast to the wireless monitoring device 23 the received first audio data, the second audio data obtained based on the first audio data, or the audio and video data obtained based on the first audio data.

[0073] In this embodiment, the second wireless communication module 223 can perform one-to-many data transmission in a broadcast manner, and can synchronously transmit first audio data, second audio data obtained based on the first audio data, or audio and video data obtained based on the first audio data to multiple wireless monitoring devices 23. There is no need for the second wireless communication module 223 to establish a one-to-one pairing connection with each wireless monitoring device 23, which can reduce the latency of data transmission and meet the needs of teams or multiple users to monitor through the wireless monitoring devices 23.

[0074] In some embodiments, the first wireless communication module 222 is configured to: establish a first wireless communication connection with the wireless acquisition device 21 for transmitting the first audio data; obtain the transmission power of the wireless acquisition device 21 and / or the channel quality of the first wireless communication connection, determine the operating frequency band based on the transmission power and / or the channel quality of the first wireless communication connection; and switch the first wireless communication connection to the operating frequency band using an adaptive frequency hopping (AFH) method.

[0075] In this embodiment, the first wireless communication module 222 evaluates the received signal quality by acquiring the transmission power of the wireless acquisition device 21. For example, if the wireless acquisition device 21 has used its maximum transmission power to transmit a signal but the signal quality received by the mobile terminal 22 is poor, the current operating frequency band is determined to be unsuitable based on the transmission power, requiring a switch to the operating frequency band and triggering a reselection of the operating frequency band. Alternatively, the operating frequency band suitable for the first wireless communication connection can be determined by acquiring the channel quality of the first wireless communication connection. Or, the operating frequency band with low signal interference and high channel quality can be jointly determined by acquiring the transmission power of the wireless acquisition device 21 and the channel quality of the first wireless communication connection, and then the first wireless communication connection can be switched to the determined operating frequency band using the AFH method. This instructs the wireless acquisition device 21 to transmit signals and transmit the first audio data using the operating frequency band, and the mobile terminal 22 to receive the first audio data using the receiving signal of the operating frequency band, thereby improving the efficiency and quality of transmitting the first audio data.

[0076] In some embodiments, the operating frequency band of the first wireless communication connection is from 2.4 GHz to 5 GHz.

[0077] In some optional embodiments, the operating frequency band of the first wireless communication connection can be a high-frequency band of 2.4 GHz, such as 2.470-2.483 GHz. In this operating frequency band, the first wireless communication connection can avoid frequency bands where other networks operate densely, such as Wi-Fi (Wireless Fidelity), to avoid interference with other networks, thereby reducing interference to the first wireless communication connection and improving the efficiency and quality of the wireless acquisition device 21 transmitting the first audio data to the mobile terminal 22.

[0078] In some embodiments, the mobile terminal 22 simultaneously acquires audio data from no more than four wireless acquisition devices 21 via wireless communication connection.

[0079] In this embodiment, the mobile terminal 22 can simultaneously receive audio data transmitted by multiple wireless acquisition devices 21 via wireless communication connection. Due to bandwidth limitations, the mobile terminal 22 is restricted from simultaneously acquiring audio data from no more than four wireless acquisition devices 21 via wireless communication connection to avoid bandwidth overload which would reduce the efficiency and quality of the mobile terminal 22 receiving audio data.

[0080] In some embodiments, the channel priority of the first wireless communication connection is higher than that of the second wireless communication connection; when the channel bandwidth resources cannot simultaneously meet the channel bandwidth requirements of the first wireless communication connection and the second wireless communication connection, the mobile terminal 22 prioritizes allocating bandwidth resources to the channel occupied by the first wireless communication connection.

[0081] In this embodiment, due to bandwidth resource limitations, priority is given to ensuring the communication quality and latency requirements of the first wireless communication connection. That is, the channel priority of the first wireless communication connection is higher than that of the second wireless communication connection. When the channel bandwidth resources cannot simultaneously meet the channel bandwidth requirements of the first wireless communication connection and the second wireless communication connection, bandwidth resources are allocated to the channel occupied by the first wireless communication connection first. This allows the mobile terminal 22 to receive the first audio and video data collected by the wireless acquisition device 21 efficiently and with high quality. After the mobile terminal 22 receives the first audio and video data, the bandwidth resources required by the second wireless communication connection are further considered. This ensures the communication efficiency and quality of both stages and avoids the efficiency and quality of the first wireless communication connection being compromised, which in turn affects the efficiency and quality of the second wireless communication connection.

[0082] In some embodiments, the first wireless communication module 222 is further configured to determine the Quality of Service (QoS) parameters based on the network status information of the first wireless communication connection, and configure the communication parameters of the first wireless communication connection based on the QoS parameters.

[0083] The communication parameters may include the operating frequency band, bandwidth resources, and the number of wireless acquisition devices 21.

[0084] In this embodiment, the first wireless communication module 222 obtains network status information of the first wireless communication connection established with the wireless acquisition device 21, determines the Quality of Service (QoS) parameters of the first wireless communication connection based on the network status information, and configures the communication parameters of the first wireless communication connection based on the QoS parameters. That is, it dynamically adjusts the transmission strategy between the wireless acquisition device 21 and the mobile terminal 22 based on the real-time network status of the first wireless communication connection during communication, so that the transmission quality and efficiency of audio data can reach the optimal level under the constraints of real-time network status.

[0085] In some embodiments, the second audio data is obtained by the processor 224 performing audio processing on the first audio data; and / or, the mobile terminal 22 further includes a memory for storing the video data, the audio and video data being obtained by the processor 224 merging the first audio data and the video data; and / or, the mobile terminal 22 further includes a memory for storing the video data, the audio and video data being obtained by the processor 224 merging the second audio data and the video data.

[0086] In this embodiment, after the first wireless communication module 222 receives the first audio data transmitted by the wireless acquisition device 21, the processor 224 performs audio processing on the first audio data through audio processing and video processing functions to obtain second audio data, thereby improving the sound quality of the second audio data or meeting personalized needs for the first audio data. The processor 224 can also merge the first audio data with video data stored in the memory or captured by the camera module 221 to obtain audio-visual data, and / or merge the second audio data with video data stored in the memory or captured by the camera module 221 to obtain audio-visual data, thus realizing the merging of the first audio data acquired by the wireless acquisition device 21 with the video data stored in the memory or captured by the camera module 221, satisfying personalized needs such as acquiring audio data for monitoring and acquiring audio-visual data for sharing.

[0087] In some embodiments, the processor 224 is configured to synchronize the first audio data with the video data and obtain the audio and video data based on the first audio data and the video data.

[0088] In this embodiment, the processor 224, based on audio and video processing functions, can synchronously process the first audio data and video data, optimizing the audio-visual asynchrony problem between the received first audio data and the video data captured by the camera module 221. After resolving the audio-visual asynchrony problem between the first audio data and video data, audio and video data are obtained based on the first audio data and video data, achieving audio and video synchronization.

[0089] In some embodiments, the processor 224 synchronously processes the first audio data and the video data, and obtains the audio and video data based on the first audio data and the video data, including the following.

[0090] The processor 224 processes the first audio data to obtain the second audio data.

[0091] The audio processing of the first audio data by the processor 224 may include: performing AI noise reduction, EQ adjustment, mixing, sound quality enhancement, adding sound effects, separating human voice and ambient sound, and extracting or synthesizing ambient sound or human voice, etc., to optimize the quality and effect of the first audio data.

[0092] The processor 224 sets a first timestamp for the second audio data that corresponds to the second timestamp of the video data, determines the time difference between the second audio data and the video data based on the first timestamp and the second timestamp, and adjusts the time difference to within a preset threshold range to complete the synchronization processing of the second audio data and the video data.

[0093] The processor 224 adjusts the time difference between the first timestamp of the second audio data and the second timestamp of the video data to reduce the time difference to within a preset threshold range, such as 20ms. When the time difference is within the preset threshold range, the user cannot perceive the audio-visual asynchrony, thus completing the synchronization processing of the second audio data and video data.

[0094] The processor 224 synthesizes the second audio data and the video data to obtain the audio and video data.

[0095] In this embodiment, the processor 224 obtains the second audio data by processing the first audio data. First, it processes the first audio data transmitted by the wireless acquisition device 21 to obtain second audio data that meets user needs and requirements. Then, it sets a first timestamp on the second audio data corresponding to the second timestamp of the video data. Based on the first and second timestamps, it determines the time difference between the second audio data and the video data, adjusting the time difference to within a preset threshold range to complete the synchronization processing of the second audio data and the video data, thus resolving the audio-visual asynchrony problem between the second audio data and the video data. Finally, it synthesizes the second audio data and the video data to obtain the audio-visual data, thereby obtaining synchronized audio-visual data for subsequent rapid sharing or creation.

[0096] Figure 5 A schematic diagram of the structure of a wireless data acquisition device provided in an embodiment of this application is shown below. Figure 5 The wireless acquisition device 21 includes an audio acquisition module 211 and a wireless transmission module 212.

[0097] The audio acquisition module 211 is used to acquire first audio data; the wireless transmission module 212 is used to transmit the first audio data to the mobile terminal 22 with video recording function through a wireless communication connection.

[0098] In the wireless acquisition device 21 provided in this application embodiment, the audio acquisition module 211 acquires first audio data, and then the wireless transmission module 212 transmits the first audio data to the mobile terminal 22 with video recording function through a wireless communication connection. The mobile terminal 22 serves as the core hub. The wireless transmission module 212 transmits the first audio data to the mobile terminal 22, and the mobile terminal 22 also transmits the audio data acquired by the audio acquisition module 211 and the audio and video data obtained based on the audio data to the wireless monitoring device 23. Thus, by eliminating the wireless audio receiving device, the self-interference problem of the wireless audio receiving device is avoided, preventing the inefficient and high-quality transmission of audio and video data. The mobile terminal 22 realizes the transmission of audio and video data, and its complete communication performance enables efficient and high-quality transmission of audio and video data.

[0099] In some embodiments, the wireless transmission module 212 and the mobile terminal 22 establish a first wireless communication connection for transmitting the first audio data based on the LE Audio protocol.

[0100] Among them, the LE Audio protocol mainly provides low-latency, low-power, efficient, and high-quality wireless communication connections between the wireless transmission module 212 and the mobile terminal 22 at the LC3 encoder or LC3plus encoder, enhanced audio stream (CIS) and broadcast audio function levels.

[0101] In some embodiments, the first wireless communication connection satisfies at least one of the following: (1) The audio acquisition module 211 uses an LC3 encoder or an LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms.

[0102] Among them, the LC3 encoder or LC3plus encoder has the advantages of high efficiency and low power consumption, which can achieve a balance between high sound quality and low power consumption. Furthermore, the audio acquisition module 211 can encode with a frame duration of less than or equal to 5ms, which can reduce latency and improve retransmission efficiency. Therefore, using the LC3 encoder or LC3plus encoder to obtain the first audio data with a frame duration of less than or equal to 5ms can reduce the power consumption of the wireless acquisition device 21 and the latency of the first wireless communication connection, thereby reducing the overall power consumption of the wireless audio and video system and the latency of the entire link.

[0103] (2) The wireless transmission module 212 sends the first audio data to the mobile terminal 22 through the CIS mode or the low latency LL mode based on the LE Audio protocol.

[0104] Among them, the CIS mode or low-latency LL mode of the LE Audio protocol introduces a synchronous isochronous channel and a low-latency configuration, which enables the wireless transmission module 212 and the mobile terminal 22 to accurately synchronize the first audio data, ensuring that the wireless transmission module 212 transmits the first audio data to the mobile terminal 22 on time.

[0105] In this embodiment, the wireless transmission module 212 and the mobile terminal 22 establish a first wireless communication connection based on the LE Audio protocol for transmitting the first audio data, satisfying (1) the audio acquisition module 211 uses an LC3 encoder or an LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms, so as to ensure that the sound quality of the first audio data transmitted to the mobile terminal 22 is improved while reducing power consumption and latency. and / or (2) the wireless transmission module 212 sends the first audio data to the mobile terminal 22 through the CIS mode or low latency LL mode based on the LE Audio protocol, so as to reduce the latency of transmitting the first audio data and maximize the timeliness of the wireless transmission module 212 in synchronizing the first audio data acquired by the audio acquisition module 211 to the mobile terminal 22.

[0106] In some embodiments, the operating frequency band of the first wireless communication connection is from 2.4 GHz to 5 GHz.

[0107] In some optional embodiments, the operating frequency band of the first wireless communication connection can be a high-frequency band of 2.4 GHz, such as 2.470-2.483 GHz. In this operating frequency band, the first wireless communication connection can avoid frequency bands where other networks operate densely, such as Wi-Fi (Wireless Fidelity), thus avoiding interference with other networks, thereby reducing interference to the first wireless communication connection and improving the efficiency and quality of the wireless transmission module 212 transmitting the first audio data to the mobile terminal 22.

[0108] Figure 6 This paper shows a schematic diagram of the structure of a wireless audio and video component provided in an embodiment of this application. See also: Figure 6 The wireless audio / video component 600 includes at least one wireless monitoring device 61 and a wireless acquisition device 62 as described in the above embodiments. The wireless acquisition device 62 is independently configured from the at least one wireless monitoring device 61.

[0109] In the wireless audio and video component 600 provided in this application embodiment, audio data is collected by a wireless acquisition device 62 and received and output by at least one wireless monitoring device 61. The audio data collected by the wireless acquisition device 62 is received by a mobile terminal that has established a first wireless communication connection with it, and then transmitted by the mobile terminal to the wireless monitoring device 61 based on a second wireless communication connection established with the wireless monitoring device 61. The wireless acquisition device 62 and at least one wireless monitoring device 61 are set independently, with the mobile terminal as the core hub. This avoids the self-interference problem of the wireless audio receiving device, which would prevent the efficient and high-quality transmission of audio and video data. The mobile terminal realizes the transmission of audio and video data from the wireless acquisition device 62 to at least one wireless monitoring device 61. The complete communication performance of the mobile terminal 22 enables the efficient and high-quality transmission of audio and video data.

[0110] In some embodiments, the wireless acquisition device 62 and the mobile terminal establish a first wireless communication connection for transmitting first audio data based on the LE Audio protocol.

[0111] Among them, the LE Audio protocol mainly provides low-latency, low-power, efficient, and high-quality wireless communication connections for wireless acquisition devices 62 and mobile terminals at the LC3 encoder or LC3plus encoder, enhanced audio stream (CIS) and broadcast audio function levels.

[0112] In some embodiments, the first wireless communication connection satisfies at least one of the following: (1) The wireless acquisition device 62 sends the first audio data to the mobile terminal through the CIS mode or the low latency LL mode based on the LE Audio protocol.

[0113] Among them, the CIS mode or low-latency LL mode of the LE Audio protocol introduces a synchronous isochronous channel and a low-latency configuration, which enables the mobile terminal and the wireless acquisition device 62 to accurately synchronize the first audio data, ensuring that the wireless acquisition device 62 transmits the first audio data to the mobile terminal on time.

[0114] (2) The wireless acquisition device 62 uses an LC3 encoder or an LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms.

[0115] The LC3 encoder or LC3plus encoder offers advantages such as high efficiency and low power consumption, achieving a balance between high audio quality and low power consumption. Furthermore, its frame duration encoding of less than or equal to 5ms allows the wireless acquisition device 62 to encode within a shorter time, reducing latency and improving retransmission efficiency. Therefore, using the LC3 encoder or LC3plus encoder with a frame duration of less than or equal to 5ms to obtain the first audio data reduces the power consumption of the wireless acquisition device 21 and the latency of the first wireless communication connection, thereby reducing the overall power consumption of the wireless audio-visual system and the latency of the entire link.

[0116] In this embodiment, the wireless acquisition device 62 in the wireless audio-visual component 600 establishes a first wireless communication connection with the mobile terminal based on the LEAudio protocol for transmitting the first audio data, satisfying (1) the wireless acquisition device 62 sends the first audio data to the mobile terminal through the CIS mode or low-latency LL mode based on the LE Audio protocol, so as to ensure that the sound quality of the first audio data transmitted to the mobile terminal 22 is improved while reducing power consumption and latency. and / or (2) the wireless acquisition device 62 uses an LC3 encoder or an LC3plus encoder to obtain the first audio data by fixing the frame length encoding corresponding to the lowest latency configuration, so as to reduce the latency of transmitting the first audio data and maximize the timely synchronization of the acquired first audio data to the mobile terminal by the wireless acquisition device 62.

[0117] In some embodiments, the operating frequency band of the first wireless communication connection is from 2.4 GHz to 5 GHz.

[0118] In some optional embodiments, the operating frequency band of the first wireless communication connection can be a high-frequency band of 2.4 GHz, such as 2.470-2.483 GHz. In this operating frequency band, the first wireless communication connection can avoid frequency bands where other networks operate densely, such as Wi-Fi (Wireless Fidelity), thus avoiding interference with other networks, thereby reducing interference to the first wireless communication connection and improving the efficiency and quality of the wireless acquisition device 62 transmitting the first audio data to the mobile terminal.

[0119] In some embodiments, the wireless monitoring device 61 is used to receive, in a broadcast manner, first audio data transmitted by the mobile terminal, second audio data obtained based on the first audio data, or audio-visual data obtained based on the first audio data.

[0120] In this embodiment of the application, at least one wireless listening device 61 receives first audio data transmitted by a mobile terminal, second audio data obtained based on the first audio data, or audio and video data obtained based on the first audio data via broadcast. Each wireless listening device 61 does not need to establish a one-to-one pairing connection with the mobile terminal, which can reduce the latency of data transmission and meet the needs of teams or multiple users to conduct listening through the wireless listening device 61.

[0121] In some embodiments, the second audio data is obtained by the mobile terminal through audio processing of the first audio data; and / or, the audio and video data is obtained by the mobile terminal by merging the first audio data and the video data stored or captured by the mobile terminal; and / or, the audio and video data is obtained by the mobile terminal by merging the second audio data and the video data stored or captured by the mobile terminal.

[0122] In this embodiment, after the wireless acquisition device 62 transmits the first audio data to the mobile terminal, the mobile terminal uses audio processing and video processing functions to process the first audio data to obtain second audio data, thereby improving the sound quality of the second audio data or meeting personalized needs for the first audio data. The mobile terminal can also merge the first audio data with video data stored or captured by the mobile terminal to obtain audio-visual data, and / or merge the second audio data with video data stored or captured by the mobile terminal to obtain audio-visual data, thus achieving the merging of the first audio data acquired by the wireless acquisition device 62 with the video data stored or captured by the mobile terminal, and meeting personalized needs for transmitting audio data and audio-visual data to the wireless monitoring device 61 for monitoring or sharing.

[0123] Figure 7 This paper illustrates a schematic diagram of another wireless audio-visual system provided in an embodiment of this application. See also... Figure 7 The wireless audio-visual system 700 may include a wireless acquisition device 71, a mobile phone 72, and a wireless monitoring headset 73. The wireless acquisition device 71 includes two microphones 711 and 712 for receiving audio data, a main control module 713, and two wireless modules 714 and 715. The mobile phone 72 includes two wireless modules 721 and 722, an audio processing module 723, a neural network processing module 724, and a main chip module 725. The wireless monitoring headset 73 includes a wireless module 731 and a main control module 732.

[0124] The wireless acquisition device 71 and the mobile phone 72 establish a first wireless communication connection via the standard Bluetooth protocol, which uses LE Audio. The first wireless communication connection is configured with CIS mode or low-latency LL mode to significantly reduce latency and bandwidth usage. It also uses an LC3 encoder or LC3plus encoder to encode audio data with a frame duration of less than or equal to 5ms, achieving a balance between high sound quality and low power consumption by reducing the bit rate to a lower level without sacrificing too much sound quality. The mobile phone 72 and the wireless monitoring headphones 73 establish a second wireless communication connection via the standard Bluetooth protocol. The mobile phone 72 broadcasts the received audio data or audio / video data obtained based on the audio data to at least one wireless monitoring headphone 73.

[0125] The wireless audio / video system 700 enables AFH (Active Frequency Control), dynamic power control, and network status monitoring. Specifically, it acquires the transmit power of the wireless acquisition device 71 and / or the channel quality of the first wireless communication connection, determines the operating frequency band based on the transmit power and / or the channel quality, and uses AFH to switch the first wireless communication connection to the determined operating frequency band. During audio data transmission, it avoids dense Wi-Fi channels and preferentially uses the 2.4GHz high-frequency band (2.470-2.483GHz).

[0126] Among them, the wireless acquisition device 71 uses an LC3 encoder or an LC3plus encoder for encoding, which can maintain high sound quality at 128kbps and reduce bandwidth usage by 33%.

[0127] Among them, the number of wireless acquisition devices 71 that simultaneously transmit audio data to the mobile terminal via wireless communication connection is less than or equal to 4, in order to avoid bandwidth overload.

[0128] In this system, the wireless audio-visual system 700 configures the channel priority of the first wireless communication connection between the wireless acquisition device 71 and the mobile phone 72 to be higher than the channel priority of the second wireless communication connection between the mobile phone 72 and the wireless monitoring headset 73. The main chip module 725 of the mobile phone 72 can also obtain network status information of the first wireless communication connection, determine the Quality of Service (QoS) parameters based on the network status information, and configure the communication parameters of the first wireless communication connection between the wireless acquisition device 71 and the mobile phone 72 based on the QoS parameters. That is, the transmission strategy between the wireless acquisition device 71 and the mobile phone 72 is dynamically adjusted based on the real-time network status during the communication process of the first wireless communication connection, so that the transmission quality and efficiency of audio data can reach the optimal level under the constraints of real-time network conditions.

[0129] In other embodiments, the wireless acquisition device in the wireless audio and video system can be integrated into the mobile phone. That is, the mobile phone can simultaneously turn on the microphone and camera to record audio and video data. After acquiring the audio and video data, it can directly perform noise reduction, extraction, synchronization, synthesis, sharing and other processing on both.

[0130] Figure 8 This illustration shows a flowchart of a signaling interaction method for a wireless audio / video system according to an embodiment of this application, which can be applied to, for example... Figure 7 The wireless audio / video system 700 shown is described in the following document. Figure 8 The signaling interaction method may include the following steps.

[0131] Step 801: The wireless acquisition device and the mobile phone are paired via the standard Bluetooth protocol to establish a wireless communication connection.

[0132] This standard Bluetooth protocol includes LE Audio.

[0133] Step 802: The mobile phone and the wireless monitoring headphones are paired via the standard Bluetooth protocol to establish a wireless communication connection.

[0134] The mobile phone transmits received audio data or audio / video data obtained based on the audio data to at least one wireless monitoring device via a broadcast (Auracast).

[0135] Step 803a: The wireless acquisition device acquires audio data and encodes, compresses, and modulates the audio data using an LC3 encoder or an LC3plus encoder before transmitting it to the mobile phone.

[0136] The wireless acquisition device also includes a backup storage of the bass track after acquiring the audio data.

[0137] Step 803b: Record video data on the phone and add a timestamp.

[0138] Step 804: The mobile phone receives audio data, demodulates, decompresses, converts digital to analog, decodes, and performs noise reduction on the audio data before marking it with a timestamp.

[0139] Among them, see Figure 9a , Figure 9a The illustration shows a time difference between audio data and video data provided in an embodiment of this application. The mobile phone generally acquires video data first, and then receives audio data at time N. Based on the delay of standard wireless communication protocols, such as the standard Bluetooth protocol, it can be concluded that the time difference between video data and audio data in this embodiment of the application uses the standard Bluetooth protocol and is constrained as follows: 20 < (MN) < 40 ms.

[0140] Step 805: The mobile phone transmits the processed audio data to the wireless monitoring headphones via broadcast.

[0141] Step 806: The wireless monitoring headphones receive and output the processed audio data.

[0142] In this embodiment, the audio data transmission link, the protocol and encoding used, and the data flow are as follows: (1) Transmission link from wireless acquisition device to mobile phone: The protocol and encoding used are LE Audio and LC3plus encoder encoding; the data flow is audio data connected to the mobile phone via a connected isochronous stream (CIS), and the mobile phone performs multi-dimensional processing on the audio data.

[0143] (2) Transmission link from mobile phone to wireless monitoring headphones: The protocol and encoding used are Auracast and LC3plus encoders; the data flow is audio data in Broadcast Isochronous Streams (BIS) from mobile phone to wireless monitoring headphones, and the wireless monitoring headphones decode and output the audio data.

[0144] Both links utilize the public standard Bluetooth protocol combined with the efficient compression features of the LC3plus encoder, ensuring compatibility while reducing bandwidth requirements and maintaining sound quality. Auracast supports one-to-many audio data distribution, enabling mobile phones to transmit audio data to multiple wireless monitoring headphones simultaneously, meeting the needs of team collaboration or multi-user monitoring scenarios.

[0145] Following step 804 described above, the method further includes the following steps.

[0146] Step 807: The mobile phone calculates the time difference between the timestamp corresponding to the audio data and the timestamp corresponding to the video data, performs synchronization verification on the time difference, and achieves audio-visual synchronization.

[0147] Among them, see Figure 9b , Figure 9b This illustration shows another time difference between audio and video data provided in an embodiment of this application. Based on the synchronization verification of the time difference in this application, the time difference can be reduced to: 0 ≤ (MN) < 20ms. Furthermore, see... Figure 9c , Figure 9cThis diagram illustrates a comparison between the related technologies provided in this application and the latency issues in this application's embodiments. In the related technologies, the wireless audio system's wireless acquisition device needs to transmit audio data to a wireless audio receiving device, which then transmits it to the mobile phone via a data cable and physical interface. This results in an added latency in the audio data received by the mobile phone, which is not optimized. In this application's embodiment, however, the wireless acquisition device directly transmits the audio data to the mobile phone, where the phone timestamps the audio data and calculates the time difference with the timestamp of the acquired video data for synchronization processing. This optimizes the latency, thereby reducing the problem of audio-visual asynchrony.

[0148] Step 808: The mobile phone will perform personalized synthesis of the synchronized audio and video data to generate audio and video data.

[0149] In this embodiment, the mobile phone acts as the core hub, receiving, processing, and broadcasting the audio data collected by the wireless acquisition device to at least one wireless monitoring headset. This simplifies the audio-visual system, eliminates the physical interface limitations of connecting wireless audio receiving devices, avoids the wireless audio receiving device occupying the mobile phone's charging port, supports receiving audio data while charging, and reduces latency and optimizes the problem of audio-visual asynchrony by directly processing and distributing audio and video on the mobile phone.

[0150] Figure 10 This illustration shows a flowchart of an audio / video data transmission method provided in an embodiment of this application. This method can be applied to, for example... Figure 2 or Figure 3 The wireless audio and video system provided in the embodiment is described in [reference]. Figure 10 The method may include the following steps.

[0151] Step 1002: Collect video and audio data.

[0152] The video data is recorded directly by mobile terminals in the wireless audio-visual system, such as mobile phones and cameras, while the audio data is collected and transmitted to the mobile terminals by wireless acquisition devices in the wireless audio-visual system, such as wireless microphones and lavalier microphones.

[0153] Step 1004: Process the audio data and synchronize the processed audio data with the video data.

[0154] The collected audio and video data reside on mobile terminals, such as smartphones, where audio data processing and synchronous processing of audio and video data are handled. Modern smartphones possess powerful audio and video processing capabilities, including Digital Signal Processing (DSP) and Artificial Intelligence (AI) capabilities. These capabilities enable AI noise reduction, equalizer (EQ) adjustment, mixing, sound quality enhancement, sound effect addition, separation of vocals and ambient sound, extraction or synthesis of ambient or vocal sounds, and synchronized audio and video data. This efficiently improves sound quality and addresses audio-visual asynchrony issues. Furthermore, with the smartphone's microphone also enabled, personalized processing of audio tracks from different locations—the wireless acquisition device and the smartphone—can be achieved. For example, audio tracks from different locations can be mixed or overlaid to output a stereo or panoramic video with clear vocals and a sense of spatial environment. Compared to the lightweight chips used in wireless audio receiving devices in related technologies, this technology can achieve more powerful audio and video processing capabilities to meet users' personalized needs, while also reducing the complexity and cost of wireless audio and video systems.

[0155] Step 1006: Output the processed audio data.

[0156] The wireless monitoring device of the wireless audio-visual system outputs the processed audio data for users to listen to.

[0157] In some embodiments, step 1004 above, which synchronizes the processed audio data and the video data, may include: stamping the processed audio data with a first timestamp; determining the time difference between the audio data and the video data based on the first timestamp and the second timestamp corresponding to the video data; and adjusting the time difference to within a preset threshold range.

[0158] The wireless audio and video system timestamps the processed audio data, calculates the time difference between the timestamps and the timestamps of the acquired video data, adjusts the time difference to keep it within a preset threshold range, and achieves synchronous processing of audio and video data.

[0159] In some embodiments, after synchronizing the processed audio data and video data in step 1004 above, the method further includes: synthesizing the synchronized audio data and video data to generate audio-visual data.

[0160] The wireless audio and video system synthesizes synchronized audio and video data to generate audio and video data, which includes sound and picture. This allows users to efficiently share the audio and video data, meeting their needs for creating and sharing vlogs.

[0161] In some embodiments, the above-described audio and video data transmission method may further include: acquiring network status information of the wireless communication connection between the wireless acquisition device and the mobile terminal; determining Quality of Service (QoS) parameters based on the network status information; and configuring communication parameters of the wireless communication connection based on the QoS parameters.

[0162] The wireless audio and video system can also obtain network status information of the wireless communication connection between the wireless acquisition device and the mobile terminal, determine the Quality of Service (QoS) parameters based on the network status information, and configure the communication parameters for transmitting audio data based on the QoS parameters. In other words, it can dynamically adjust the audio data transmission strategy based on the real-time network status of the wireless communication connection between the wireless acquisition device and the mobile terminal, so that the audio data transmission quality can reach the optimal level even under the constraints of real-time network conditions.

[0163] The wireless audio and video system provided in this application eliminates the wireless audio receiving device in related technologies, reconstructs the wireless audio and video system, and uses a mobile terminal, such as a mobile phone, as the core hub to receive, process, and distribute audio data collected by the wireless acquisition device to the wireless monitoring device in the form of broadcast. This optimizes the problems of continuous wireless communication between the wireless acquisition device and the mobile terminal, as well as interference, network resource occupation, and limited users of the wireless monitoring device in the wireless communication connection between the mobile terminal and the wireless monitoring device. It can also perform audio-visual synchronization processing and synthesis processing on audio and video data, making full use of the functions of the mobile phone and optimizing the problem of audio-visual asynchrony.

[0164] This application also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the audio and video data transmission method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0165] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0167] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A wireless audio and video system, characterized in that, include: Wireless acquisition devices, mobile terminals with video recording capabilities, and wireless listening devices; The wireless acquisition device is used to acquire first audio data and transmit the first audio data to the mobile terminal via a wireless communication connection. The mobile terminal is used to transmit the received first audio data, the second audio data obtained based on the first audio data, or the audio and video data obtained based on the first audio data to the wireless monitoring device via a wireless communication connection.

2. The wireless audio and video system according to claim 1, characterized in that, The wireless acquisition device and the mobile terminal establish a first wireless communication connection based on the LE Audio protocol for transmitting the first audio data.

3. The wireless audio and video system according to claim 2, characterized in that, The first wireless communication connection satisfies at least one of the following: The wireless acquisition device sends the first audio data to the mobile terminal in CIS mode or low-latency LL mode based on the LE Audio protocol. The wireless acquisition device uses an LC3 encoder or an LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms. When transmitting the first audio data through the first wireless communication connection, the sound enhancement function or noise reduction function of the mobile terminal is turned off.

4. The wireless audio and video system according to claim 1, characterized in that, The transmission of the first audio data to the mobile terminal via a wireless communication connection includes: The wireless acquisition device and the mobile terminal establish a first wireless communication connection for transmitting the first audio data. Obtain the transmit power of the wireless acquisition device and / or the channel quality of the first wireless communication connection, and determine the operating frequency band based on the transmit power and / or the channel quality of the first wireless communication connection; The first wireless communication connection is switched to the operating frequency band using an adaptive frequency hopping (AFH) method.

5. The wireless audio and video system according to claim 1, characterized in that, The wireless acquisition device and the mobile terminal establish a first wireless communication connection for transmitting the first audio data; the mobile terminal and the wireless monitoring device establish a second wireless communication connection. The channel priority of the first wireless communication connection is higher than that of the second wireless communication connection; if the channel bandwidth resources cannot simultaneously meet the channel bandwidth requirements of the first wireless communication connection and the second wireless communication connection, the mobile terminal shall prioritize allocating bandwidth resources to the channel occupied by the first wireless communication connection.

6. The wireless audio and video system according to claim 1, characterized in that, The mobile terminal is used to transmit, in a broadcast manner, the received first audio data, the second audio data obtained based on the first audio data, or the audio and video data obtained based on the first audio data to at least one of the wireless monitoring devices.

7. The wireless audio and video system according to claim 1, characterized in that, The second audio data is obtained by the mobile terminal through audio processing of the first audio data; and / or, the audio and video data is obtained by the mobile terminal by merging the first audio data and the video data stored or captured by the mobile terminal; and / or, the audio and video data is obtained by the mobile terminal by merging the second audio data and the video data stored or captured by the mobile terminal.

8. The wireless audio and video system according to claim 1, characterized in that, The mobile terminal is also used to acquire video data, and the mobile terminal is used to synchronize the first audio data and the video data, and obtain the audio and video data based on the first audio data and the video data.

9. The wireless audio and video system according to claim 8, characterized in that, The mobile terminal synchronizes the first audio data and the video data, and obtains the audio and video data based on the first audio data and the video data, including: The mobile terminal processes the first audio data to obtain the second audio data; The mobile terminal sets a first timestamp corresponding to the second timestamp of the video data for the second audio data, determines the time difference between the second audio data and the video data based on the first timestamp and the second timestamp, and adjusts the time difference to within a preset threshold range to complete the synchronization processing of the second audio data and the video data. The mobile terminal synthesizes the second audio data and the video data to obtain the audio and video data.

10. A mobile terminal, characterized in that, include: Camera module, used to record video data; The first wireless communication module is used to establish a first wireless communication connection with an external wireless acquisition device to obtain first audio data; The second wireless communication module is used to establish a second wireless communication connection with the wireless monitoring device to transmit the received first audio data, the second audio data obtained based on the first audio data, or the audio and video data obtained based on the first audio data. A processor is used to process the first audio data to obtain the second audio data or the audio / video data.

11. The mobile terminal according to claim 10, characterized in that, The first wireless communication module and the wireless acquisition device establish the first wireless communication connection based on the LE Audio protocol.

12. The mobile terminal according to claim 10, characterized in that, The first wireless communication module satisfies at least one of the following: The mobile terminal receives the first audio data via CIS mode or low-latency LL mode based on the LE Audio protocol. The first audio data is generated based on LC3 encoding or LC3plus encoding. When transmitting the first audio data through the first wireless communication connection, the sound enhancement function or noise reduction function of the mobile terminal is turned off.

13. The mobile terminal according to claim 10, characterized in that, The second wireless communication module is used to transmit the received first audio data, the second audio data obtained based on the first audio data, or the audio and video data obtained based on the first audio data to the wireless monitoring device in a broadcast manner.

14. The mobile terminal according to claim 10, characterized in that, The first wireless communication module is used for: Establish a first wireless communication connection with the wireless acquisition device for transmitting the first audio data; Obtain the transmit power of the wireless acquisition device and / or the channel quality of the first wireless communication connection, and determine the operating frequency band based on the transmit power and / or the channel quality of the first wireless communication connection; The first wireless communication connection is switched to the operating frequency band using an adaptive frequency hopping (AFH) method.

15. The mobile terminal according to claim 10, characterized in that, The channel priority of the first wireless communication connection is higher than that of the second wireless communication connection; if the channel bandwidth resources cannot simultaneously meet the channel bandwidth requirements of the first wireless communication connection and the second wireless communication connection, the mobile terminal shall prioritize allocating bandwidth resources to the channel occupied by the first wireless communication connection.

16. The mobile terminal according to claim 10, characterized in that, The second audio data is obtained by the processor performing audio processing on the first audio data; and / or, the mobile terminal further includes a memory for storing the video data, the audio and video data being obtained by the processor merging the first audio data and the video data; and / or, the mobile terminal further includes a memory for storing the video data, the audio and video data being obtained by the processor merging the second audio data and the video data.

17. A wireless data acquisition device, characterized in that, include: An audio acquisition module is used to acquire the first audio data; A wireless transmission module is used to transmit the first audio data to a mobile terminal with video recording capabilities via a wireless communication connection.

18. The wireless data acquisition device according to claim 17, characterized in that, The wireless transmission module and the mobile terminal establish a first wireless communication connection based on the LE Audio protocol for transmitting the first audio data.

19. The wireless data acquisition device according to claim 18, characterized in that, The first wireless communication connection satisfies at least one of the following: The audio acquisition module uses an LC3 encoder or an LC3plus encoder to encode the first audio data with a frame duration of less than or equal to 5ms. The wireless transmission module sends the first audio data to the mobile terminal via CIS mode or low-latency LL mode based on the LE Audio protocol.

20. A wireless audio / video component, characterized in that, include: At least one wireless eavesdropping device and a wireless acquisition device as described in any one of claims 17 to 19, wherein the wireless acquisition device is independently configured from the at least one wireless eavesdropping device.

Images