Multimedia broadcast control method, terminal device, storage medium and computer program product

Abstract

The invention provides a multimedia playing control method, terminal equipment, a storage medium and a computer program product. The method comprises the following steps: scanning a broadcast isochronous stream broadcast to obtain broadcast isochronous stream audio data; performing sound mixing on first audio data sent by a first application and the broadcast isochronous stream audio data to form second audio data; establishing a first Bluetooth connection between the first terminal device and a second terminal device; and sending the second audio data to the second terminal device based on the first Bluetooth connection, so that the second terminal device plays the second audio data. According to the embodiment of the invention, the problem that the Bluetooth earphone cannot simultaneously listen to the Bluetooth low-power-consumption BIS broadcast and the Bluetooth low-power-consumption audio (Audio) can be solved, and the problem that the Bluetooth earphone which does not support the Bluetooth low-power-consumption audio cannot receive the BIS broadcast can also be solved, so that a user can experience the BIS broadcast without replacing the earphone.

Description

Technical Field

[0001] This application relates to the field of wireless communication technologies, and in particular, to a multimedia broadcast control method, a terminal device, a storage medium, and a computer program product. Background Art

[0002] With the continuous evolution of technologies, Bluetooth technology has evolved from the early Bluetooth 1.0 to the Bluetooth 5.2 version. The Bluetooth 5.2 version supports the low energy audio (LE Audio) technology, breaking the limitation that traditional Bluetooth cannot perform point-to-multipoint transmission of synchronous stream data, enabling multiple stream transmissions, and enhancing the Bluetooth audio experience in combination with broadcast audio sharing.

[0003] Low energy audio (LE Audio) supports two communication methods, namely low energy connected isochronous stream (LE-CIS) and low energy broadcast isochronous stream (LE-BIS). When a slave Bluetooth device in a Bluetooth system simultaneously accesses a low energy connected isochronous stream (LE-CIS) communication link and a low energy broadcast isochronous stream (LE-BIS) communication link, transmission conflicts may occur between the low energy connected isochronous stream (LE-CIS) transmission window and the low energy broadcast isochronous stream (LE-BIS) transmission window. A Bluetooth audio playback device cannot simultaneously process multiple types of audio streams from different devices, resulting in data loss. Moreover, if a Bluetooth audio playback device itself does not support low energy audio (LE Audio), it cannot receive and process broadcasts either. Summary of the Invention

[0004] In view of at least one of the above technical problems existing in the prior art, this application is proposed. According to one aspect of this application, a multimedia broadcast control method is provided, which is applied to a first terminal device. The method includes:

[0005] Scanning for broadcast isochronous stream broadcasts to obtain at least one set of broadcast isochronous stream audio data;

[0006] Mixing first audio data sent by a first application and the at least one set of broadcast isochronous stream audio data to form second audio data;

[0007] Establishing a first Bluetooth connection between the first terminal device and a second terminal device;

[0008] Based on the first Bluetooth connection, sending the second audio data to the second terminal device so that the second terminal device plays the second audio data.

[0009] The multimedia playback control method according to the embodiment of the present application scans the Broadcast Isochronous Stream (BIS) broadcast to obtain BIS audio data, mixes the first audio data sent by the first application and the BIS broadcast audio data to form second audio data, and sends the second audio data to the second terminal device through the first Bluetooth connection, so that the second terminal device plays the second audio data. By adopting the technical solution of the embodiment of the present application, the problem that a Bluetooth headset cannot simultaneously listen to the Bluetooth Low Energy (BLE) Broadcast Isochronous Stream (BIS) broadcast and the Bluetooth Low Energy (BLE) audio can be solved, and the problem that a Bluetooth headset that does not support Low Energy Audio (LEA) cannot receive the BIS broadcast can also be solved, enabling users to experience the BIS broadcast without having to replace the headset.

[0010] On the other hand, the embodiment of the present application provides a terminal device, which is used to execute the multimedia playback control method as described above; the terminal device includes a broadcast scanning module, a first application, a mixing module, and a Bluetooth module; wherein,

[0011] The broadcast scanning module is configured to scan the BIS broadcast to obtain BIS audio data, and send the BIS audio data to the mixing module;

[0012] The first application is used to obtain or generate first audio data;

[0013] The mixing module is configured to mix the first audio data sent by the first application and the BIS audio data to form second audio data;

[0014] The Bluetooth module is configured to send the second audio data to a second terminal device, so that the second terminal device plays the second audio data.

[0015] On yet another aspect, the embodiment of the present application provides a storage medium, on which a computer program is stored, and when the computer program is run by a processor, the processor is caused to execute the multimedia playback control method as described above.

[0016] On still another aspect, the embodiment of the present application provides a computer program product, and when the computer program / instructions are executed by a processor, the steps of the multimedia playback control method as described above are implemented.

[0017] The multimedia broadcast control method according to the embodiment of the present application scans the Broadcast Isochronous Stream (BIS) broadcast to obtain the BIS audio data, mixes the first audio data sent by the first application and the BIS audio data to form the second audio data, and sends the second audio data to the second terminal device through the first Bluetooth connection, so that the second terminal device plays the second audio data. By adopting the technical solution of the embodiment of the present application, the problem that a Bluetooth headset cannot simultaneously listen to the Bluetooth Low Energy (BLE) Broadcast Isochronous Stream (BIS) broadcast and the BLE audio can be solved, and the problem that a Bluetooth headset that does not support Low Energy Audio (LEA) cannot receive the BIS broadcast can also be solved, enabling the user to experience the BIS broadcast without replacing the headset. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 A schematic flowchart showing the multimedia broadcast control method according to the embodiment of the present application;

[0020] Figure 2 A schematic flowchart showing step S101 according to the embodiment of the present application;

[0021] Figure 3 A schematic flowchart showing step S102 according to the embodiment of the present application;

[0022] Figure 4 A schematic flowchart showing step S302 according to the embodiment of the present application;

[0023] Figure 5 A schematic flowchart showing step S303 according to the embodiment of the present application;

[0024] Figure 6 A schematic flowchart showing step S305 according to the embodiment of the present application;

[0025] Figure 7 A schematic flowchart showing step S306 according to the embodiment of the present application;

[0026] Figure 8Schematic flowchart showing that when there is a conflict between BIS audio data and first audio data, BIS audio data is preferentially transmitted according to an embodiment of the present application;

[0027] Figure 9 Schematic flowchart showing receiving broadcast isochronous stream data when the bandwidth of BIS audio data is less than a preset bandwidth threshold according to an embodiment of the present application;

[0028] Figure 10 Schematic flowchart showing pausing the transmission of first audio data when scanning a broadcast isochronous stream and resuming the transmission of first audio data after receiving broadcast timed stream audio data according to an embodiment of the present application;

[0029] Figure 11 Schematic flowchart showing a multimedia broadcast control method according to another embodiment of the present application;

[0030] Figure 12 Schematic flowchart showing a multimedia broadcast control method according to yet another embodiment of the present application;

[0031] Figure 13 Schematic diagram of time slot conflict between broadcast isochronous stream (BIS) and connection isochronous stream (CIS) according to an embodiment of the present application;

[0032] Figure 14 Schematic diagram showing the connection of a smart phone to BIS broadcast and a Bluetooth headset according to an embodiment of the present application;

[0033] Figure 15 Schematic block diagram of a terminal device according to an embodiment of the present application. Detailed implementation manners

[0034] To enable those skilled in the art to better understand the technical solutions of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0035] Based on the foregoing at least one technical problem, the present application provides a multimedia playback control method, which is applied to a first terminal device. The method includes: scanning a Broadcast Isochronous Stream (BIS) broadcast based on a broadcast scanning function to obtain BIS audio data; mixing the first audio data sent by a first application and the BIS audio data to form second audio data; establishing a first Bluetooth connection between the first terminal device and a second terminal device; and sending the second audio data to the second terminal device through the first Bluetooth connection so that the second terminal device plays the second audio data. In the multimedia playback control method according to the embodiments of the present application, by scanning a BIS broadcast to obtain BIS audio data, mixing the first audio data sent by a first application and the BIS broadcast audio data to form second audio data, and sending the second audio data to the second terminal device through a first Bluetooth connection so that the second terminal device plays the second audio data, the technical solution of the embodiments of the present application can solve the problem that a Bluetooth headset cannot simultaneously listen to a Bluetooth Low Energy (BLE) BIS broadcast and BLE audio, and can also solve the problem that a Bluetooth headset that does not support Low Energy Audio (LEA) cannot receive a BIS broadcast, enabling a user to experience a BIS broadcast without replacing the headset.

[0036] The multimedia playback control method according to the embodiments of the present application can be applied to a first terminal device. Here, the first terminal device supports the Low Energy Audio (LEA) function. Among them, the first terminal includes at least one of the following terminal devices: a smart phone, a tablet computer, a desktop computer, a laptop computer, a smart watch, a Virtual Reality (VR) glasses, an Augmented Reality (AR) glasses, a wireless vehicle system, a media player (such as an MP3, an MP4, etc.); the second terminal device includes at least a Bluetooth headset, a Bluetooth speaker, etc.

[0037] Figure 1 Fig. shows a schematic flowchart of the multimedia playback control method according to the embodiments of the present application; as Figure 1 shown, the multimedia playback control method 100 according to the embodiments of the present application may include the following steps S101, step S102, step S103, and step S104:

[0038] In step S101, scan a BIS broadcast to obtain at least one set of BIS audio data.

[0039] In an embodiment of the present application, the first terminal device should have the function of scanning the Broadcast Isochronous Stream (BIS) broadcast. When scanning the BIS broadcast, the function of scanning the BIS broadcast can be started by configuring the first terminal device.

[0040] Among them, the BIS audio data at least includes a broadcast period and frequency data.

[0041] Among them, the broadcast period of the BIS audio data refers to the periodic broadcast associated with the BIS, which is used to send isochronous broadcast synchronization information. In Bluetooth 5.2 and higher versions, Isochronous Broadcast is an important communication method, which is implemented based on the defined Broadcast Isochronous Stream (BIS) and Broadcast Isochronous Group (BIG). The BIS is a one-way isochronous broadcast communication method, which is used to transmit data between a transmitter and multiple receivers, and is particularly suitable for the transmission of audio data streams. In order to receive the BIS, the receiving device needs to obtain synchronization information such as the timing, frequency hopping, and access address information of the isochronous broadcast. These synchronization information (BIGInfo) are sent on the periodic broadcast associated with the isochronous broadcast. After the receiving device obtains the BIS synchronization information from the periodic broadcast, it can synchronize the BIS, so as to correctly receive and reconstruct the audio data stream. Therefore, the BIS periodic broadcast plays a crucial role in the isochronous broadcast system, which ensures that the receiving device can accurately synchronize and receive the audio data stream from the transmitter.

[0042] In an example of the present application, as Figure 2 shown, the step of scanning the BIS broadcast in step S101 to obtain at least one set of BIS audio data includes step S201:

[0043] In step S201, at least one set of synchronized BIS audio data is obtained according to the broadcast period and the frequency data.

[0044] In an embodiment of the present application, the first terminal device (such as an audio receiving device such as a smart phone) needs to turn on the Bluetooth function and be in a state where it can receive broadcasts in order to receive BIS audio data. The first terminal device can scan the surrounding periodic broadcasts, obtain the synchronization information of the BIS from them, then establish a synchronized broadcast with the BIS (broadcast source), and start receiving the audio data stream in the BIS.

[0045] It should be noted that the first terminal device of the present application should support the technology of Bluetooth 5.2 or higher versions, because the Broadcast Isochronous Stream (BIS) was introduced in the Bluetooth technology of Bluetooth 5.2 or higher versions.

[0046] Specifically, the process of obtaining synchronized Broadcast Isochronous Stream (BIS) audio data according to the broadcast period and the frequency data is as follows: First, it is necessary to turn on the Bluetooth function of the first terminal device. The first terminal device scans periodic broadcasts, that is, the first terminal device scans the periodic broadcasts sent by surrounding Bluetooth devices. These periodic broadcasts contain synchronization information about BIS broadcasts, such as the period and frequency of the broadcasts. The first terminal device parses the synchronization information of the BIS broadcast from the scanned periodic broadcasts. This information is crucial for the first terminal device because it determines how the device synchronizes with the Broadcast Isochronous Stream (BIS) and receives the audio data stream. Then, the first terminal device establishes synchronization with the Broadcast Isochronous Stream (BIS) and starts receiving the audio data stream. Once the synchronization is established, the first terminal device can start receiving the audio data stream in the Broadcast Isochronous Stream (BIS). The first terminal device can directly play the received audio data stream, or send the received audio data stream to devices such as headphones and speakers for playback, or perform other processing according to the needs of the user.

[0047] In step S102, the first audio data sent by the first application and the at least one group of broadcast isochronous stream audio data are mixed to form second audio data.

[0048] The first application can be installed on the terminal device. For example, the first application can be an application program with a voice function, such as the KuGou Music application program (APP), Tencent Video, iQIYI and other application programs with audio playback functions. The first audio data can be the audio data in the music application program, or the audio data in the audio-visual file, etc. It can also be the voice data during a call, recording, etc.

[0049] In an embodiment of the present application, as Figure 3 shown, the mixing of the first audio data sent by the first application and the at least one group of broadcast isochronous stream audio data in step S102 to form second audio data includes the following steps S301, step S302, step S303, step S304, step S305 and step S306:

[0050] In step S301, the at least one group of Broadcast Isochronous Stream (BIS) audio data and the first audio data are respectively converted into first Pulse Code Modulation (PCM) audio data and second Pulse Code Modulation (PCM) audio data;

[0051] In step S302, align the sampling rates of the first pulse code modulation (PCM) audio data and the second PCM audio data;

[0052] In step S303, align the channels of the first PCM audio data and the second PCM audio data;

[0053] In step S304, adjust the volume ratio of the first PCM audio data and the second PCM audio data based on a preset volume weight coefficient;

[0054] In step S305, mix the first PCM audio data and the second PCM audio data frame by frame to form third audio data;

[0055] In step S306, perform normalization processing on the third audio data to form the second audio data in a preset format.

[0056] Wherein, the preset format includes any one of the following: Advanced Audio Coding (AAC) format, Sub-band Coding (SBC) format, Low Complexity Communication Codec (LC3) format.

[0057] Converting Broadcast Isochronous Stream (BIS) audio data and first audio data into Pulse Code Modulation (PCM) means converting this digital audio stream of BIS audio data into audio data in PCM coding format.

[0058] Pulse Code Modulation (PCM) is a technology that converts an analog signal (such as an audio signal) into a digital pulse sequence, enabling the signal to be transmitted through a digital channel. The PCM coding process mainly includes three steps: (1) Sampling, under the condition of satisfying the sampling theorem, taking the sampling values of the analog signal at fixed time intervals, converting the continuous-time analog signal into a discrete-time continuous-amplitude sampling signal; (2) Quantization, converting the discrete-time continuous-amplitude sampling signal obtained by sampling into a discrete-time discrete-amplitude digital signal, that is, converting the level amplitude value obtained by sampling into a corresponding digital value according to a certain grading standard; (3) Coding, the process of encoding the quantized signal into a binary code group.

[0059] Since the PCM audio data has higher flexibility and compatibility, after converting the broadcast isochronous stream (BIS) audio data and the first audio data into first pulse code modulation (PCM) audio data and second pulse code modulation (PCM) audio data respectively, they can be applied to various audio processing, transmission and storage scenarios.

[0060] In one example, as Figure 4 shown, aligning the sampling rates of the first pulse code modulation audio data and the second pulse code modulation audio data in step S302 includes step S401:

[0061] In step S401, when the sampling rates of the first pulse code modulation audio data and the second pulse code modulation audio data are different, the sampling rates of the two are matched by a preset resampling algorithm so that the sampling rates of the first pulse code modulation audio data and the second pulse code modulation audio data are the same.

[0062] The sampling rate in the embodiments of the present application refers to the number of samples collected per second, which determines the clarity and sound quality of the audio. Different audio sources may use different sampling rates for recording or transmission. For example, the first pulse code modulation audio data corresponding to the broadcast isochronous stream (BIS) audio data may use one sampling rate, while the second pulse code modulation audio data corresponding to the first audio data may use another sampling rate. When these two audio data need to be merged or played simultaneously, if their sampling rates are inconsistent, it will cause audio distortion or out-of-sync. To avoid this situation, it is necessary to use a resampling algorithm to match the sampling rates of the two, that is, to adjust the sampling rates of the two to be the same to ensure that the audio data can be smoothly merged, processed or played. The resampling algorithm is a statistical method for changing the audio sampling rate, which can adapt to the new sampling rate by recombining or replacing the old observed values. During the resampling process, the algorithm will consider the characteristics of the original audio data and the requirements of the target sampling rate to ensure that the converted audio data can achieve the best results in terms of sound quality and synchronization. Therefore, when the sampling rates of the first pulse code modulation audio data corresponding to the broadcast isochronous stream (BIS) audio data and the second pulse code modulation audio data corresponding to the first audio data are different, matching through the resampling algorithm is a key step to ensure that the audio data can be smoothly merged, processed or played.

[0063] In one example, as Figure 5 shown, aligning the channels of the first pulse code modulation audio data and the second pulse code modulation audio data in step S303 includes step S501 and step S502:

[0064] In step S501, detect whether the number of channels of the first pulse-code modulation (PCM) audio data and the second PCM audio data is the same;

[0065] In step S502, if the number of channels of the first PCM audio data and the second PCM audio data is different, convert the number of channels of both to the same number of channels.

[0066] Since both the broadcast isochronous stream audio data and the first audio data may be mono, stereo, etc., when the number of channels of the two is different, mixing them may cause sound distortion, etc. Therefore, it is necessary to first convert the number of channels of the two so that they have the same number of channels.

[0067] For example, in Bluetooth Low Energy (BLE) audio transmission, channel alignment of the first pulse-code modulation (PCM) audio data and the second PCM audio data can be achieved by using the Connection Isochronous Stream (CIS) technology. The Connection Isochronous Stream (CIS) is a synchronous data transmission mechanism based on a Bluetooth connection, which allows the transmission of synchronous data streams between connected devices. In BLE audio transmission, the Connection Isochronous Stream (CIS) can be used to ensure the synchronization of multiple audio data streams, thereby achieving the alignment of different PCM audio data channels. Specifically, when two devices establish a connection via BLE, they can negotiate to establish one or more Connection Isochronous Streams (CIS) for transmitting synchronous audio data streams. These Connection Isochronous Streams (CIS) will be based on a time synchronization mechanism to ensure that the audio data is transmitted within a specified time, thus achieving the alignment of different PCM audio data channels.

[0068] For another example, it is also possible to use the synchronous channel transmission strategy in the LE Audio feature in Bluetooth Low Energy (BLE) version 5.2 to enable synchronous transmission and channel alignment of audio data.

[0069] In one example, as Figure 6 shown, the step of mixing the first PCM audio data and the second PCM audio data frame by frame in step S305 to form the third audio data includes step S601:

[0070] In step S601, an equal number of sampling points are respectively collected from the first PCM audio data and the second PCM audio data, and each sampling point is superimposed according to a preset sampling point weight to form the third audio data.

[0071] For example, in Bluetooth Low Energy (BLE) audio transmission, to achieve frame-by-frame mixing of the first Pulse Code Modulation (PCM) audio data and the second PCM audio data, synchronous transmission can be performed through the CIS (Connected Isochronous Stream) mechanism, and mixing processing can be carried out at the receiving end. Specifically, when two Pulse Code Modulation (PCM) audio data streams need to be transmitted and mixed through Bluetooth Low Energy (BLE), they can be encapsulated in their respective Connected Isochronous Streams (CIS) for synchronous transmission. During the transmission process, each Connected Isochronous Stream (CIS) will carry a separate isochronous synchronous data stream, and these Connected Isochronous Streams (CIS) can be synchronously received and processed at the receiving end. Since the CIS provides a time synchronization mechanism, it can ensure the frame alignment of the two Pulse Code Modulation (PCM) audio data streams during the transmission process, thus providing a basis for the mixing process. At the receiving end, once the two Pulse Code Modulation (PCM) audio data streams are synchronously received, frame-by-frame mixing processing can be carried out. The specific algorithms and parameters of the mixing process can be adjusted according to actual requirements to achieve the desired audio effects. Combining with the embodiments of the present application, the first terminal device can still be used as the receiving end after mixing. After the first terminal device receives the mixed third audio data, the third audio data is then processed subsequently to form the second audio data.

[0072] In one example, as Figure 7 shown, the step of normalizing the third audio data to form the second audio data in a preset format in step S306 includes step S701:

[0073] In step S701, when the sampling value of the third audio data exceeds the preset sampling value range, normalization processing is performed on each sampling point of the third audio data.

[0074] In addition, the embodiments of the present application can set the preset volume weight coefficients of the Broadcast Isochronous Stream (BIS) audio data and the first audio data according to user settings or default values to adjust the volume ratio of the Broadcast Isochronous Stream (BIS) audio data and the first audio data. So that the playback volume of the Broadcast Isochronous Stream (BIS) can be kept clear enough, and the first audio data can also maintain a certain volume. The multimedia playback control method of the embodiments of the present invention realizes the function of playing different audio contents simultaneously, enables users to hear important audio, improves the practicability, prevents the user from being unable to clearly hear important contents due to audio mixing and simultaneous playback of different audios, which affects the user experience. At the same time, without increasing the cost of peripheral devices, only by adding a data control channel and control logic in the system design, the volume regulation can be achieved.

[0075] In step S103, establish a first Bluetooth connection between the first terminal device and the second terminal device.

[0076] Wherein, the first Bluetooth connection includes a connection isochronous stream (CIS) Bluetooth connection or an advanced audio distribution profile (A2DP) Bluetooth connection.

[0077] Generally, a connected isochronous stream (CIS) is a connection technology introduced in the Bluetooth Core Specification version 5.2. It is mainly used to establish a data stream transmission channel with a fixed time interval between the sending and receiving devices on the basis of an established low-power Bluetooth asynchronous connectionless link (ACL) connection. This technology is very suitable for transmitting audio streams between a single audio source and a single receiver, and can provide a more stable and synchronous audio transmission effect. In LE Audio, the connected isochronous stream (CIS) technology is widely used in the connection of devices such as true wireless stereo (TWS) Bluetooth headsets, and can establish synchronous data channels between the master device and the left and right ears respectively, effectively solving problems such as high latency, asynchronous power consumption of both ears, and audio distortion in audio transmission.

[0078] The advanced audio distribution profile (A2DP) is a specification widely used in classic Bluetooth audio transmission, and it is mainly used for audio stream transmission. Through an advanced audio distribution profile (A2DP) connection, users can wirelessly transmit audio content such as music on their mobile phones to Bluetooth headsets or other audio devices.

[0079] For example, if the first terminal device is a smart phone and the second terminal device is a true wireless stereo (TWS) headset, a connection isochronous stream (CIS) Bluetooth connection or an advanced audio distribution profile (A2DP) Bluetooth connection can be established between the smart phone and the true wireless stereo Bluetooth headset to transmit audio data.

[0080] In step S104, send the second audio data to the second terminal device through the first Bluetooth connection, so that the second terminal device plays the second audio data.

[0081] In traditional technologies, the Bluetooth specification itself has certain limitations on device connection and audio playback. When designing Bluetooth headsets, they usually need to be developed and certified according to the Bluetooth specification to ensure their compatibility and stability. In the Bluetooth specification, a headset may only be able to connect to one device and play audio simultaneously, or even if it supports multi-point connection, it may not be able to handle multiple types of audio streams from different devices at the same time due to technical or design reasons, such as Broadcast Isochronous Stream (BIS) broadcasts and Bluetooth Low Energy (BLE) audio. Additionally, Broadcast Isochronous Stream (BIS) is a specific Bluetooth Low Energy broadcast that uses a different communication protocol or data format from conventional Bluetooth audio. Therefore, if a Bluetooth headset does not support Low Energy Audio (LEA) or the relevant broadcast protocol, it cannot receive and process broadcasts.

[0082] In an embodiment of the present application, by using a first terminal device, through mixing at least one group of Broadcast Isochronous Stream (BIS) audio data and the first audio data, it is achieved that one headset can simultaneously process multiple types of audio streams from different devices. Moreover, the first terminal device mixes at least one group of Broadcast Isochronous Stream (BIS) audio data and the first audio data to form the second mixed audio data. In this way, what the second terminal device (e.g., a Bluetooth headset) receives is not the traditional Broadcast Isochronous Stream (BIS) based on the broadcast protocol, but an audio stream transmitted based on the Connection Isochronous Stream (CIS) protocol, solving the technical problem that non-Low Energy Audio (LEA) Bluetooth headsets cannot receive and process broadcasts.

[0083] The Connection Isochronous Stream (CIS) in an embodiment of the present application is a connection synchronous data stream. One or more Connection Isochronous Streams (CIS) are called a Connected Isochronous Group (CIG). The multiple Connection Isochronous Streams (CIS) in a Connected Isochronous Group (CIG) have a common timing reference based on central timing and are synchronized in time. This synchronization helps devices synchronize their input or output data. For example, when the left and right channels of an audio stereo stream need to be presented simultaneously, the audio data of the left and right channels of the Connection Isochronous Stream (CIS) needs to be synchronized. In other words, the Connection Isochronous Stream (CIS) is a specific synchronous data stream transmission channel, while the Connected Isochronous Group (CIG) is a group that manages these synchronous data stream transmission channels.

[0084] In addition, slot conflict refers to the time overlap between different sessions or data packets during communication, resulting in conflicts and interference. In the context of the present application, the slots of Broadcast Isochronous Stream (BIS) audio data and the slots of Connected Isochronous Stream (CIS) audio data may experience slot conflicts due to resource occupancy or data transmission conflicts. When the BIS slots and the CIS slots conflict in time, it may lead to data transmission interruption, data loss, or a decline in communication quality.

[0085] Among them, the priority of the Broadcast Isochronous Stream (BIS) audio data is higher than the priority of the first audio data.

[0086] In an embodiment of the present application, as Figure 8 shown, the method further includes step S801:

[0087] In step S801, when a conflict occurs between obtaining the Broadcast Isochronous Stream (BIS) audio data and receiving the first audio data, the Broadcast Isochronous Stream (BIS) audio data is preferentially obtained, and the first audio data is received at the next Connected Isochronous Group event (CIGevent).

[0088] For example, if the first terminal device is a smartphone, and the smartphone establishes a Connected Isochronous Stream (CIS) link with a Bluetooth headset to listen to music on the smartphone, and if the user wants to listen to a Broadcast Isochronous Stream (BIS) broadcast at this time, then the music playback on the smartphone needs to be paused, that is, the Connected Isochronous Stream (CIS) link is disconnected. In scenarios where the user is waiting at a railway station or an airport, the user often turns on the smartphone to watch movies, videos, or listen to music, making it impossible to listen to the broadcast at the station or airport) simultaneously and missing important announcements. Based on this, the embodiment of the present application considers the importance of both and sets the priority of the Broadcast Isochronous Stream (BIS) audio data to be higher than the priority of the first audio data, so that when the user listens to the first audio data, important broadcasts can be avoided being missed.

[0089] In addition, the Connected Isochronous Stream (CIS) needs to establish a point-to-point two-way data stream transmission channel with a fixed time interval (ISO Interval) between connected devices based on the low-power Bluetooth ACL connection established between devices. Each Connected Isochronous Stream (CIS) corresponds to an independent audio stream or other synchronous data streams for time-limited data transmission between devices. At the same time, the Broadcast Isochronous Stream (BIS) also needs to establish a point-to-point two-way data stream transmission channel with a fixed time interval (ISO Interval) between connected devices.

[0090] As Figure 13As shown in the figure, it is a schematic diagram of slot conflict between Broadcast Isochronous Stream (BIS) and Connection Isochronous Stream (CIS) in an embodiment of the present application. The Broadcast Isochronous Group anchor point (BIG anchor point) indicates the arrival of a Broadcast Isochronous Group event (BIG event), and at this time, the broadcast isochronous stream audio data starts to be sent. The audio data sent in the first Broadcast Isochronous Stream period (BIS ISO Interval) is P0, P0, P0. Similarly, the Connection Isochronous Group anchor point (CIG anchor point) indicates the arrival of a Connection Isochronous Group event (BIG event), and at this time, the connection isochronous stream audio data starts to be sent. Continuing to combine Figure 13 When the first Connection Isochronous Stream event (CIG event) in the first Connection Isochronous Stream period (CIG ISO Interval) arrives, the audio data sent by the first application is M0, M0, M0, M0. From Figure 13 it can be seen that there is an overlapping part between the above two periods, that is, a slot conflict occurs. The first terminal device receives all the broadcast isochronous stream audio data, but there is loss of connection isochronous stream audio data. The first M0 is actually not received, the second M0 is received, and the third M0 and the fourth M0 are received incorrectly and need to be retransmitted. Therefore, when the second Connection Isochronous Stream event (CIG event) arrives, M0 is retransmitted.

[0091] In an embodiment of the present application, as Figure 9 shown, the method further includes step S901 and step S902:

[0092] In step S901, broadcast parameters of the at least one group of broadcast isochronous streams are obtained through at least one data packet of the at least one group of broadcast isochronous stream audio data;

[0093] In step S902, the first terminal device calculates the bandwidth required by the at least one group of broadcast isochronous stream audio data through the broadcast parameters. If the required bandwidth is less than a preset bandwidth threshold, the at least one group of broadcast isochronous stream audio data is received.

[0094] Among them, the broadcast parameters mainly can include data transmission rate, signal frequency range, modulation method, etc. The specific calculation formula and method may vary depending on the specific implementation and technical standards of the broadcast system of the Broadcast Isochronous Group (BIG).

[0095] Combined with Figure 13, the bandwidth required for broadcasting isochronous stream audio data refers to the bandwidth occupied by a broadcast isochronous group event (BIG event) or a broadcast isochronous stream period (BIS ISO Interval). For example, if the preset bandwidth threshold is 40%, when the data stream of the broadcast isochronous group (BIG) occupies no more than 40% of the total bandwidth of the first terminal device, the first terminal device will synchronously obtain the broadcast isochronous stream audio data. If the data stream of the broadcast isochronous group (BIG) occupies more than 40% of the bandwidth, the first terminal device will not obtain the broadcast isochronous stream audio data.

[0096] In an embodiment of the present application, as Figure 10 shown, the method further includes step S1001 and step S1002 for the bandwidth occupied by the data stream of the broadcast isochronous group (BIG):

[0097] In step S1001, if the first audio data sent by the first application has been received before scanning the broadcast isochronous stream (BIS) broadcast and the first audio data has been sent to the second terminal device for playing, then the reception and transmission of the first audio data are paused to cause the second terminal device to stop playing the first audio data and receive the broadcast isochronous stream audio data;

[0098] In step S1002, after the first terminal device sends the second audio data to the second terminal device through the first Bluetooth connection, the connection parameters between the first terminal device and the second terminal device are calculated to create a second Bluetooth connection, and the first audio data is sent to the second terminal device through the second Bluetooth connection.

[0099] Among them, both the first Bluetooth connection and the second Bluetooth connection include a connection isochronous stream (CIS) Bluetooth connection or an advanced audio distribution profile (A2DP) Bluetooth connection.

[0100] In the implementation of the present application, if the first terminal device has established a connection isochronous stream (CIS) link with a Bluetooth headset through the first terminal device before starting the scanning function or before scanning the broadcast isochronous stream (BIS) broadcast, and has sent the first audio data to the Bluetooth headset and the first audio data has been played on the Bluetooth headset. Since the priority of the broadcast isochronous stream (BIS) is higher than that of the connection isochronous stream (CIS), the first terminal device needs to temporarily stop sending the first audio data to the Bluetooth headset or temporarily disconnect the connection isochronous stream (CIS) link between the first terminal device and the Bluetooth headset.

[0101] Then, the first terminal device starts to receive broadcast isochronous group (BIG) audio data. After successful reception, it is necessary to recalculate the connection isochronous group (CIG) parameters to re - establish the Bluetooth connection between the first terminal device and the Bluetooth headset and re - transmit the first audio data. Among them, the connection isochronous group (CIG) parameters can include (BN, FT>1, NSE) and (BIG ISOinterval - BIG event - 2*Offset), etc.

[0102] Then, a connection isochronous group (CIG) connection between the first terminal device and the Bluetooth headset is re - created according to the recalculated connection isochronous group (CIG) parameters. Among them, the calculation formula of the CIG anchor point is as follows: CIG anchor point = BIG event+Offset, where CIG anchor point represents the connection isochronous group reference point, BIG event represents the connection isochronous group event, and Offset represents the time offset.

[0103] As Figure 11 shown, it is a schematic flowchart of a multimedia playback control method according to another embodiment of the present application. In Figure 11 this, the first terminal device can be a smart phone, and the smart phone includes a mixing module and a Bluetooth module. The smart phone establishes a Bluetooth connection with the Bluetooth headset through the Bluetooth module. The multimedia playback control method 1100 according to the embodiment of the present application may include the following steps S1101, step S1102, step S1103, step S1104, and step S1105;

[0104] In step S1101, the smart phone scans the BIS broadcast and synchronizes the required BIS broadcast audio.

[0105] In step S1102, the smart phone establishes a CIS or A2DP link with the Bluetooth headset.

[0106] In step S1103, the mixing module of the smart phone mixes the BIS broadcast audio and the music played on the phone of the smart phone.

[0107] In step S1104, the mixed audio data is sent by the mixing module of the smart phone to the Bluetooth module of the smart phone.

[0108] In step S1105, the Bluetooth module of the smart phone sends the mixed audio data through the CIS or A2DP link established with the Bluetooth headset.

[0109] As Figure 12As shown, it is a schematic flowchart of a multimedia playback control method according to another embodiment of the present application. The multimedia playback control method 1200 according to an embodiment of the present application may include the following steps S1201, step S1202, step S1203, step S1204, step S1205, step S1206, step S1207, step S1208, step S1209, step S1210, and step S1211;

[0110] In step S1201, receive BIS data.

[0111] In step S1202, decode the BIS data into the first PCM.

[0112] In step S1203, receive local music data sent by the first application.

[0113] In step S1204, decode the local music data into the second PCM.

[0114] In step S1205, resample the first PCM and align the channels with the second PCM. This makes the sampling rates and the number of channels of the first PCM and the second PCM consistent.

[0115] In step S1206, resample the second PCM and align the channels with the first PCM. This makes the sampling rates and the number of channels of the second PCM and the first PCM consistent.

[0116] In step S1207, adjust the first volume ratio of the first PCM.

[0117] In step S1208, adjust the second volume ratio of the second PCM.

[0118] In step S1209, mix the first PCM and the second PCM frame by frame to form mixed audio data.

[0119] In step S1210, perform normalization processing on the mixed audio data;

[0120] In step S1211, output the mixed audio data in AAC / SBC / LC3 format.

[0121] As Figure 14As shown in the figure, it is a schematic diagram of a smart phone in an embodiment of the present application connected to a BIS broadcast and a Bluetooth headset. The smart phone establishes a CIS or A2DP connection with the Bluetooth headset through its own Bluetooth module. The smart phone scans the BIS broadcast to obtain BIS data, and sends the BIS data to the mixing module. At the same time, the first application pre-installed on the smart phone sends the first audio data (for example, music) to the mixing module. The mixing module mixes the BIS data with the first audio data (for example, music) to obtain mixed audio data, and then the smart phone sends the mixed audio data to the Bluetooth headset through the CIS or A2DP connection for playback.

[0122] The multimedia playback control method according to the embodiment of the present application scans the Broadcast Isochronous Stream (BIS) broadcast to obtain the Broadcast Isochronous Stream (BIS) audio data, mixes the first audio data sent by the first application and the Broadcast Isochronous Stream (BIS) audio data to form the second audio data, and sends the second audio data to the second terminal device through the first Bluetooth connection so that the second terminal device plays the second audio data. By adopting the technical solution of the embodiment of the present application, the problem that the Bluetooth headset cannot simultaneously listen to the Bluetooth Low Energy (BLE) Broadcast Isochronous Stream (BIS) broadcast and the Bluetooth Low Energy (BLE) Audio can be solved, and the problem that the Bluetooth headset that does not support Bluetooth Low Energy Audio (LEA) cannot receive the Broadcast Isochronous Stream (BIS) broadcast can also be solved, enabling users to experience the Broadcast Isochronous Stream (BIS) broadcast without having to replace the headset.

[0123] The following Figure 15 describes the terminal device of the present application, where Figure 15 FIG. shows a schematic block diagram of a terminal device 1500 according to an embodiment of the present application. As Figure 15 shown, the terminal device 1500 includes a broadcast scanning module 1501, a first application 1502, a mixing module 1503, and a Bluetooth module 1504.

[0124] Among them, the broadcast scanning module 1501 is configured to scan the broadcast isochronous stream broadcast to obtain the broadcast isochronous stream audio data, and send the broadcast isochronous stream audio data to the mixing module;

[0125] The first application 1502 is used to obtain or generate the first audio data;

[0126] The mixing module 1503 is configured to mix the first audio data sent by the first application and the broadcast isochronous stream audio data to form the second audio data;

[0127] The Bluetooth module 1504 is configured to send the second audio data to the second terminal device so that the second terminal device plays the second audio data.

[0128] In addition, according to the embodiments of the present application, a storage medium is further provided, on which program instructions are stored, and when the program instructions are run by a computer or a processor, they are used to execute the corresponding steps of the multimedia playback control method of the embodiments of the present application. The storage medium may, for example, include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.

[0129] In addition, according to the embodiments of the present application, a computer program product is further provided, and when the computer program / instructions are executed by a processor, the steps of the method described above are implemented.

[0130] Since the terminal device, storage medium, and computer program product of the embodiments of the present application can implement the foregoing multimedia playback control method, they have the same advantages as the foregoing multimedia playback control method.

[0131] Although example embodiments have been described herein with reference to the drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application thereto. Those of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as claimed by the appended claims.

[0132] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0133] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0134] In the specification provided herein, numerous specific details are set forth. However, it will be understood that embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0135] Similarly, it should be understood that, in order to streamline the present application and assist in understanding one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together in a single embodiment, figure, or description thereof. However, the methods of the present application should not be construed as reflecting an intention that the claimed present application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, the inventive point lies in that the corresponding technical problems can be solved with features less than all the features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, where each claim stands on its own as a separate embodiment of the present application.

[0136] Those skilled in the art will understand that, except where features are mutually exclusive, any combination may be employed of all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and all the processes or units of any method or apparatus so disclosed. Each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0137] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not others, combinations of features of different embodiments are meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0138] The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that in practice, a microprocessor or a digital signal processor (DSP) may be used to implement some or all of the functions of some of the modules according to the embodiments of the present application. The present application may also be implemented as a device program (e.g., a computer program and a computer program product) for performing part or all of the methods described herein. Such a program for implementing the present application may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0139] It should be noted that the above embodiments are illustrative of the present application rather than restrictive of the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names.

[0140] As described above, this is only the specific implementation manner or the description of the specific implementation manner of the present application, and the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A multimedia broadcast control method, characterized in that: Applied to a first terminal device, the method includes: Scanning the broadcast isochronous stream broadcast to obtain at least one set of broadcast isochronous stream audio data; Mixing the first audio data sent by the first application and the at least one group of broadcast isochronous stream audio data to form second audio data; Enable the first terminal device to establish a first Bluetooth connection with the second terminal device; Based on the first Bluetooth connection, the second audio data is sent to the second terminal device, so that the second terminal device plays the second audio data.

2. The method according to claim 1, characterized in that Mixing the first audio data sent by the first application and the at least one group of broadcast isochronous stream audio data to form second audio data includes the following steps: converting the at least one set of broadcast isochronous stream audio data and the first audio data into first pulse code modulated audio data and second pulse code modulated audio data respectively; aligning sampling rates of the first pulse code modulated audio data and the second pulse code modulated audio data; aligning channels of the first pulse code modulated audio data and the second pulse code modulated audio data; Based on a preset volume weight coefficient, adjusting the volume ratio of the first pulse code modulation audio data and the second pulse code modulation audio data; Mixing the first pulse code modulation audio data and the second pulse code modulation audio data frame by frame to form third audio data; Normalizing the third audio data to form the second audio data in a preset format; The preset format includes any one of the following: an advanced audio coding format, a sub-band coding format and a low complexity communication coding and decoding format.

3. The method according to claim 2, characterized in that Aligning the sampling rates of the first pulse code modulation audio data and the second pulse code modulation audio data, comprising: When the sampling rates of the first pulse code modulated audio data and the second pulse code modulated audio data are different, the sampling rates of the two are matched through a preset resampling algorithm to make the sampling rates of the first pulse code modulated audio data and the second pulse code modulated audio data consistent.

4. The method according to claim 2, characterized in that: Aligning channels of the first pulse code modulation audio data and the second pulse code modulation audio data, comprising: Detecting whether the number of channels of the first pulse code modulation audio data and the second pulse code modulation audio data is the same; If the first pulse code modulation audio data and the second pulse code modulation audio data have different channel numbers, the channel numbers of the two are converted to the same channel number.

5. The method according to claim 2, characterized in that: Mixing the first pulse code modulation audio data and the second pulse code modulation audio data frame by frame to form third audio data, comprising: An equal number of sampling points are collected from the first pulse code modulation audio data and the second pulse code modulation audio data respectively, and each sampling point is superimposed according to a preset sampling point weight to form third audio data.

6. The method according to claim 2, characterized in that Normalizing the third audio data to form the second audio data in a preset format includes: When the sampling value of the third audio data exceeds a preset sampling value range, normalization processing is performed on each sampling point of the third audio data.

7. The method according to claim 1, characterized in that in, The broadcast isochronous stream audio data at least includes broadcast period and frequency data; Scan the broadcast isochronous stream broadcast to obtain at least one set of broadcast isochronous stream audio data, including: At least one set of synchronized broadcast isochronous stream audio data is acquired according to the broadcast cycle and the frequency data.

8. The method according to claim 1, characterized in that in, The priority of the broadcast isochronous stream (BIS) audio data is higher than the priority of the first audio data; The method further comprises: When there is a conflict between obtaining the broadcast isochronous stream audio data and receiving the first audio data, the broadcast isochronous stream audio data is obtained first, and the first audio data is received in the next connection isochronous group event.

9. The method according to claim 8, characterized in that The method further comprises: Acquire the broadcast parameters of the at least one group of broadcast isochronous streams through at least one data packet of the at least one group of broadcast isochronous stream audio data; The first terminal device calculates the bandwidth required for the at least one group of broadcast isochronous stream audio data through the broadcast parameters, and receives the at least one group of broadcast isochronous stream audio data if the required bandwidth is less than a preset bandwidth threshold.

10. The method according to claim 8, characterized in that The method further comprises: If, before scanning the broadcast isochronous stream broadcast, the first audio data sent by the first application has been received, and the first audio data has been sent to the second terminal device for playback, then the reception and transmission of the first audio data are suspended so that the second terminal device stops playing the first audio data and receives the broadcast isochronous stream audio data; After the first terminal device sends the second audio data to the second terminal device through the first Bluetooth connection, the connection parameters of the first terminal device and the second terminal device are calculated to create a second Bluetooth connection, and the first audio data is sent to the second terminal device through the second Bluetooth connection.

11. The method according to any one of claims 1 to 10, characterized in that: in, The first Bluetooth connection and the second Bluetooth connection both include an isochronous streaming Bluetooth connection or an Advanced Audio Distribution Profile Bluetooth connection.

12. The method according to any one of claims 1 to 10, characterized in that: The first terminal device includes at least one of the following: a smart phone, a tablet computer, a desktop computer, a laptop computer, a smart watch, virtual reality glasses, augmented reality glasses, a wireless car system and a media player; the second terminal device includes at least a Bluetooth headset and / or a Bluetooth speaker.

13. A terminal device, characterized in that: The terminal device is used to execute the method according to any one of claims 1 to 12; the terminal device includes a broadcast scanning module, a first application, a mixing module and a Bluetooth module; wherein, The broadcast scanning module is configured to scan the broadcast isochronous stream broadcast to obtain the broadcast isochronous stream audio data, and send the broadcast isochronous stream audio data to the mixing module; The first application is used to obtain or generate first audio data; The mixing module is configured to mix the first audio data sent by the first application and the broadcast isochronous stream audio data to form second audio data; The Bluetooth module is configured to send the second audio data to a second terminal device so that the second terminal device plays the second audio data.

14. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the processor executes the multimedia broadcast control method according to any one of claims 1 to 12.

15. A computer program product, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images