Method, device, electronic device, storage medium and program product for audio processing

Abstract

The present disclosure relates to the technical field of audio processing, and specifically provides a method and apparatus for audio processing, an electronic device, a storage medium and a program product. The method comprises: receiving second audio data sent by a target audio device when first audio data is sent to an audio listening device; and performing an audio listening prompt operation and playing the second audio data if the second audio data meets an audio screening condition. In this way, when a user listens to the first audio data through the audio listening device, the user can also listen to the second audio data sent by the target audio device in time, and the audio listening of different devices can be taken into account, and the problem of missing important audio is reduced.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, specifically to an audio processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In audio listening scenarios, users typically listen to audio sent from audio source devices such as mobile phones through audio listening devices such as Bluetooth headphones.

[0003] However, when users listen to audio from an audio source device through an audio listening device, they often find it difficult to hear other audio sent by other devices in a timely manner, making it impossible to listen to audio sent by different devices simultaneously, resulting in significant audio loss issues. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide an audio processing method, apparatus, electronic device, storage medium, and program product.

[0005] On one hand, this application provides an audio processing method, including:

[0006] While sending the first audio data to the audio listening device, the system receives the second audio data sent by the target audio device.

[0007] If the second audio data meets the audio filtering criteria, then an audio listening prompt will be executed, and the second audio data will be played.

[0008] This way, you can listen to audio sent from different devices and reduce the problem of missing audio.

[0009] In one embodiment, the method further includes:

[0010] Pause the transmission of the first audio data to the audio listening device;

[0011] Send a listening prompt audio to the audio listening device; the listening prompt audio is used to prompt the user to listen to the audio.

[0012] In this way, users can be reminded to listen to audio through audio listening devices.

[0013] In one embodiment, the method further includes:

[0014] The listening prompt audio is mixed and synthesized with the first audio data to obtain composite audio data; wherein, the listening prompt audio is used to prompt the user to listen to the audio.

[0015] The composite audio data is sent to the audio listening device.

[0016] In this way, users can be reminded to listen to audio through audio listening devices.

[0017] In one embodiment, the method further includes:

[0018] The second audio data is mixed and synthesized with the first audio data to obtain composite audio data;

[0019] The composite audio data is sent to the audio listening device.

[0020] In this way, the second audio data can be played through an audio listening device.

[0021] In one embodiment, the second audio data is mixed and synthesized with the first audio data to obtain composite audio data, including:

[0022] The first audio data is subjected to audio gain adjustment to reduce the volume of the unicast audio.

[0023] The second audio data is decoded to obtain the decoded second audio data;

[0024] The decoded second audio data is sampled according to the sampling rate corresponding to the first audio data.

[0025] By using a linear superposition method, the adjusted first audio data and the sampled second audio data are time-domain mixed to obtain composite audio data.

[0026] This allows different audio tracks to be combined, enabling the simultaneous playback of different audio tracks.

[0027] In one embodiment, before receiving second audio data sent by the target audio device, the method further includes:

[0028] If the second audio data is broadcast audio data, obtain the target broadcast source information subscribed to by the audio listening device;

[0029] Based on the target broadcast source information, perform a broadcast subscription operation;

[0030] Based on the target broadcast source information subscribed to, start the broadcast audio monitoring service to monitor the broadcast audio data sent by the broadcast source device corresponding to the target broadcast source information.

[0031] In this way, the audio source device can also listen to the audio of the broadcast source device corresponding to the target broadcast source information subscribed to by the audio listening device.

[0032] In one embodiment, the method further includes:

[0033] Based on preset audio attribute information, audio filtering conditions are generated to filter broadcast audio that meets the user's needs.

[0034] In this way, audio filtering conditions can be configured using various audio attribute information to filter broadcast audio that meets the user's needs.

[0035] In one implementation, the audio attribute information includes at least one of the following:

[0036] Audio type, audio content, audio keywords, audio identifier, audio broadcast time, audio duration, and audio producer.

[0037] In this way, broadcast audio that meets the user's needs can be filtered through various audio attribute information.

[0038] In one implementation, performing an audio listening prompt operation includes:

[0039] Perform at least one of the following:

[0040] The equipment vibrates according to a specified vibration frequency.

[0041] The screen displays according to a specified frequency and / or specified lighting;

[0042] Broadcast notifications will be output in a multimedia format.

[0043] Pause sending the first audio data.

[0044] This serves as a reminder for users to listen to the radio.

[0045] On one hand, this application provides an audio processing apparatus, including:

[0046] The receiving unit is used to receive second audio data sent by the target audio device when sending first audio data to the audio listening device;

[0047] The playback unit is used to perform an audio listening prompt operation and play the second audio data if the second audio data meets the audio filtering conditions.

[0048] In one embodiment, the playback unit is further configured to:

[0049] Pause the transmission of the first audio data to the audio listening device;

[0050] Send a listening prompt audio to the audio listening device; the listening prompt audio is used to prompt the user to listen to the audio.

[0051] In one embodiment, the playback unit is further configured to:

[0052] The listening prompt audio is mixed and synthesized with the first audio data to obtain composite audio data; wherein, the listening prompt audio is used to prompt the user to listen to the audio.

[0053] The composite audio data is sent to the audio listening device.

[0054] In one embodiment, the playback unit is further configured to:

[0055] The second audio data is mixed and synthesized with the first audio data to obtain composite audio data;

[0056] The composite audio data is sent to the audio listening device.

[0057] In one embodiment, the playback unit is further configured to:

[0058] The first audio data is subjected to audio gain adjustment to reduce the volume of the unicast audio.

[0059] The second audio data is decoded to obtain the decoded second audio data;

[0060] The decoded second audio data is sampled according to the sampling rate corresponding to the first audio data.

[0061] By using a linear superposition method, the adjusted first audio data and the sampled second audio data are time-domain mixed to obtain composite audio data.

[0062] In one embodiment, the receiving unit is further configured to:

[0063] If the second audio data is broadcast audio data, obtain the target broadcast source information subscribed to by the audio listening device;

[0064] Based on the target broadcast source information, perform a broadcast subscription operation;

[0065] Based on the target broadcast source information subscribed to, start the broadcast audio monitoring service to monitor the broadcast audio data sent by the broadcast source device corresponding to the target broadcast source information.

[0066] In one embodiment, the playback unit is further configured to:

[0067] Based on preset audio attribute information, audio filtering conditions are generated to filter broadcast audio that meets the user's needs.

[0068] In one implementation, the audio attribute information includes at least one of the following:

[0069] Audio type, audio content, audio keywords, audio identifier, audio broadcast time, audio duration, and audio producer.

[0070] In one embodiment, the playback unit is used for:

[0071] Perform at least one of the following:

[0072] The equipment vibrates according to a specified vibration frequency.

[0073] The screen displays according to a specified frequency and / or specified lighting;

[0074] Broadcast notifications will be output in a multimedia format.

[0075] Pause sending the first audio data.

[0076] On one hand, embodiments of this application provide a computer-readable storage medium storing computer instructions for causing a computer to perform steps of the methods provided in various alternative implementations of any of the above-described audio processing methods.

[0077] On one hand, this application provides a computer program product including computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the steps of the method provided in various optional implementations of any of the above-described audio processing methods.

[0078] The audio processing method in this application includes receiving second audio data sent by a target audio device while sending first audio data to an audio listening device; if the second audio data meets the audio filtering conditions, then performing an audio listening prompt operation and playing the second audio data. In this way, while listening to the first audio data through the audio listening device, the user can also promptly hear the second audio data sent by the target audio device, thus ensuring audio listening on different devices and reducing the problem of missing important audio. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of an audio listening scenario in an embodiment of this application.

[0080] Figure 2 This is a schematic diagram of an image segmentation comparison in an embodiment of this application.

[0081] Figure 3 This is a schematic diagram of the structure of a broadcast source device in an embodiment of this application.

[0082] Figure 4 This is a schematic diagram of the structure of a broadcast subscription system according to an embodiment of this application.

[0083] Figure 5 This is a schematic diagram of the structure of an audio source device according to an embodiment of this application.

[0084] Figure 6 This is a schematic diagram of the structure of an audio receiving device according to an embodiment of this application.

[0085] Figure 7 This is a structural block diagram of an audio processing apparatus according to an embodiment of this application.

[0086] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0087] The technical solution of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0088] In audio listening scenarios, users typically use audio listening devices such as Bluetooth headphones to listen to broadcast audio or unicast audio sent by an audio source device. In Bluetooth audio transmission, broadcast audio uses a connectionless, unidirectional communication method, achieving one-to-many audio distribution through a broadcast channel, suitable for scenarios such as public address systems. Unicast audio, on the other hand, establishes a unicast audio communication link between devices through technologies such as Connected Isochronous Stream (CIS), achieving point-to-point audio transmission and featuring low latency and high reliability.

[0089] In some scenarios, some audio listening devices lack the ability to receive, decode, and play broadcast audio. Therefore, they can only support unicast audio listening and not broadcast audio listening, thus making it impossible to listen to the broadcast audio from the broadcast source device.

[0090] For audio listening devices that support both broadcast and unicast audio playback, only one type of audio (unicast or broadcast) can typically be playing at a time. These devices cannot automatically switch between broadcast and unicast playback, meaning that when listening to unicast audio, it's easy to miss broadcast notifications or announcements, which presents a limitation.

[0091] Because some important broadcast audio is often broadcast irregularly, audio listening devices typically need to constantly switch to broadcast listening mode while listening to unicast music to avoid missing important broadcast audio. However, this method requires the audio listening device to frequently switch audio, which consumes a lot of time and power. Unicast audio is constantly interrupted, and there are often issues such as no sound or sound stuttering during switching, resulting in a poor user experience and an inability to simultaneously receive and play both unicast and broadcast audio.

[0092] Based on the deficiencies of the aforementioned related technologies, this application provides an audio processing method, apparatus, electronic device, storage medium, and program product, which aims to allow for the listening of audio from other devices while listening to audio from an audio source device through an audio listening device.

[0093] This application provides an audio processing method that can be applied to electronic devices. This application does not limit the type of electronic device, which can be any suitable type of device, such as terminal devices and servers, etc. This application will not elaborate further.

[0094] See Figure 1 As shown in the diagram, this application provides an embodiment of an audio listening scenario. The audio listening scenario includes an audio source device, a broadcast source device, and an audio listening device.

[0095] The audio listening device is used to listen to audio, and it can be a Low Energy Audio (LE Audio) device, such as a Bluetooth headset. It may support only unicast audio listening or both broadcast and unicast audio listening. There can be one or more audio listening devices. The broadcast source device is used to send second audio data. The audio source device can also be called a central device (e.g., a mobile phone or tablet). It can be used to directly send first audio data to the audio listening device, and can also receive second audio data from the broadcast source device (e.g., airport / exhibition hall broadcasting equipment) and play the second audio data. LE Audio is a next-generation Bluetooth audio technology standard launched by the Bluetooth Special Interest Group (SIG).

[0096] See Figure 2 The diagram shown is a flowchart of an audio processing method according to an embodiment of this application, applied to... Figure 1 The audio source device shown below, in conjunction with Figure 1 and Figure 2 The method is described below, and the specific implementation process is as follows:

[0097] Step 201: While sending the first audio data to the audio listening device, receive the second audio data sent by the target audio device.

[0098] In this embodiment, the target audio device is a device used to transmit the second audio data. In this embodiment, the target audio device is a broadcast source device as an example.

[0099] Step 202: If the second audio data meets the audio filtering criteria, then perform the audio listening prompt operation and play the second audio data.

[0100] In one implementation, when performing step 201 above, the following steps may be adopted:

[0101] When the audio source device sends the first audio data to the audio receiving device, it broadcasts and listens to the target audio device to receive the second audio data sent by the target audio device.

[0102] Optionally, the unicast audio communication link can be a CIS link. For example, the audio source device sends the first audio data to the audio receiving device via the CIS link and listens for the broadcast.

[0103] Optionally, the first audio data can be unicast audio data. The second audio data can be broadcast audio data.

[0104] Figure 3 This is a schematic diagram of a broadcast source device. The broadcast source device includes an Auracast control module 301, an audio processing module 302, an EA / PA control module 303, a Bluetooth encoding module 304, a data packet processing module 305, a signal transmission module 306, and a pause broadcast module 307.

[0105] The Auracast control module 301 is used to send audio control commands to the audio processing module. These audio control commands are used to control the playback or pause of different audio types at different times, and also include audio content. For example, in one moment, audio content with an audio type context (Streaming_Audio_Contexts) of "Media" can be played; in the next moment, it can be changed to a sound announcement with an audio type such as "Notification" or "Alert" in the Streaming_Audio_Contexts.

[0106] The audio processing module 302 is used to: process the audio in the audio control commands, and send broadcast notification commands to the EA / PA control module 303 based on the processed audio data. As an example, the audio is rendered and resampled to obtain the processed audio data.

[0107] The EA / PA control module 303 is used to: broadcast the broadcast source information via Extended Advertising (EA) and broadcast the broadcast configuration information via Periodic Advertising (PA) if the broadcast notification command is to play; and send audio processing data to the Bluetooth encoding module 304 if the broadcast notification command is to pause.

[0108] As an example, broadcast source information may include at least one of the following: Broadcast Name and Audio Program Info. Broadcast configuration information may include at least one of the following fields: Audio Streaming_Audio_Contexts, BroadcastIsochronous Streams (BIS) link parameters (BIGInfo), and Audio codec configuration (Broascast Audio Source Endpoint (BASE)). For example, the audio playback status may refer to the BASE audio transmission status (Audio_Active_State) and the Broadcast_Audio_Immediate_Rendering_Flag.

[0109] The Bluetooth encoding module 304 is used to encode audio processing data and send the encoded audio data to the data packet processing module 305. For example, a Low Complexity Communications Codec (LC3) is used to encode the audio processing data.

[0110] The data packet processing module 305 is used to: perform Bluetooth packet processing on the audio encoded data to obtain second audio data, i.e., broadcast audio data. Furthermore, the second audio data can also be encrypted.

[0111] The signal transmitting module 306 is used to transmit second audio data using a BIS (Broadcast Media Sender) method, waiting for the audio receiving device to synchronize and listen to it. The signal transmitting module 306 can be a Broadcast Media Sender (BMS).

[0112] The pause broadcast module 307 is used to: if a pause control command is received, send a pause broadcast command to the signal transmission module 306 to interrupt the transmission of BIS audio (i.e., second audio data).

[0113] In this way, the target audio device can broadcast the second audio data.

[0114] Furthermore, a broadcast subscription is required before broadcast monitoring can be performed.

[0115] In one embodiment, when the second audio data is broadcast audio data, the target broadcast source information subscribed by the audio listening device is obtained; based on the target broadcast source information, a broadcast subscription operation is performed; and according to the subscribed target broadcast source information, a broadcast audio monitoring service is started to monitor the broadcast audio data sent by the broadcast source device corresponding to the target broadcast source information.

[0116] Optionally, the target broadcast source information subscribed to by the audio listening device can be subscribed to with the assistance of the audio device.

[0117] In one embodiment, a broadcast source is scanned to obtain at least one broadcast source information; based on the at least one broadcast source information, a target broadcast source information is selected from the broadcast source information, and the target broadcast source information is sent to an audio listening device to assist the audio listening device in subscribing to the target broadcast source information.

[0118] In this way, the audio source device can also monitor the audio of the broadcast source device corresponding to the target broadcast source information subscribed to by the audio listening device, and provide reminders for the broadcast audio data through the audio source device. On the one hand, this avoids frequent switching of the audio listening device and saves power. On the other hand, it can effectively prevent the reminder information from being ignored and provide more comprehensive and accurate reminders to the user.

[0119] The following is combined with Figure 4 This section explains how to subscribe to a radio program. See [link / reference]. Figure 4 The diagram shown is a schematic of a broadcast subscription system applied to audio source devices. Figure 4 The module includes a broadcast source scanning module 401, a broadcast source selection module 402, a broadcast subscription module 403, a Bluetooth protocol stack module 404, a synchronization PA module 405, a synchronization BIS module 406, and an RF signal receiving module 407.

[0120] The broadcast source scanning module 401 is used to scan and search for broadcast sources of broadcast source devices to obtain broadcast source information. Specifically, it can scan and parse EA broadcast packets to obtain key information such as relevant broadcast names. The broadcast source selection module 402 is used to select target broadcast source information based on at least one broadcast source information to select the corresponding broadcast source device. The broadcast subscription module 403 is used to subscribe to the corresponding broadcast source device. Specifically, it can lock the PA and BIS that need to be synchronized and send them to the Bluetooth protocol stack module 404. The Bluetooth protocol stack module 404 is used to establish synchronization. The PA synchronization module 405 is used to establish PA synchronization. The BIS synchronization module 406 is used to establish BIS synchronization. The radio frequency (RF) signal receiving module 407 is used to receive the RF signal of the PA / BIS in real time, that is, the second audio data of the broadcast source device.

[0121] In this way, the audio source device can subscribe to and listen to the second audio data according to the broadcast audio profile (BAP) of the broadcast source device.

[0122] Specifically, when performing step 202 above, that is, when the second audio data meets the audio filtering conditions, the audio listening prompt operation is performed and the second audio data is played, the following steps can be adopted:

[0123] In one implementation, it is determined whether the second audio data meets the audio filtering criteria. If so, an audio listening prompt operation is performed, and the second audio data is played through an audio player.

[0124] Furthermore, audio filtering conditions can be generated based on preset audio attribute information to filter broadcast audio that meets user needs.

[0125] In this way, audio filtering conditions can be configured using various audio attribute information to filter broadcast audio that meets the user's needs.

[0126] Optionally, the preset audio attribute information can be user-defined or system default settings. The audio attribute information includes at least one of the following: audio type, audio content, audio keywords, audio identifier, audio broadcast time, audio duration, and audio producer. This allows users to filter broadcast audio that meets their needs using various audio attribute information.

[0127] As an example, the audio filtering criteria are: the audio type of the second audio data is a user-defined emergency notification type.

[0128] Furthermore, preset audio attribute information can be adjusted in real time according to user needs.

[0129] The above items can be obtained from the audio configuration information of the second audio data. The audio filtering conditions can be set according to the actual application scenario, and there are no restrictions here.

[0130] Optionally, users can be notified to listen to the radio through vibration, sound, light, or unicast audio pause.

[0131] In one implementation, at least one of the following may be performed:

[0132] Method 1: Generate equipment vibration according to the specified vibration frequency.

[0133] For example, the vibration module in the mobile phone vibrates once every 1 second.

[0134] Furthermore, different vibration frequencies can be set for different types of broadcast audio, and the device vibration can be controlled according to the vibration frequency corresponding to the audio type of the second audio data.

[0135] Method 2: Display the screen according to a specified frequency and / or specified lighting.

[0136] For example, the screen can be alternately displayed and then turned off every 2 seconds. Another example is displaying multiple colors sequentially on the screen in a loop.

[0137] Method 3: Use multimedia to output broadcast notifications.

[0138] The multimedia format may include at least one of the following: text, sound, images, and animation.

[0139] Method 4: Pause the transmission of the first audio data.

[0140] In this way, users can be reminded to listen to the radio through the audio source device.

[0141] Furthermore, users can be prompted to listen to audio via audio listening devices.

[0142] In one implementation, any one of the following can be used:

[0143] Method 1: Pause sending the first audio data to the audio listening device; send a listening prompt audio to the audio listening device; wherein, the listening prompt audio is used to prompt the user to listen to the audio.

[0144] In one implementation, the transmission of first audio data to the audio listening device is paused, and a listening prompt audio is sent to the audio listening device via a unicast audio communication link. The listening prompt audio is the following voice: Please listen to the important broadcast.

[0145] Furthermore, after sending the listening prompt audio, the first audio data can be sent again.

[0146] Method 2: Mix the listening prompt audio with the first audio data to obtain composite audio data; wherein, the listening prompt audio is used to prompt the user to listen to the audio; send the composite audio data to the audio listening device.

[0147] Composite audio data can also be sent via a unicast audio communication link.

[0148] In this way, users can be reminded to listen to audio through audio listening devices.

[0149] Furthermore, a second audio data can be played to the user through an audio listening device.

[0150] In one implementation, any one of the following can be used:

[0151] Method 1: Pause the transmission of the first audio data to the audio listening device, and transmit the second audio data to the audio listening device through a unicast audio communication link, so as to play the broadcast through the audio listening device.

[0152] Method 2: Mix the second audio data with the first audio data to obtain composite audio data; send the composite audio data to the audio listening device.

[0153] In this way, the second audio data can be played through an audio listening device.

[0154] In one implementation, the following steps can be used to generate composite audio data:

[0155] S2021: Adjust the audio gain of the first audio data to reduce the volume of the unicast audio;

[0156] S2022: Decode the second audio data to obtain the decoded second audio data;

[0157] In one embodiment, the second audio data is decrypted to obtain a decrypted audio packet; the decrypted audio packet is grouped to obtain a combined audio packet; and the combined audio packet is decoded to obtain a broadcast audio stream, i.e., the decoded second audio data.

[0158] S2023: Sample the decoded second audio data according to the sampling rate corresponding to the first audio data;

[0159] S2024: Using a linear superposition method, the adjusted first audio data and the sampled second audio data are time-domain mixed to obtain composite audio data.

[0160] This allows different audio tracks to be combined, enabling the simultaneous playback of different audio tracks.

[0161] The following is combined with Figure 5 This section explains how to play audio from an audio source device. Figure 5 This is a schematic diagram of the structure of an audio source device.

[0162] Figure 5 It includes an audio player 501, an audio processing module 502, a Bluetooth encoding module 503, a data packet processing module 504, a unicast signal transceiver module 505, a broadcast signal transceiver module 506, an EA / PA parsing module 507, a BIS audio category filtering module 508, a data packet parsing module 509, a broadcast audio decoding module 510, an audio buffer module 511, an audio playback control module 512, a vibration module 513, and a speaker 514.

[0163] The audio player 501 controls the playback or pause of music according to playback control commands. The audio processing module 502 renders and resamples the audio. The Bluetooth encoding module 503 encodes the audio using Bluetooth LC3. The data packet processing module 504 performs packetization and encryption of Bluetooth audio packets. The unicast transceiver module 505 transmits first audio data, which may be Connected Isochronous Stream (CIS) audio. The unicast transceiver module 505 may be a unicast media sender (UMS).

[0164] When the audio source device plays CIS unicast audio music (i.e., unicast audio) to the audio receiving device, it can follow the standard playback process of BAP unicast audio, that is, send CIS unicast audio to the audio receiving device in sequence through the audio player 501, audio processing module 502, Bluetooth encoding module 503, data packet processing module 504, and unicast signal transceiver module 505.

[0165] The broadcast signal transceiver module 506 is used to receive second audio data. The second audio data may include at least one of the following: EA, PA, and BIS.

[0166] The EA / PA parsing module 507 is used to parse the second audio data to obtain the audio type.

[0167] For example, you can check the Audio_Active_State, Broadcast_Audio_Immediate_Rendering_Flag, and BIGInfo flags or parameters in the Audio Announcement in the PA to get the audio playback status, and you can parse Streaming_Audio_Contexts to get the current audio type.

[0168] The BIS audio category filtering module 508 is used to filter out second audio data that meets the audio filtering criteria.

[0169] In practical applications, audio filtering conditions can be set according to the specific needs and playback strategies of the application (i.e., the audio client) to determine whether to parse the received BIS broadcast data. For example, filtering can be performed based on audio type and / or audio content, which will not be elaborated here.

[0170] For example, the application requires that important announcement audio cannot be missed. When the audio source device is sending unicast audio to the audio receiving device, and the EA, PA, and BIS of the second audio data are all broadcasting normally, it is necessary to filter out the audio broadcasts of important types such as notifications, sound effects, and alerts. It is also possible to filter other audio types and playback content of the second audio data to decide whether to play the second audio data directly through the audio source device.

[0171] In this way, only the second audio data that meets the audio filtering criteria can be further decrypted according to actual needs, so that the second audio data can be directly played out through the speakers of the audio source device.

[0172] The data packet parsing module 509 is used to: decrypt the second audio data to obtain a decrypted audio packet; and assemble the decrypted audio packet to obtain a combined audio packet.

[0173] The broadcast audio decoding module 510 is used to decode the combined audio packets to obtain the broadcast audio stream.

[0174] The audio buffer module 511 is used to buffer broadcast audio streams.

[0175] The audio playback control module 512 is used to: if it is determined that there is currently unicast audio in a playback state, then perform a broadcast output operation.

[0176] For example, a pause notification can be sent to the audio player 501 to control the audio player 501 to pause the playback of unicast audio, thereby pausing the transmission of the first audio data. Then, the vibration module 513 performs a vibration operation to issue a vibration prompt, and then the broadcast audio stream is sent to the speaker 514. The speaker 514 plays the second audio data to realize the playback of the broadcast audio.

[0177] The following is combined with Figure 6 This section explains how to listen to audio using an audio receiving device. Figure 6 This is a schematic diagram of the structure of an audio receiving device. Figure 6 It includes: a unicast signal transceiver module 601, a data packet processing module 602, a unicast audio decoding module 603, an audio playback focus module 604, a CIS audio playback module 605, a player 606, a broadcast signal transceiver module 607, an EA / PA parsing module 608, a data packet parsing module 609, a BIS status module 610, a broadcast audio decoding module 611, and a BIS audio playback module 612.

[0178] The unicast transceiver module 601 is used to receive the first audio data and, upon determining that a request to switch to unicast audio playback has been received, to send a switch unicast audio playback command to the broadcast transceiver module 607. For example, the unicast transceiver module 601 can be a unicast media receiver (UMR).

[0179] The data packet processing module 602 performs Bluetooth packet processing on the first audio data to obtain the packet data. The unicast audio decoding module 603 decodes the packet data to obtain the unicast audio. The audio playback focus 604 requests audio playback focus for the unicast audio. The CIS audio playback 605 executes the unicast audio playback operation after confirming the request for audio playback focus. The speaker 606 outputs audio sound.

[0180] The broadcast signal transceiver module 607 is used to receive the second audio data and, upon receiving a unicast signal transceiver module 601's instruction to switch unicast audio playback, to stop playing the broadcast audio. The EA / PA parsing module 608 is used to parse the second audio data to obtain parsed data. The data packet parsing module 609 is used to parse data packets based on the second audio data and the parsed data to obtain data packet parsing data.

[0181] The BIS status module 610 is used to control whether to decode the second audio data. For example, the BIS audio stream playback status can be obtained from the data packet parsing data. If the BIS audio stream playback status is broadcast playback, a switching command is sent to the unicast signal transceiver module 601 to stop unicast audio playback and a decoding command is sent. Alternatively, if the data packet parsing data determines that the audio filtering conditions are met, the broadcast process is stopped.

[0182] The broadcast audio decoding module 611 is used to decode the data packet parsing data based on LC3 to obtain the broadcast audio if a decoding command is received. The BIS audio playback module 612 is used to execute the broadcast audio playback operation through the speaker 606 after determining the requested audio playback focus.

[0183] In this embodiment, when a user listens to audio from an audio source device through an audio listening device, if the audio source device receives second audio data sent by another device (i.e., the target audio), it promptly alerts and plays the second audio data, allowing for simultaneous listening to audio from different devices and reducing the problem of missing important audio. Specifically, in unicast audio listening scenarios, when listening to unicast audio through an audio listening device, the second audio data can be directly played through the speaker of the audio source device. This solves the problem of not being able to listen to broadcasts in a timely manner when listening to unicast audio through an audio listening device, allowing timely access to broadcast audio that meets the user's needs, reducing the problem of missing broadcast audio, and balancing the listening of both unicast and broadcast audio. It also enriches the playback methods of broadcast audio, allowing playback not only on the Auracast Receiver (e.g., headphones) but also on the Auracast Assistant (e.g., a mobile phone), resulting in shorter processing time and better real-time performance for receiving and playing emergency broadcast audio. Furthermore, users can be reminded to listen to the radio through vibration, screen display, and multimedia, as well as through audio listening devices. Additionally, secondary audio data can be sent to users via audio listening devices, further increasing the likelihood of users hearing the radio promptly, reducing cumbersome operations, and improving the user experience.

[0184] Based on the same inventive concept, this application also provides an audio processing apparatus. Since the principle of the above-mentioned apparatus and device in solving the problem is similar to that of an audio processing method, the implementation of the above-mentioned apparatus can refer to the implementation of the method, and repeated details will not be elaborated further. This apparatus can be applied to electronic devices. This application does not limit the type of electronic device; it can be any suitable type of device, such as terminal devices and servers, etc., which will not be elaborated further in this application. The apparatus embodiment can be implemented by software, or by hardware, or a combination of software and hardware. Taking software implementation as an example, as a logically defined apparatus, it is formed by the processor of the electronic device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0185] See Figure 7 The diagram shown is a structural block diagram of an audio processing apparatus according to an embodiment of this application. In some embodiments, the audio processing apparatus exemplified in this application includes:

[0186] The receiving unit 701 is used to receive second audio data sent by the target audio device when sending first audio data to the audio listening device;

[0187] The playback unit 702 is used to perform an audio listening prompt operation and play the second audio data if the second audio data meets the audio filtering conditions.

[0188] In one embodiment, the playback unit 702 is further configured to:

[0189] Pause the transmission of the first audio data to the audio listening device;

[0190] Send a listening prompt audio to the audio listening device; the listening prompt audio is used to prompt the user to listen to the audio.

[0191] In one embodiment, the playback unit 702 is further configured to:

[0192] The listening prompt audio is mixed and synthesized with the first audio data to obtain composite audio data; wherein, the listening prompt audio is used to prompt the user to listen to the audio.

[0193] The composite audio data is sent to the audio listening device.

[0194] In one embodiment, the playback unit 702 is further configured to:

[0195] The second audio data is mixed and synthesized with the first audio data to obtain composite audio data;

[0196] The composite audio data is sent to the audio listening device.

[0197] In one embodiment, the playback unit 702 is further configured to:

[0198] The first audio data is subjected to audio gain adjustment to reduce the volume of the unicast audio.

[0199] The second audio data is decoded to obtain the decoded second audio data;

[0200] The decoded second audio data is sampled according to the sampling rate corresponding to the first audio data.

[0201] By using a linear superposition method, the adjusted first audio data and the sampled second audio data are time-domain mixed to obtain composite audio data.

[0202] In one embodiment, the receiving unit 701 is further configured to:

[0203] If the second audio data is broadcast audio data, obtain the target broadcast source information subscribed to by the audio listening device;

[0204] Based on the target broadcast source information, perform a broadcast subscription operation;

[0205] Based on the target broadcast source information subscribed to, start the broadcast audio monitoring service to monitor the broadcast audio data sent by the broadcast source device corresponding to the target broadcast source information.

[0206] In one embodiment, the playback unit 702 is further configured to:

[0207] Based on preset audio attribute information, audio filtering conditions are generated to filter broadcast audio that meets the user's needs.

[0208] In one implementation, the audio attribute information includes at least one of the following:

[0209] Audio type, audio content, audio keywords, audio identifier, audio broadcast time, audio duration, and audio producer.

[0210] In one embodiment, the playback unit 702 is used for:

[0211] Perform at least one of the following:

[0212] The equipment vibrates according to a specified vibration frequency.

[0213] The screen displays according to a specified frequency and / or specified lighting;

[0214] Broadcast notifications will be output in a multimedia format.

[0215] Pause sending the first audio data.

[0216] The audio processing method in this application includes receiving second audio data sent by a target audio device while sending first audio data to an audio listening device; if the second audio data meets the audio filtering conditions, then performing an audio listening prompt operation and playing the second audio data. In this way, while listening to the first audio data through the audio listening device, the user can also promptly hear the second audio data sent by the target audio device, thus ensuring audio listening on different devices and reducing the problem of missing important audio.

[0217] In this embodiment of the application, an electronic device is also provided, including:

[0218] Processor; and

[0219] The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.

[0220] In this application embodiment, a computer-readable storage medium is provided, storing computer instructions for causing a computer to perform the methods of any of the above embodiments.

[0221] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the method of any of the above-described embodiments.

[0222] Figure 8 A schematic diagram of the structure of an electronic device 8000 is shown. (See attached diagram.) Figure 8 As shown, the electronic device 8000 includes a processor 8010 and a memory 8020, and optionally may also include a power supply 8030, a display unit 8040, and an input unit 8050.

[0223] The processor 8010 is the control center of the electronic device 8000. It connects various components through various interfaces and lines, and performs various functions of the electronic device 8000 by running or executing software programs and / or data stored in the memory 8020, thereby performing overall monitoring of the electronic device 8000.

[0224] In this embodiment, when the processor 8010 calls the computer program stored in the memory 8020, it executes the steps in the above embodiments.

[0225] Optionally, the processor 8010 may include one or more processing units; preferably, the processor 8010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 8010. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented separately on independent chips.

[0226] The memory 8020 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 8000, etc. In addition, the memory 8020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0227] The electronic device 8000 also includes a power supply 8030 (such as a battery) that supplies power to various components. The power supply can be logically connected to the processor 8010 through a power management system, thereby enabling the management of charging, discharging, and power consumption.

[0228] The display unit 8040 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 8000. In this embodiment, it is mainly used to display the display interface of various applications in the electronic device 8000, as well as text, images, and other objects displayed on the display interface. The display unit 8040 may include a display panel 8041. The display panel 8041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0229] The input unit 8050 can be used to receive information such as numbers or characters input by the user. The input unit 8050 may include a touch panel 8051 and other input devices 8052. The touch panel 8051, also known as a touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 8051).

[0230] Specifically, the touch panel 8051 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 8010, and receive and execute commands from the processor 8010. Furthermore, the touch panel 8051 can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors. Other input devices 8052 can include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0231] Of course, the touch panel 8051 can cover the display panel 8041. When the touch panel 8051 detects a touch operation on or near it, it transmits the information to the processor 8010 to determine the type of touch event. Subsequently, the processor 8010 provides corresponding visual output on the display panel 8041 based on the type of touch event. Although in Figure 8 In this embodiment, the touch panel 8051 and the display panel 8041 are two separate components to realize the input and output functions of the electronic device 8000. However, in some embodiments, the touch panel 8051 and the display panel 8041 can be integrated to realize the input and output functions of the electronic device 8000.

[0232] The electronic device 8000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity sensor, etc. Of course, depending on the specific application, the electronic device 8000 may also include other components such as a camera. Since these components are not the focus of this application's embodiments, therefore... Figure 8 It is not shown in the text and will not be described in detail here.

[0233] Those skilled in the art will understand that Figure 8 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.

[0234] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

Claims

1. An audio processing method, characterized in that, The method includes: While sending the first audio data to the audio listening device, the system receives the second audio data sent by the target audio device. If the second audio data meets the audio filtering criteria, an audio listening prompt will be executed, and the second audio data will be played.

2. The method according to claim 1, characterized in that, The method further includes: Pause the transmission of the first audio data to the audio listening device; Send a listening prompt audio to the audio listening device; wherein the listening prompt audio is used to prompt the user to listen to the audio.

3. The method according to claim 1, characterized in that, The method further includes: The listening prompt audio is mixed with the first audio data to obtain composite audio data; wherein, the listening prompt audio is used to prompt the user to listen to the audio; The composite audio data is sent to the audio listening device.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: The second audio data is mixed and synthesized with the first audio data to obtain composite audio data; The composite audio data is sent to the audio listening device.

5. The method according to claim 4, characterized in that, The step of mixing and synthesizing the second audio data with the first audio data to obtain composite audio data includes: The audio gain of the first audio data is adjusted to reduce the volume of the unicast audio. The second audio data is decoded to obtain the decoded second audio data; The decoded second audio data is sampled according to the sampling rate corresponding to the first audio data; The adjusted first audio data and the sampled second audio data are time-domain mixed using a linear superposition method to obtain the composite audio data.

6. The method according to any one of claims 1-3, characterized in that, Before receiving the second audio data sent by the target audio device, the method further includes: If the second audio data is broadcast audio data, obtain the target broadcast source information subscribed to by the audio listening device; Based on the target broadcast source information, perform a broadcast subscription operation; Based on the subscribed target broadcast source information, start the broadcast audio monitoring service to monitor the broadcast audio data sent by the broadcast source device corresponding to the target broadcast source information.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: Based on preset audio attribute information, the audio filtering conditions are generated to filter broadcast audio that meets the user's needs.

8. The method according to claim 7, characterized in that, The audio attribute information includes at least one of the following: Audio type, audio content, audio keywords, audio identifier, audio broadcast time, audio duration, and audio producer.

9. The method according to any one of claims 1-3, characterized in that, The operation of providing audio listening prompts includes: Perform at least one of the following: The equipment vibrates according to a specified vibration frequency; The screen displays according to a specified frequency and / or specified lighting; Broadcast notifications will be output in a multimedia format. Pause sending the first audio data.

10. An audio processing apparatus, characterized in that, The device includes: The receiving unit is used to receive second audio data sent by the target audio device when sending first audio data to the audio listening device; The playback unit is configured to perform an audio listening prompt operation and play the second audio data if the second audio data meets the audio filtering conditions.

11. An electronic device, characterized in that, include: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer contains computer instructions for causing the computer to perform the method described in any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the method according to any one of claims 1 to 9.

Images