Audio playing method and device, storage medium and program product
By directly processing the head tracking information of wireless headphones in the application of the terminal device, the problems of large burden on digital signal processors and poor output quality are solved, and efficient spatial audio rendering is achieved, improving the user experience.
Patent Information
- Application Number
- CN202411201620.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
In the prior art, digital signal processors need to process a variety of services, resulting in the inability to ensure the output quality of spatial audio, and the head tracking information transmission path is complex, affecting the user experience.
By directly receiving and processing the head tracking information of wireless headphones in the application of the terminal device, the application performs spatial audio rendering processing on the audio data, avoiding the intervention of traditional digital signal processors and ensuring audio output quality and somatosensory.
It improves the output quality and user sense of spatial audio, simplifies the transmission path of head tracking information, and enhances the efficiency and effect of audio rendering processing.
Smart Images

Figure CN119094972A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of Internet technology, and in particular to an audio playback method, device, storage medium and program product. Background Art
[0002] In recent years, spatial audio technology has developed rapidly. Spatial audio can make sound directional, giving users a more immersive listening experience.
[0003] In the related technology, the wireless headset sends head tracking information to the terminal device, and the digital signal processor (DSP) of the terminal device performs spatial audio rendering processing based on the head tracking information sent by the wireless headset and the audio data sent by the application, and then sends the processed spatial audio data to the wireless headset through the communication module.
[0004] However, digital signal processors need to handle a lot of tasks, including not only audio data processing, but also image data processing, radar signal processing, etc. And the output spatial audio quality cannot be guaranteed. Summary of the invention
[0005] In order to solve the related technical problems, the present disclosure provides an audio playback method, device, equipment, storage medium and program product. The technical solution is as follows:
[0006] In a first aspect, the present disclosure provides an audio playback method, the method being applied to an audio playback system, the audio playback system comprising a terminal device and an audio output device, the terminal device running an operating system and an application, the method comprising:
[0007] The application determines the audio data to be played and obtains the attribute information of the audio data;
[0008] The application decodes the audio data to obtain decoded audio data;
[0009] The application sends an audio play notification to the audio output device, wherein the audio play notification carries the attribute information;
[0010] The audio output device sends head tracking information of the audio output device to the terminal device, and the head tracking information is directly sent to the application through a channel;
[0011] The audio output device sends an audio request to the terminal device, and the audio request is sent to the application;
[0012] The application performs spatial audio rendering processing on the decoded audio data based on the head tracking information to obtain spatial audio data;
[0013] The application sends the spatial audio data to the audio output device;
[0014] The audio output device plays the spatial audio data based on the attribute information.
[0015] In a possible implementation manner, the head tracking information is sent through a first communication protocol, and the audio request and the spatial audio data are sent through a second communication protocol.
[0016] In a possible implementation, the head tracking information is sent directly to the application by a channel, including:
[0017] The head tracking information is sent from the communication hardware interface of the terminal device to the communication module, and then sent from the communication module to the application.
[0018] In a possible implementation, the audio request is sent directly to the application via the path;
[0019] Or the audio request is sent to the application through the operating system.
[0020] In a possible implementation manner, the head tracking information and the audio request are sent via a first communication protocol.
[0021] In a possible implementation, before the application determines the audio data to be played, the method further includes:
[0022] The application establishes a transmission channel with the audio output device based on the received discovery message broadcast by the audio output device.
[0023] In a possible implementation, the method further includes:
[0024] The audio output device obtains the authentication information sent by the application in the process of establishing the transmission channel;
[0025] After establishing the transmission channel, the audio output device sends the authentication information to the application;
[0026] The application keeps the transmission channel connected when it is determined that the received authentication information matches the sent authentication information, and disconnects the transmission channel when it is determined that the received authentication information does not match the sent authentication information.
[0027] In a possible implementation manner, the attribute information includes at least one of a sampling rate, a number of channels, bit depth information, and encoding information.
[0028] In a second aspect, the present disclosure provides an audio playback method, the method being applied to an audio playback system, the audio playback system comprising a terminal device and an audio output device, the terminal device running an operating system and an application, the method comprising:
[0029] The application sends an audio recording and playing notification to the audio output device, wherein the audio recording and playing notification carries attribute information;
[0030] The audio output device records the first audio data based on the attribute information;
[0031] The audio output device sends the first audio data to the terminal device, and the first audio data is sent to the application;
[0032] The audio output device sends head tracking information of the audio output device to the terminal device, and the head tracking information is directly sent to the application through a channel;
[0033] The audio output device sends an audio request to the terminal device, and the audio request is sent to the application;
[0034] The application performs mixing processing on the first audio data and the second audio data;
[0035] The application performs spatial audio rendering processing on the audio data after the mixing processing based on the head tracking information to obtain spatial audio data;
[0036] The application sends the spatial audio data to the audio output device;
[0037] The audio output device plays the spatial audio data based on the attribute information.
[0038] In a third aspect, a method for audio playback is provided, the method being applied to an audio playback system, the audio playback system comprising a terminal device and an audio output device, the terminal device running an application, the method comprising:
[0039] The application determines the audio data to be played and obtains the attribute information of the audio data;
[0040] The application decodes the audio data to obtain decoded audio data;
[0041] The application sends an audio play notification to the audio output device, wherein the audio play notification carries the attribute information;
[0042] The audio output device sends an audio request to the application;
[0043] The application sends the decoded audio data to the audio output device;
[0044] The audio output device performs spatial audio rendering processing on the decoded audio data based on its own head tracking information to obtain spatial audio data;
[0045] The audio output device plays the spatial audio data based on the attribute information.
[0046] In a fourth aspect, a device for audio playback is provided, the device comprising:
[0047] A determination module, used to determine the audio data to be played and obtain attribute information of the audio data;
[0048] A decoding module, used to decode the audio data to obtain decoded audio data;
[0049] a sending module, configured to send an audio play notification to the audio output device, wherein the audio play notification carries the attribute information; send head tracking information of the audio output device to the terminal device, wherein the head tracking information is directly sent to the application program through a channel; send an audio request to the terminal device, wherein the audio request is sent to the application program;
[0050] A rendering module, configured to perform spatial audio rendering processing on the decoded audio data based on the head tracking information to obtain spatial audio data;
[0051] The sending module is used to send the spatial audio data to the audio output device;
[0052] A playing module is used to play the spatial audio data based on the attribute information.
[0053] In a possible implementation manner, the head tracking information is sent through a first communication protocol, and the audio request and the spatial audio data are sent through a second communication protocol.
[0054] In a possible implementation, the sending module includes:
[0055] The head tracking information is sent from the communication hardware interface of the terminal device to the communication module, and then sent from the communication module to the application.
[0056] In a possible implementation, the audio request is sent directly to the application via the path;
[0057] Or the audio request is sent to the application through the operating system.
[0058] In a possible implementation manner, the head tracking information and the audio request are sent via a first communication protocol.
[0059] In a possible implementation manner, the device further includes an establishing module, configured to:
[0060] The application establishes a transmission channel with the audio output device based on the received discovery message broadcast by the audio output device.
[0061] In a possible implementation, the device further includes an authentication module, configured to:
[0062] The audio output device obtains the authentication information sent by the application in the process of establishing the transmission channel;
[0063] After establishing the transmission channel, the audio output device sends the authentication information to the application;
[0064] The application keeps the transmission channel connected when it is determined that the received authentication information matches the sent authentication information, and disconnects the transmission channel when it is determined that the received authentication information does not match the sent authentication information.
[0065] In a possible implementation manner, the attribute information includes at least one of a sampling rate, a number of channels, bit depth information, and encoding information.
[0066] In a fifth aspect, a device for audio playback is provided, the device comprising:
[0067] A sending module, used for sending an audio recording and playing notification to the audio output device, wherein the audio recording and playing notification carries attribute information;
[0068] A recording module, configured to record first audio data based on the attribute information;
[0069] The sending module is used to send the first audio data to the terminal device, and the first audio data is sent to the application; send the head tracking information of the audio output device to the terminal device, and the head tracking information is directly sent to the application through the channel; send an audio request to the terminal device, and the audio request is sent to the application;
[0070] A mixing module, used for mixing the first audio data and the second audio data;
[0071] A rendering module, configured to perform spatial audio rendering processing on the audio data after the mixing processing based on the head tracking information to obtain spatial audio data;
[0072] The sending module is used to send the spatial audio data to the audio output device;
[0073] A playing module is used to play the spatial audio data based on the attribute information.
[0074] In a sixth aspect, a device for audio playback is provided, the device comprising:
[0075] A determination module, used to determine the audio data to be played and obtain attribute information of the audio data;
[0076] A decoding module, used to decode the audio data to obtain decoded audio data;
[0077] A sending module, configured to send an audio play notification to the audio output device, wherein the audio play notification carries the attribute information; send an audio request to the application; and send the decoded audio data to the audio output device;
[0078] A rendering module, configured to perform spatial audio rendering processing on the decoded audio data based on its own head tracking information to obtain spatial audio data;
[0079] A playing module is used to play the spatial audio data based on the attribute information.
[0080] In the seventh aspect, a computer device is provided, which includes a memory and a processor, the memory is used to store computer instructions, and the processor executes the computer instructions stored in the memory so that the computer device executes the methods provided by the first aspect, the second aspect, the third aspect and their possible implementation methods.
[0081] In an eighth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by a computer device, the computer device executes the methods provided by the first aspect, the second aspect, the third aspect and their possible implementations.
[0082] In the ninth aspect, a computer program product is provided, which includes a computer program code. When the computer program code is executed by a computer device, the computer device executes the methods provided by the first aspect, the second aspect, the third aspect and their possible implementation methods.
[0083] In the disclosed embodiment, the wireless headset sends the head tracking information directly to the application through the channel, and the information is not sent to the digital signal processor. The application performs spatial audio rendering processing on the audio data according to the head tracking information, and the output quality and output effect of the spatial audio can be guaranteed by the application, thereby improving the user's spatial audio perception. BRIEF DESCRIPTION OF THE DRAWINGS
[0084] Figure 1 is a structural schematic diagram of an audio playback system provided by an embodiment of the present disclosure;
[0085] Figure 2 is a schematic diagram of the structure of a terminal device provided by an embodiment of the present disclosure;
[0086] Figure 3 is a structural schematic diagram of an audio output device provided by an embodiment of the present disclosure;
[0087] Figure 4 It is a schematic diagram of a processing flow of an audio playback method provided by an embodiment of the present disclosure;
[0088] Figure 5 is a schematic diagram of an audio playback process provided by an embodiment of the present disclosure;
[0089] Figure 6 is a schematic diagram of an audio playback process provided by an embodiment of the present disclosure;
[0090] Figure 7 is a schematic diagram of an audio playback process provided by an embodiment of the present disclosure;
[0091] Figure 8 is a schematic diagram of an audio playback process provided by an embodiment of the present disclosure;
[0092] Fig. 9 is a schematic diagram of a processing flow for establishing a channel provided by an embodiment of the present disclosure;
[0093] Fig.10 It is a schematic diagram of a processing flow of authentication verification provided by an embodiment of the present disclosure;
[0094] Fig.11 is a schematic diagram of a processing flow of an audio playback processing flow provided by an embodiment of the present disclosure;
[0095] Fig.12 is a schematic diagram of a processing flow of an audio playback processing flow provided by an embodiment of the present disclosure;
[0096] Fig.13 is a structural schematic diagram of an audio playback device provided by an embodiment of the present disclosure;
[0097] Fig.14 is a structural schematic diagram of an audio playback device provided by an embodiment of the present disclosure;
[0098] Fig.15 It is a structural diagram of an audio playback device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0099] The present disclosure provides an audio playback method, which can be performed by an audio playback system. The audio playback system can be composed of a terminal device 110 and an audio output device 120. The system architecture diagram can be as follows: Figure 1 shown.
[0100] The terminal device 110 is used to complete the main processing work during the audio playback process, and continuously sends the audio data of the terminal device to the audio output device for playback. The terminal device can run an operating system and an application of the terminal device. In the embodiment of the present disclosure, an audio playback application is used as an example for detailed description. Other situations are similar and will not be repeated.
[0101] The audio output device 120 is used to play the audio data sent by the terminal device, and to record the user's audio and send it to the terminal device 110. The audio output device can be a wireless headset, a speaker, an earphone, etc. The embodiment of the present disclosure is described in detail using a wireless headset as an example, and other situations are similar and will not be repeated.
[0102] like Figure 2 As shown, the terminal device may include a processor 210, a memory 220, a communication component 230, a display component 240, and the like.
[0103] The processor 210 may be a central processing unit (CPU) or a system on chip (SoC), etc. The processor 210 may be used to execute various instructions or data information involved in the method, for example, determining audio data to be played.
[0104] The memory 220 may include various volatile memories or non-volatile memories, such as a solid state disk (SSD), a dynamic random access memory (DRAM), etc. The memory 220 may be used to store pre-stored data, intermediate data, and result data in content type conversion and display processing, for example, to store decoded audio data, etc.
[0105] The communication component 230 may be a wired network connector, a wireless fidelity (WiFi) module, a Bluetooth module, a cellular network communication module, etc. The communication component 230 may be used to transmit data with other devices, which may be servers or other terminal devices, etc. For example, sending an audio playback notification to an audio output device.
[0106] The display component 240 can be an independent screen, or a screen integrated with the user device body, a projector, etc. The screen can be a touch screen or a non-touch screen (which can be displayed remotely). The display component 240 is used to display the system interface, application interface, etc., for example, displaying audio types with multiple sound qualities, etc.
[0107] like Figure 3 As shown, the audio output device may include a processor 310, a memory 320, a communication component 330, and the like.
[0108] The processor 310 may be a CPU or a SoC, etc. The processor 310 may be used to execute various instructions or data information involved in the method, for example, to perform analog-to-digital conversion, etc.
[0109] The memory 320 may include various volatile memories or non-volatile memories, such as SSD, DRAM memory, etc. The memory 320 may be used to store pre-stored data, intermediate data, and result data in content type conversion and display processing, for example, to store attribute information of audio data, etc.
[0110] The communication component 330 may be a wired network connector, a WiFi module, a Bluetooth module, a cellular network communication module, etc. The communication component 330 may also be used to transmit data with other devices, which may be other servers or terminal devices, etc. For example, sending head tracking information of an audio output device.
[0111] Based on the above application scenario, the present disclosure provides an audio playback method. The following is a detailed description of the corresponding processing flow. Figure 4 As shown, the following steps may be included:
[0112] 401. The application determines the audio data to be played and obtains the attribute information of the audio data.
[0113] For a song, there are a variety of audio data with different sound qualities in audio playback applications, such as master, panoramic sound, high quality, etc. After selecting a song, the user can choose the desired sound quality. Different audio data corresponds to different attribute information, and the attribute information includes at least one of sampling rate, number of channels, bit depth information and encoding information. The sampling rate is the number of samples extracted from a continuous signal per second to form a discrete signal. The number of channels is the independent audio signals collected or accessed at different spatial locations when the sound is recorded or played. There can be mono, dual channels and stereo channels, etc. Bit depth information refers to how many bits a sample contains. Common bit depth information is 16 bits, 24 bits and 32 bits.
[0114] 402. The application decodes the audio data to obtain decoded audio data.
[0115] The application determines a corresponding decoding algorithm according to the attribute information of the audio data, decodes the audio data, and obtains decoded audio data.
[0116] 403: The application sends an audio playback notification to the audio output device.
[0117] The audio play notification carries attribute information. There is no necessary sequence between step 402 and step 403.
[0118] 404. The audio output device sends the head tracking information of the audio output device to the terminal device. The head tracking information is directly sent to the application through the channel.
[0119] The head tracking information may be the posture information of the user's head collected by the gyroscope, or may be the data obtained by calculating the angle of the posture information of the user's head collected by the gyroscope, which is not limited in the embodiments of the present disclosure. The posture information may represent the rotation and movement of the user's head.
[0120] The head tracking information is sent directly to the application through the channel, that is, the head tracking information is sent to the communication module through the communication hardware interface of the terminal device, and then sent to the application by the communication module, without going through the operating system, that is, without the need for processing by the digital signal processor.
[0121] 405 , the audio output device sends an audio request to the terminal device, and the audio request is sent to the application.
[0122] The head tracking information is sent through a first communication protocol, and the audio request data is sent through a second communication protocol. The first communication protocol may be a wireless fidelity peer-to-peer (WIFIP2P) protocol, or others; the second communication protocol may be a Bluetooth transmission protocol or others, and the Bluetooth transmission protocol includes an advanced audio distribution profile (A2DP) protocol, a low energy audio transmission (LE Audio) protocol, etc.
[0123] There are several ways in which audio requests can be routed to your application.
[0124] Method 1: The audio request is sent to the application through the operating system, that is, the audio request is sent from the communication hardware interface to the communication module, then sent to the operating system through the communication module, and finally sent to the application by the operating system, for example Figure 5 shown.
[0125] In the second method, the audio request is sent directly to the application through the channel, that is, the audio request is sent from the communication hardware interface of the terminal device to the communication module, and then sent from the communication module to the application without going through the operating system, for example Figure 6 shown.
[0126] 406 , the application performs spatial audio rendering processing on the decoded audio data based on the head tracking information to obtain spatial audio data.
[0127] The application performs spatial audio rendering on the decoded audio data based on the head tracking information, and the obtained spatial audio data can change as the user's head rotates or moves. For example, when the user turns his head to the right, the user's left ear hears the audio data first, and the volume of the left ear is slightly louder than that of the right ear.
[0128] 407 , the application sends the spatial audio data to the audio output device.
[0129] There are several ways to transmit spatial audio data to an audio output device.
[0130] Method 1: spatial audio data is sent to the audio output device through the operating system, that is, the spatial audio data is first sent from the application to the operating system, then sent from the operating system to the communication module, and finally sent to the audio output device through the communication hardware interface, for example Figure 7 shown.
[0131] In the second method, the spatial audio data is directly sent to the audio output device through the channel, that is, the spatial audio data is sent from the application to the communication module, and then sent from the communication module to the communication hardware interface of the terminal device, and then sent to the audio output device, for example Figure 8 shown.
[0132] 408 , the audio output device plays the spatial audio data based on the attribute information.
[0133] The audio output device performs analog-to-digital conversion on the spatial audio data, converts the digital signal into an analog signal, and plays it. The audio output device determines the characteristic parameters of the analog-to-digital conversion based on the attribute information. The characteristic parameters of the analog-to-digital conversion include resolution, conversion time, accuracy, etc. The characteristic parameters of the analog-to-digital conversion vary depending on the attribute information of the audio data.
[0134] In a possible implementation, the process of establishing a channel can be as follows: Fig. 9 As shown, the following steps are included:
[0135] 901 , the application establishes a transmission channel with the audio output device based on the received discovery message broadcast by the audio output device.
[0136] After the audio output device is turned on, it periodically broadcasts a discovery message. When an application receives the discovery message, a transmission channel between the application and the audio output device is established.
[0137] 902. The audio output device obtains authentication information sent by the application program during the process of establishing the transmission channel.
[0138] The authentication information may include the name of the audio output device, the media access control address (MAC), etc.
[0139] In a possible implementation, the process of the terminal device performing authentication verification on the audio output device can be as follows: Fig.10 As shown, the following steps are included:
[0140] 1001. After the transmission channel is established, the audio output device sends authentication information to the application.
[0141] The authentication information may include the name of the audio output device, the media access control address (MAC), etc.
[0142] 1002, when the application determines that the received authentication information matches the sent authentication information, the application maintains the transmission channel connection; when the application determines that the received authentication information does not match the sent authentication information, the application disconnects the transmission channel.
[0143] In a possible implementation, the processing flow of audio playback can also be as follows: Fig.11 As shown, the following steps are included:
[0144] 1101, the application sends an audio recording and playing notification to the audio output device.
[0145] The audio recording and playback notification carries attribute information, which includes at least one of sampling rate, number of channels, bit depth information, and encoding information.
[0146] 1102. The audio output device records first audio data based on the attribute information.
[0147] 1103. The audio output device sends first audio data to the terminal device, and the first audio data is sent to the application.
[0148] 1104. The audio output device sends head tracking information of the audio output device to the terminal device. The head tracking information is directly sent to the application via the channel.
[0149] The specific content of the head tracking information is similar to that of the above step 404 and will not be repeated here.
[0150] 1105. The audio output device sends an audio request to the terminal device, and the audio request is sent to the application.
[0151] There are many ways to transmit audio requests and spatial audio data. The detailed description is similar to the above step 405 and will not be repeated here. There is no necessary order between the above steps 1104 and 1105.
[0152] 1106. The application performs mixing processing on the first audio data and the second audio data.
[0153] The first audio data is generally human voice audio data, and the second audio data is generally accompaniment or accompaniment plus original singing. The application mixes the first audio data and the second audio data to obtain audio data of human voice plus accompaniment.
[0154] 1107. The application performs spatial audio rendering processing on the mixed audio data based on the head tracking information to obtain spatial audio data.
[0155] The processing method of the spatial audio rendering process is similar to the above step 406, which will not be repeated here.
[0156] At 1108 , the application sends the spatial audio data to the audio output device.
[0157] 1109. The audio output device plays the spatial audio data based on the attribute information.
[0158] The audio output device performs analog-to-digital conversion on the spatial audio data, converts the digital signal into an analog signal, and plays it. The audio output device determines the characteristic parameters of the analog-to-digital conversion based on the attribute information. The characteristic parameters of the analog-to-digital conversion include resolution, conversion time, accuracy, etc. The characteristic parameters of the analog-to-digital conversion vary depending on the attribute information of the audio data.
[0159] In a possible implementation, the processing flow of audio playback can also be as follows: Fig.12 As shown, the following steps are included:
[0160] 1201. The application determines the audio data to be played and obtains the attribute information of the audio data.
[0161] The attribute information includes at least one of a sampling rate, a number of channels, bit depth information, and encoding information.
[0162] 1202. The application decodes the audio data to obtain decoded audio data.
[0163] The application determines a corresponding decoding algorithm according to the attribute information of the audio data, decodes the audio data, and obtains decoded audio data.
[0164] 1203. The application sends an audio play notification to the audio output device.
[0165] The audio playing notification carries attribute information. There is no necessary order between the above steps 1202 and 1203.
[0166] 1204. The audio output device sends an audio request to the application.
[0167] There are many ways to transmit the audio request and the spatial audio data. The detailed description is similar to the above step 405 and will not be repeated here.
[0168] 1205, the application sends the decoded audio data to the audio output device.
[0169] 1206. The audio output device performs spatial audio rendering processing on the decoded audio data based on its own head tracking information to obtain spatial audio data.
[0170] The specific content of the head tracking information is similar to that of the above step 404 and will not be repeated here.
[0171] 1207. The audio output device plays the spatial audio data based on the attribute information.
[0172] In the disclosed embodiment, the wireless headset sends the head tracking information directly to the application through the channel, and the information is not sent to the digital signal processor. The application performs spatial audio rendering processing on the audio data according to the head tracking information, and the output quality and output effect of the spatial audio can be guaranteed by the application, thereby improving the user's spatial audio perception.
[0173] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.
[0174] Based on the same technical concept, the embodiment of the present disclosure also provides an audio playback device, such as Fig.13 As shown, the device comprises:
[0175] The determination module 1310 is used to determine the audio data to be played and obtain the attribute information of the audio data. Specifically, it can implement the processing function of the above step 401 and other implicit steps.
[0176] The decoding module 1320 is used to decode the audio data to obtain decoded audio data, and can specifically implement the processing function of the above step 402 and other implicit steps.
[0177] The sending module 1330 is used to send an audio play notification to the audio output device, wherein the audio play notification carries attribute information; send head tracking information of the audio output device to the terminal device, and the head tracking information is directly sent to the application by the channel; send an audio request to the terminal device, and the audio request is sent to the application. Specifically, the processing functions of the above steps 403, 404, and 405, as well as other implicit steps, can be realized.
[0178] The rendering module 1340 is used to perform spatial audio rendering processing on the decoded audio data based on the head tracking information to obtain spatial audio data. Specifically, it can implement the processing function of the above step 406 and other implicit steps.
[0179] The sending module 1330 is used to send the spatial audio data to the audio output device, and can specifically implement the processing function of the above step 407 and other implicit steps.
[0180] The playing module 1350 is used to play the spatial audio data based on the attribute information, and can specifically implement the processing function of the above step 408 and other implicit steps.
[0181] In a possible implementation, the head tracking information is sent via a first communication protocol, and the audio request and the spatial audio data are sent via a second communication protocol. Specifically, the processing function of the above step 405 and other implicit steps can be implemented.
[0182] In a possible implementation, the sending module 1330 is used to send the head tracking information from the communication hardware interface of the terminal device to the communication module, and then the communication module sends it to the application. Specifically, it can implement the processing function of the above step 405 and other implicit steps.
[0183] In a possible implementation, the audio request is sent directly to the application program through a path; or the audio request is sent to the application program through an operating system. Specifically, the processing function of the above step 405 and other implicit steps can be implemented.
[0184] In a possible implementation, the head tracking information and the audio request are sent via the first communication protocol. Specifically, the processing function of the above step 405 and other implicit steps can be implemented.
[0185] In a possible implementation, the device further includes an establishment module 1360, which is used for: the application program establishes a transmission channel with the audio output device based on the received discovery message broadcast by the audio output device. Specifically, the processing functions of the above steps 901 and 902 and other implicit steps can be implemented.
[0186] In a possible implementation, the device further includes an authentication module 1370, which is used to: the audio output device obtains the authentication information sent by the application during the process of establishing the transmission channel; after the transmission channel is established, the audio output device sends the authentication information to the application; when the application determines that the received authentication information matches the sent authentication information, the transmission channel connection is maintained, and when the application determines that the received authentication information does not match the sent authentication information, the transmission channel is disconnected. Specifically, the processing functions of the above steps 1001 and 1002, as well as other implicit steps, can be implemented.
[0187] In a possible implementation manner, the attribute information includes at least one of a sampling rate, a number of channels, bit depth information, and encoding information.
[0188] The present disclosure also provides an audio playback device, such as Fig.14 As shown, the device comprises:
[0189] The sending module 1410 is used to send an audio recording and playing notification to the audio output device, wherein the audio recording and playing notification carries attribute information. Specifically, the processing function of the above step 1101 and other implicit steps can be implemented.
[0190] The recording module 1420 is used to record the first audio data based on the attribute information, and can specifically implement the processing function of the above step 1102 and other implicit steps.
[0191] The sending module 1410 is used to send the first audio data to the terminal device, and the first audio data is sent to the application; send the head tracking information of the audio output device to the terminal device, and the head tracking information is directly sent to the application by the channel; send the audio request to the terminal device, and the audio request is sent to the application. Specifically, the processing functions of the above steps 1103, 1104, and 1105, as well as other implicit steps, can be realized.
[0192] The audio mixing module 1430 is used to mix the first audio data and the second audio data, and can specifically implement the processing function of the above step 1106 and other implicit steps.
[0193] The rendering module 1440 is used to perform spatial audio rendering processing on the mixed audio data based on the head tracking information to obtain spatial audio data. Specifically, it can implement the processing function of the above step 1107 and other implicit steps.
[0194] The sending module 1410 is used to send the spatial audio data to the audio output device, and can specifically implement the processing function of the above step 1108 and other implicit steps.
[0195] The playing module 1450 is used to play the spatial audio data based on the attribute information, and can specifically implement the processing function of the above step 1109 and other implicit steps.
[0196] The present disclosure also provides an audio playback device, such as Fig.15 As shown, the device comprises:
[0197] The determination module 1510 is used to determine the audio data to be played and obtain the attribute information of the audio data. Specifically, it can implement the processing function of the above step 1201 and other implicit steps.
[0198] The decoding module 1520 is used to decode the audio data to obtain decoded audio data, and can specifically implement the processing function of the above step 1202 and other implicit steps.
[0199] The sending module 1530 is used to send an audio play notification to the audio output device, wherein the audio play notification carries attribute information; send an audio request to the application; and send the decoded audio data to the audio output device. Specifically, the processing functions of the above steps 1203, 1204, and 1205, as well as other implicit steps, can be realized.
[0200] The rendering module 1540 is used to perform spatial audio rendering processing on the decoded audio data based on its own head tracking information to obtain spatial audio data. Specifically, it can implement the processing function of the above step 1206 and other implicit steps.
[0201] The playing module 1550 is used to play the spatial audio data based on the attribute information, and can specifically implement the processing function of the above step 1207 and other implicit steps.
[0202] The above-mentioned determination module 1310, decoding module 1320, sending module 1330, rendering module 1340, playing module 1350, establishment module 1360, authentication module 1370, sending module 1410, recording module 1420, mixing module 1430, rendering module 1440, playing module 1450, determination module 1510, decoding module 1520, sending module 1530, rendering module 1540, and playing module 1550 can be implemented by a processor, or by a processor in combination with a memory and a display.
[0203] In the disclosed embodiment, the wireless headset sends the head tracking information directly to the application via the channel. The wireless headset sends the head tracking information directly to the application via the channel and the information is not sent to the digital signal processor. The application performs spatial audio rendering processing on the audio data according to the head tracking information. The output quality and output effect of the spatial audio can be guaranteed by the application, thereby improving the user's spatial audio perception.
[0204] It should be noted that: the audio playback device provided in the above embodiment only uses the division of the above functional modules as an example when performing audio playback. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the audio playback device provided in the above embodiment and the audio playback method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0205] The embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD (digital video disk)), or a semiconductor medium (e.g., a solid-state hard disk), etc. The computer-readable storage medium includes instructions that instruct the computing device to perform a business processing method, or instruct the computing device to play audio.
[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for audio playback, characterized in that: The method is applied to an audio playback system, the audio playback system includes a terminal device and an audio output device, the terminal device runs an operating system and an application program, and the method includes: The application determines the audio data to be played and obtains the attribute information of the audio data; The application program decodes the audio data to obtain decoded audio data; The application sends an audio play notification to the audio output device, wherein the audio play notification carries the attribute information; The audio output device sends head tracking information of the audio output device to the terminal device, and the head tracking information is directly sent to the application through a channel; The audio output device sends an audio request to the terminal device, and the audio request is sent to the application; The application performs spatial audio rendering processing on the decoded audio data based on the head tracking information to obtain spatial audio data; The application sends the spatial audio data to the audio output device; The audio output device plays the spatial audio data based on the attribute information.
2. The method according to claim 1, characterized in that The head tracking information is sent via a first communication protocol, and the audio request and the spatial audio data are sent via a second communication protocol.
3. The method according to claim 1, characterized in that The head tracking information is sent directly to the application by a channel, including: The head tracking information is sent from the communication hardware interface of the terminal device to the communication module, and then sent from the communication module to the application.
4. The method according to claim 1, characterized in that: The audio request is sent directly to the application via the path; Or the audio request is sent to the application through the operating system.
5. The method according to claim 1, characterized in that The head tracking information and the audio request are sent using the same communication protocol.
6. The method according to claim 1, characterized in that Before the application determines the audio data to be played, the method further includes: The application establishes a transmission channel with the audio output device based on the received discovery message broadcast by the audio output device.
7. The method according to claim 6, characterized in that The method further comprises: The audio output device obtains the authentication information sent by the application in the process of establishing the transmission channel; After establishing the transmission channel, the audio output device sends the authentication information to the application; The application keeps the transmission channel connected when it is determined that the received authentication information matches the sent authentication information, and disconnects the transmission channel when it is determined that the received authentication information does not match the sent authentication information.
8. The method according to claim 1, characterized in that The attribute information includes at least one of a sampling rate, a number of channels, bit depth information, and encoding information.
9. A method for audio playback, characterized in that: The method is applied to an audio playback system, the audio playback system includes a terminal device and an audio output device, the terminal device runs an operating system and an application program, and the method includes: The application sends an audio recording and playing notification to the audio output device, wherein the audio recording and playing notification carries attribute information; The audio output device records the first audio data based on the attribute information; The audio output device sends the first audio data to the terminal device, and the first audio data is sent to the application; The audio output device sends head tracking information of the audio output device to the terminal device, and the head tracking information is directly sent to the application through a channel; The audio output device sends an audio request to the terminal device, and the audio request is sent to the application; The application performs mixing processing on the first audio data and the second audio data; The application performs spatial audio rendering processing on the audio data after the mixing processing based on the head tracking information to obtain spatial audio data; The application sends the spatial audio data to the audio output device; The audio output device plays the spatial audio data based on the attribute information.
10. A method for audio playback, characterized in that: The method is applied to an audio playback system, the audio playback system includes a terminal device and an audio output device, the terminal device runs an application program, and the method includes: The application determines the audio data to be played and obtains the attribute information of the audio data; The application decodes the audio data to obtain decoded audio data; The application sends an audio play notification to the audio output device, wherein the audio play notification carries the attribute information; The audio output device sends an audio request to the application; The application sends the decoded audio data to the audio output device; The audio output device performs spatial audio rendering processing on the decoded audio data based on its own head tracking information to obtain spatial audio data; The audio output device plays the spatial audio data based on the attribute information.
11. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein the memory is used to store computer instructions; The processor executes the computer instructions stored in the memory to enable the computer device to perform the method according to any one of claims 1 to 10.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program code. When the computer program code is executed by a computer device, the computer device executes the method according to any one of claims 1 to 10.
13. A computer program product, characterized in that The computer program product comprises a computer program code. When the computer program code is executed by a computer device, the computer device executes the method according to any one of claims 1 to 10.
