Audio Processing Method, Apparatus, System, and Storage Medium

Abstract

The present invention provides an audio processing method, apparatus, system, and storage medium. First, the wireless headphone terminal receives a to-be-presented audio signal sent by a playback device through a wireless transmission manner. The to-be-presented audio signal includes an audio signal rendered and processed by the playback device, i.e., a first audio signal, and an audio signal to be rendered, i.e., a second audio signal. Then, if the to-be-presented audio signal includes the second audio signal, the wireless headphone terminal performs rendering processing on the second audio signal to obtain a third audio signal. Finally, the wireless headphone terminal performs subsequent audio playback according to the first audio signal and / or the third audio signal. Thereby, the technical effect that the wireless headphones can present high-quality surround sound and panoramic sound effects is achieved.

Description

Technical Field

[0001] The present invention relates to the field of electronic technology, and in particular to an audio processing method, device, system and storage medium. Background Art

[0002] With the development of smart mobile devices, headphones have become a must-have for people to listen to sounds in their daily lives. Wireless headphones are becoming more and more popular in the market due to their convenience, and have even gradually become mainstream headphone products. As a result, people's requirements for sound quality are getting higher and higher. Not only are they gradually pursuing lossless sound quality, but they are also gradually pursuing a sense of space and immersion in sound, from the initial mono and stereo to now more pursuing 360° surround sound and truly immersive three-dimensional panoramic sound.

[0003] At present, existing wireless headphones, such as traditional wireless Bluetooth headphones and TWS true wireless headphones, can only present a two-channel stereo sound field experience, which is increasingly unable to meet people's actual needs, especially the need for sound spatial sense when watching movies, and the need for sound orientation positioning when playing games.

[0004] Therefore, how to present true surround sound and panoramic sound effects on headphones, especially on the increasingly popular wireless headphones, has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides an audio processing method, device, system and storage medium to solve the technical problem of how wireless headphones can present high-quality surround sound and panoramic sound effects.

[0006] In a first aspect, the present invention provides an audio processing method, applied to a wireless headset, comprising:

[0007] Receiving an audio signal to be presented sent by a playback device in a wireless transmission manner, wherein the audio signal to be presented includes a first audio signal and / or a second audio signal, wherein the first audio signal is an audio signal rendered by the playback device, and the second audio signal is an audio signal to be rendered;

[0008] If the audio signal to be presented includes the second audio signal, rendering the second audio signal to obtain a third audio signal;

[0009] Subsequent audio playback is performed according to the first audio signal and / or the third audio signal.

[0010] In a possible design, before receiving the audio signal to be presented sent by the playback device in a wireless transmission manner, the method includes:

[0011] Send an indication signal to the playback device via a wireless transmission method, where the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the audio signal to be presented.

[0012] In a possible design, before sending the indication signal to the playback device via the wireless transmission method, it further includes:

[0013] Obtain the performance parameters of the wireless headset and determine the indication signal according to the performance parameters.

[0014] In a possible design, before sending the indication signal to the playback device via the wireless transmission method, it further includes:

[0015] Receive the audio characteristic information sent by the playback device, where the audio characteristic information includes the characteristic parameters of the original audio signal input to the playback device, and the characteristic parameters include at least one of the bitstream format, channel parameters, object parameters, and scene component parameters.

[0016] Optionally, the indication signal includes an identification code;

[0017] Wherein, if the identification code is the first field, the playback device does not render the original audio signal, the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the wireless headset renders the entire original audio signal;

[0018] If the identification code is the second field, the playback device renders the entire original audio signal, the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the wireless headset does not render the original audio signal;

[0019] If the identification code is the third field, the playback device renders a part of the original audio signal, which includes the first audio signal and the second audio signal, and the wireless headset renders the remaining part of the original audio signal.

[0020] Optionally, after receiving the audio signal to be presented sent by the playback device via the wireless transmission method, it further includes:

[0021] Perform decoding processing on the audio signal to be presented to obtain the first audio signal and / or the second audio signal.

[0022] Optionally, the rendering processing of the second audio signal to obtain the third audio signal includes:

[0023] Render the second audio signal according to the rendering metadata to obtain the third audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless headphone end.

[0024] In a possible design, the first metadata includes headphone sensor metadata, where the headphone sensor metadata is used to characterize the motion characteristics of the playback device; and / or,

[0025] The second metadata includes playback device sensor metadata and a head-related transfer function (HRTF) database, where the playback device sensor metadata is used to characterize the motion characteristics of the wireless headphones.

[0026] In a possible design, the headphone sensor metadata is obtained by a headphone sensor, and the headphone sensor includes at least one of a gyroscope sensor, a head size sensor, a distance sensor, a geomagnetic sensor, and an acceleration sensor; and / or,

[0027] The playback device sensor metadata is obtained by a playback device sensor, and the playback device sensor includes at least one of a gyroscope sensor, a head size sensor, a distance sensor, a geomagnetic sensor, and an acceleration sensor.

[0028] In a possible design, the wireless headphones include a first wireless headphone and a second wireless headphone;

[0029] The headphone sensor is provided in the first wireless headphone or the second wireless headphone; or,

[0030] If the headphone sensor is provided in both the first wireless headphone and the second wireless headphone, after the headphone sensor metadata is respectively obtained by the first wireless headphone and the second wireless headphone, the headphone sensor metadata is mutually synchronized.

[0031] In a possible design, the first wireless headphone and the second wireless headphone are used to establish a wireless connection with the playback device; receiving the audio signal to be presented sent by the playback device through a wireless transmission method includes:

[0032] The first wireless headphone receives a first audio signal to be presented sent by the playback device, and the second wireless headphone receives a second audio signal to be presented sent by the playback device;

[0033] Correspondingly, the rendering process in the wireless headphones includes:

[0034] The first wireless earphone performs rendering processing on the first audio signal to be presented to obtain a first playback audio signal, and the second wireless earphone performs rendering processing on the second audio signal to be presented to obtain a second playback audio signal;

[0035] The first wireless earphone plays the first playback audio signal, and the second wireless earphone plays the second playback audio signal.

[0036] In a possible design, before the first wireless earphone performs rendering processing on the first audio signal to be presented, it further includes:

[0037] The first wireless earphone performs decoding processing on the first audio signal to be presented to obtain a first decoded audio signal;

[0038] Correspondingly, the first wireless earphone performing rendering processing on the first audio signal to be presented includes:

[0039] The first wireless earphone performs rendering processing according to the first decoded audio signal and rendering metadata to obtain the first playback audio signal; and

[0040] Before the second wireless earphone performs rendering processing on the second audio signal to be presented, it further includes:

[0041] The second wireless earphone performs decoding processing on the second audio signal to be presented to obtain a second decoded audio signal;

[0042] Correspondingly, the second wireless earphone performing rendering processing on the second audio signal to be presented includes:

[0043] The second wireless earphone performs rendering processing according to the second decoded audio signal and rendering metadata to obtain the second playback audio signal.

[0044] In a possible design, the rendering metadata includes at least one of first wireless earphone metadata, second wireless earphone metadata, and playback device metadata.

[0045] In a possible design, the first wireless earphone metadata includes first earphone sensor metadata and a head-related transfer function (HRTF) database, where the first earphone sensor metadata is used to characterize the motion characteristics of the first wireless earphone;

[0046] The second wireless earphone metadata includes second earphone sensor metadata and a head-related transfer function (HRTF) database, where the second earphone sensor metadata is used to characterize the motion characteristics of the second wireless earphone;

[0047] The playback device metadata includes playback device sensor metadata, where the playback device sensor metadata is used to characterize the motion characteristics of the playback device.

[0048] In a possible design, before performing the rendering process, it further includes:

[0049] The first wireless earphone synchronizes the rendering metadata with the second wireless earphone.

[0050] In a possible design, if a headphone sensor is provided on the first wireless earphone, no headphone sensor is provided on the second wireless earphone, and no playback device sensor is provided on the playback device, then the first wireless earphone synchronizing the rendering metadata with the second wireless earphone includes:

[0051] The first wireless earphone sends the first headphone sensor metadata to the second wireless earphone, and the second wireless earphone uses the first headphone sensor metadata as the second headphone sensor metadata.

[0052] In a possible design, if headphone sensors are provided on both the first wireless earphone and the second wireless earphone, and no playback device sensor is provided on the playback device, then the first wireless earphone synchronizing the rendering metadata with the second wireless earphone includes:

[0053] The first wireless earphone sends the first headphone sensor metadata to the second wireless earphone, and the second wireless earphone sends the second headphone sensor metadata to the first wireless earphone;

[0054] The first wireless earphone and the second wireless earphone respectively determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm; or,

[0055] The first wireless earphone sends the first headphone sensor metadata to the playback device, and the second wireless earphone sends the second headphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm;

[0056] The first wireless earphone and the second wireless earphone respectively receive the rendering metadata.

[0057] In a possible design, if a headphone sensor is provided on the first wireless earphone, no headphone sensor is provided on the second wireless earphone, and a playback device sensor is provided on the playback device, then the first wireless earphone synchronizing the rendering metadata with the second wireless earphone includes:

[0058] The first wireless earphone sends the first earphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0059] The first wireless earphone and the second wireless earphone respectively receive the rendering metadata; or,

[0060] The first wireless earphone receives the playback device sensor metadata sent by the playback device;

[0061] The first wireless earphone determines the rendering metadata according to the first earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0062] The first wireless earphone sends the rendering metadata to the second wireless earphone.

[0063] In a possible design, if both the first wireless earphone and the second wireless earphone are provided with earphone sensors, and the playback device is provided with a playback device sensor, the synchronization of the rendering metadata by the first wireless earphone and the second wireless earphone includes:

[0064] The first wireless earphone sends the first earphone sensor metadata to the playback device, and the second wireless earphone sends the second earphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0065] The first wireless earphone and the second wireless earphone respectively receive the rendering metadata; or,

[0066] The first wireless earphone sends the first earphone sensor metadata to the second wireless earphone, and the second wireless earphone sends the second earphone sensor metadata to the first wireless earphone;

[0067] The first wireless earphone and the second wireless earphone respectively receive the playback device sensor metadata;

[0068] The first wireless earphone and the second wireless earphone respectively determine the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm.

[0069] Optionally, the audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

[0070] Optionally, the rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0071] Optionally, the wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

[0072] In a second aspect, the present invention provides another audio processing method, which is applied to a playback device and includes:

[0073] Obtain an original audio signal, and generate an audio signal to be presented according to the original audio signal. The audio signal to be presented includes a first audio signal and / or a second audio signal, where the first audio signal is the audio signal after rendering processing by the playback device, and the second audio signal is the audio signal to be rendered;

[0074] Send the audio signal to be presented to the wireless earphone through a wireless transmission method.

[0075] In a possible design, before sending the audio signal to be presented to the wireless earphone through the wireless transmission method, it includes:

[0076] Receive an indication signal sent by the wireless earphone through the wireless transmission method. The indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the audio signal to be presented.

[0077] In a possible design, before sending the audio signal to be presented to the wireless earphone through the wireless transmission method, it further includes:

[0078] Receive the performance parameters of the wireless earphone through the wireless transmission method, and determine an indication signal according to the performance parameters. The indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the audio signal to be presented.

[0079] In a possible design, receiving the performance parameters of the wireless earphone through the wireless transmission method and determining the indication signal according to the performance parameters includes:

[0080] Obtain the characteristic parameters of the original audio signal. The characteristic parameters include at least one of a bitstream format, a channel parameter, an object parameter, and a scene component parameter;

[0081] Determine the indication signal according to the characteristic parameter and the performance parameter.

[0082] In a possible design, the indication signal includes an identification code;

[0083] Wherein, if the identification code is the first field, the playback device does not render the original audio signal, then the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the wireless earphone renders all of the original audio signal;

[0084] If the identification code is the second field, the playback device renders all of the original audio signal, then the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the wireless earphone does not render the original audio signal;

[0085] If the identification code is the third field, the playback device partially renders the original audio signal, then it includes the first audio signal and the second audio signal, and the wireless earphone renders the remaining part of the original audio signal.

[0086] Optionally, the original audio signal includes a fourth audio signal and / or a fifth audio signal, wherein the fourth audio signal is used to generate the first audio signal after processing, and the fifth audio signal is used to generate the second audio signal;

[0087] Correspondingly, after obtaining the original audio signal, it further includes:

[0088] Decode the fourth audio signal to obtain a sixth audio signal, where the sixth audio signal includes a seventh audio signal and / or an eighth audio signal;

[0089] Render the seventh audio signal to obtain a ninth audio signal;

[0090] Encode the eighth audio signal and the ninth audio signal to obtain a tenth audio signal, and the audio signal to be presented includes the fifth audio signal and the tenth audio signal.

[0091] In a possible design, the rendering process of the seventh audio signal includes:

[0092] Render the seventh audio signal according to the rendering metadata to obtain the ninth audio signal, wherein the rendering metadata includes first metadata and second metadata, the first metadata is the metadata at the playback device end, and the second metadata is the metadata at the wireless earphone end.

[0093] In a possible design, the first metadata includes headphone sensor metadata, where the headphone sensor metadata is used to characterize the motion characteristics of the playback device; and / or,

[0094] The second metadata includes playback device sensor metadata and a head-related transfer function (HRTF) database, where the sensor metadata is used to characterize the motion characteristics of the wireless headphones.

[0095] In a possible design, the headphone sensor metadata is obtained through a headphone sensor, and the headphone sensor includes at least one of a gyroscope sensor, a head size sensor, a ranging sensor, a geomagnetic sensor, and an acceleration sensor; and / or,

[0096] The playback device sensor metadata is obtained through a playback device sensor, and the playback device sensor includes at least one of a gyroscope sensor, a head size sensor, a ranging sensor, a geomagnetic sensor, and an acceleration sensor.

[0097] Optionally, the audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

[0098] Optionally, the rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0099] Optionally, the wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

[0100] In a third aspect, the present invention provides an audio processing device, including:

[0101] An acquisition module, configured to receive, through a wireless transmission method, an audio signal to be presented sent by a playback device, where the audio signal to be presented includes a first audio signal and / or a second audio signal, and the first audio signal is an audio signal after being rendered by the playback device, and the second audio signal is an audio signal to be rendered;

[0102] A rendering module, configured to, when the audio signal to be presented includes the second audio signal, perform a rendering process on the second audio signal to obtain a third audio signal;

[0103] A playback module, configured to perform subsequent audio playback according to the first audio signal and / or the third audio signal.

[0104] In a possible design, before the receiving module is configured to receive, through a wireless transmission method, an audio signal to be presented sent by a playback device, it further includes:

[0105] A sending module, configured to send an indication signal to the playback device via a wireless transmission manner, where the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing manner to obtain the to-be-presented audio signal.

[0106] In a possible design, before the sending module is configured to send an indication signal to the playback device via a wireless transmission manner, it further includes:

[0107] The obtaining module is further configured to obtain performance parameters of the wireless headset and determine the indication signal according to the performance parameters.

[0108] In a possible design, before the sending module is configured to send an indication signal to the playback device via a wireless transmission manner, it further includes:

[0109] The obtaining module is further configured to receive audio characteristic information sent by the playback device, where the audio characteristic information includes characteristic parameters of the original audio signal input to the playback device, and the characteristic parameters include at least one of a bitstream format, a channel parameter, an object parameter, and a scene component parameter.

[0110] In a possible design, the indication signal includes an identification code;

[0111] Wherein, if the identification code is a first field, the playback device does not render the original audio signal, the to-be-presented audio signal includes the second audio signal and does not include the first audio signal, and the audio processing device fully renders the original audio signal;

[0112] If the identification code is a second field, the playback device fully renders the original audio signal, the to-be-presented audio signal includes the first audio signal and does not include the second audio signal, and the audio processing device does not render the original audio signal;

[0113] If the identification code is a third field, the playback device partially renders the original audio signal, which includes the first audio signal and the second audio signal, and the audio processing device renders the remaining part of the original audio signal.

[0114] In a possible design, after the obtaining module is configured to receive the to-be-presented audio signal sent by the playback device via a wireless transmission manner, it further includes:

[0115] A decoding module, configured to perform decoding processing on the to-be-presented audio signal to obtain the first audio signal and / or the second audio signal.

[0116] In a possible design, the rendering module is configured to perform rendering processing on the second audio signal to obtain a third audio signal, including:

[0117] The rendering module is configured to perform rendering processing on the second audio signal according to rendering metadata to obtain the third audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless headphone end.

[0118] In a possible design, the first metadata includes first sensor module metadata, where the first sensor module metadata is used to characterize the motion characteristics of the playback device; and / or,

[0119] The second metadata includes second sensor module metadata and a head-related transfer function (HRTF) database, where the second sensor module metadata is used to characterize the motion characteristics of the wireless headphones.

[0120] In a possible design, the headphone sensor metadata is obtained through a first sensor module, and the first sensor module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module; and / or,

[0121] The playback device sensor metadata is obtained through a second sensor module, and the second sensor module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module.

[0122] In a possible design, the audio processing device includes a first audio processing device and a second audio processing device;

[0123] The second sensor sub-module is provided in the first audio processing device or the second audio processing device; or,

[0124] The second sensor sub-module is provided in both the first audio processing device and the second audio processing device. After the acquisition modules of the first audio processing device and the second audio processing device obtain the playback device sensor metadata, it further includes:

[0125] A synchronization module, configured to synchronize the playback device sensor metadata with each other.

[0126] In a possible design, the first audio processing device includes:

[0127] A first receiving module, configured to receive a first audio signal to be presented sent by a playback device;

[0128] A first rendering module, configured to perform rendering processing on the first audio signal to be presented to obtain a first playback audio signal;

[0129] A first playback module, configured to play the first playback audio signal;

[0130] The second audio processing device includes:

[0131] A second receiving module, configured to receive a second audio signal to be presented sent by the playback device;

[0132] A second rendering module, configured to perform rendering processing on the second audio signal to be presented to obtain a second playback audio signal;

[0133] A second playback module, configured to play the second playback audio signal.

[0134] In a possible design, the first audio processing device further includes:

[0135] A first decoding module, configured to perform decoding processing on the first audio signal to be presented to obtain a first decoded audio signal;

[0136] The first rendering module is specifically configured to: perform rendering processing according to the first decoded audio signal and rendering metadata to obtain the first playback audio signal;

[0137] The second audio processing device further includes:

[0138] A second decoding module, configured to perform decoding processing on the second audio signal to be presented to obtain a second decoded audio signal;

[0139] The second rendering module is specifically configured to: perform rendering processing according to the second decoded audio signal and rendering metadata to obtain the second playback audio signal.

[0140] In a possible design, the rendering metadata includes at least one of first wireless headphone metadata, second wireless headphone metadata, and playback device metadata.

[0141] In a possible design, the first wireless headphone metadata includes first headphone sensor metadata and a head-related transfer function (HRTF) database, where the first headphone sensor metadata is used to characterize the motion characteristics of the first wireless headphone;

[0142] The second wireless headphone metadata includes second headphone sensor metadata and a head-related transfer function (HRTF) database, wherein the second headphone sensor metadata is used to characterize the motion characteristics of the second wireless headphone;

[0143] The playback device metadata includes playback device sensor metadata, wherein the playback device sensor metadata is used to characterize the motion characteristics of the playback device.

[0144] In a possible design, the first audio processing device further includes:

[0145] A first synchronization module, configured to synchronize the rendering metadata with the second wireless headphone; and / or,

[0146] The second audio processing device further includes:

[0147] A second synchronization module, configured to synchronize the rendering metadata with the first wireless headphone.

[0148] In a possible design, the first synchronization module is specifically configured to: send the first headphone sensor metadata to the second wireless headphone, so that the second synchronization module uses the first headphone sensor metadata as the second headphone sensor metadata.

[0149] In a possible design, the first synchronization module is specifically configured to:

[0150] Send the first headphone sensor metadata;

[0151] Receive the second headphone sensor metadata;

[0152] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm;

[0153] The second synchronization module is specifically configured to:

[0154] Send the second headphone sensor metadata;

[0155] Receive the first headphone sensor metadata;

[0156] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm; or,

[0157] The first synchronization module is specifically configured to:

[0158] Send the first headphone sensor metadata;

[0159] Receive the rendering metadata;

[0160] The second synchronization module is specifically configured to:

[0161] Send the second headphone sensor metadata;

[0162] Receive the rendering metadata.

[0163] In a possible design, the first synchronization module is specifically configured to:

[0164] Receive the playback device sensor metadata;

[0165] Determine the rendering metadata according to the first headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0166] Send the rendering metadata.

[0167] In a possible design, the first synchronization module is specifically configured to:

[0168] Send the first headphone sensor metadata;

[0169] Receive the second headphone sensor metadata;

[0170] Receive the playback device sensor metadata;

[0171] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0172] The second synchronization module is specifically configured to:

[0173] Send the second headphone sensor metadata;

[0174] Receive the first headphone sensor metadata;

[0175] Receive the playback device sensor metadata;

[0176] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm.

[0177] Optionally, the audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

[0178] Optionally, the rendering processing includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0179] Optionally, the wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

[0180] Fourthly, another audio processing device provided by the present invention includes:

[0181] An acquisition module, configured to receive an original audio signal and generate a to-be-presented audio signal according to the original audio signal, where the to-be-presented audio signal includes a first audio signal and / or a second audio signal, and wherein the first audio signal is an audio signal after being rendered by a playback device, and the second audio signal is an audio signal to be rendered;

[0182] A sending module, configured to send the to-be-presented audio signal to a wireless headset by a wireless transmission method.

[0183] In a possible design, before the sending module is configured to send the to-be-presented audio signal to the wireless headset by the wireless transmission method, it includes:

[0184] The acquisition module is further configured to receive an indication signal sent by the wireless headset through the wireless transmission method, where the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the to-be-presented audio signal.

[0185] In a possible design, before the sending module is configured to send the to-be-presented audio signal to the wireless headset by the wireless transmission method, it further includes:

[0186] The acquisition module is further configured to receive performance parameters of the wireless headset through the wireless transmission method, and determine an indication signal according to the performance parameters, where the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the to-be-presented audio signal.

[0187] In a possible design, the acquisition module is further configured to receive performance parameters of the wireless headset through the wireless transmission method, and determine an indication signal according to the performance parameters, including:

[0188] The acquisition module is further configured to acquire characteristic parameters of the original audio signal, where the characteristic parameters include at least one of a bitstream format, a channel parameter, an object parameter, and a scene component parameter;

[0189] The acquisition module is further configured to determine the indication signal according to the characteristic parameters and the performance parameters.

[0190] Optionally, the indication signal includes an identification code;

[0191] Wherein, if the identification code is the first field, the playback device does not render the original audio signal, then the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the audio processing device performs full rendering on the original audio signal;

[0192] If the identification code is the second field, the playback device performs full rendering on the original audio signal, then the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the audio processing device does not render the original audio signal;

[0193] If the identification code is the third field, the playback device performs partial rendering on the original audio signal, then it includes the first audio signal and the second audio signal, and the audio processing device renders the remaining part of the original audio signal.

[0194] Optionally, the original audio signal includes a fourth audio signal and / or a fifth audio signal, wherein the fourth audio signal is used to generate the first audio signal after processing, and the fifth audio signal is used to generate the second audio signal;

[0195] Correspondingly, after the obtaining module for obtaining the original audio signal, it further includes:

[0196] A decoding module for decoding the fourth audio signal to obtain a sixth audio signal, and the sixth audio signal includes a seventh audio signal and / or an eighth audio signal;

[0197] A rendering module for performing rendering processing on the seventh audio signal to obtain a ninth audio signal;

[0198] An encoding module for encoding the eighth audio signal and the ninth audio signal to obtain a tenth audio signal, and the audio signal to be presented includes the fifth audio signal and the tenth audio signal.

[0199] In a possible design, the rendering module for performing rendering processing on the seventh audio signal includes:

[0200] The rendering module is used to perform rendering processing on the seventh audio signal according to rendering metadata to obtain the ninth audio signal, wherein the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless headphone end.

[0201] In a possible design, the first metadata includes first sensor sub-module metadata, where the first sensor sub-module metadata is used to characterize the motion characteristics of the playback device; and / or,

[0202] The second metadata includes second sensor sub-module metadata and a head-related transfer function (HRTF) database, where the sensor sub-module metadata is used to characterize the motion characteristics of the wireless earphone.

[0203] In a possible design, the first sensor sub-module metadata is obtained through a first sensor sub-module, and the first sensor sub-module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module; and / or,

[0204] The second sensor sub-module metadata is obtained through a second sensor sub-module, and the second sensor sub-module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module.

[0205] Optionally, the audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

[0206] Optionally, the rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0207] Optionally, the wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

[0208] In a fifth aspect, the present invention further provides a wireless earphone, including:

[0209] A processor; and

[0210] A memory for storing the computer program of the processor;

[0211] Wherein, the processor is configured to implement any possible audio processing method in the first aspect by executing the computer program.

[0212] In a sixth aspect, the present invention further provides a playback device, including:

[0213] A processor; and

[0214] A memory for storing the computer program of the processor;

[0215] Wherein, the processor is configured to implement any possible audio processing method in the second aspect by executing the computer program.

[0216] In a seventh aspect, the present invention further provides a storage medium, in which a computer program is stored, and the computer program is used to execute any possible audio processing method provided in the first aspect.

[0217] In an eighth aspect, the present invention further provides a storage medium, in which a computer program is stored, and the computer program is used to execute any possible audio processing method provided in the second aspect.

[0218] In a ninth aspect, the present invention further provides a system, including the wireless earphone in the fifth aspect and the playback device in the sixth aspect.

[0219] The present invention provides an audio processing method, device, system and storage medium. First, the wireless earphone receives the audio signal to be presented sent by the playback device through a wireless transmission method. The audio signal to be presented includes the audio signal after rendering processing by the playback device, that is, the first audio signal, and the audio signal to be rendered, that is, the second audio signal. Then, if the audio signal to be presented includes the second audio signal, the wireless earphone renders the second audio signal to obtain a third audio signal. Finally, the wireless earphone performs subsequent audio playback according to the first audio signal and / or the third audio signal. Thus, the technical effect that the wireless earphone can present high-quality surround sound and panoramic sound effects is achieved. Description of the Drawings

[0220] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0221] Figure 1 It is a schematic structural diagram of a wireless earphone shown according to an exemplary embodiment of the present invention;

[0222] Figure 2 It is a schematic diagram of an application scenario of an audio processing method shown according to an exemplary embodiment of the present invention;

[0223] Figure 3 It is a schematic flowchart of an audio processing method shown according to an exemplary embodiment of the present invention;

[0224] Figure 4 It is a schematic diagram of the rendering methods included in an audio data rendering module provided by an embodiment of the present invention;

[0225] Figure 5 It is a schematic flow chart of a HRTF rendering method provided by an embodiment of the present invention;

[0226] Figure 6 It is a schematic flow chart of another HRTF rendering method provided by an embodiment of the present invention;

[0227] Figure 7 It is a schematic data flow diagram of audio signal rendering on the wireless headphone side provided by an embodiment of the present invention;

[0228] Figure 8 It is a schematic flow chart of another audio processing method provided by an embodiment of the present invention;

[0229] Figure 9 It is a schematic data link diagram of audio processing signals in a playback device and a wireless headphone provided by an embodiment of the present invention;

[0230] Figure 10 It is a schematic flow chart of yet another audio processing method provided by an embodiment of the present invention;

[0231] Figure 11 It is a schematic diagram of the rendering process of channel information for TWS true wireless headphones provided by an embodiment of the present invention;

[0232] Figure 12 It is a schematic structural diagram of an audio processing device provided by an embodiment of the present invention;

[0233] Figure 13 It is a schematic structural diagram of another audio processing device provided by an embodiment of the present invention;

[0234] Figure 14 It is a schematic structural diagram of a wireless headphone provided by the present invention;

[0235] Figure 15 It is a schematic structural diagram of another playback device provided by the present invention. Detailed implementation manners

[0236] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0237] In the description, claims, and above-mentioned drawings of the present invention, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0238] The technical solutions of the present invention and how the technical solutions of this application solve the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below with reference to the drawings.

[0239] Figure 1 It is a schematic structural diagram of a wireless earphone shown according to an exemplary embodiment of the present invention. Figure 2 It is a schematic diagram of an application scenario of an audio processing method shown according to an exemplary embodiment of the present invention. As Figure 1 - Figure 2 shown, the wireless transceiver device group communication method provided in this embodiment is applied to the wireless earphone 10. Among them, the wireless earphone 10 includes a first wireless earphone 101 and a second wireless earphone 102, and the wireless transceiver devices in the wireless earphone 10 are communicatively connected through a first wireless link 103. It should be noted that the communication connection between the wireless earphone 101 and the wireless earphone 102 in the wireless earphone 10 can be bidirectional or unidirectional, and no specific limitation is made in this embodiment. In addition, it should be understood that the above-mentioned wireless earphone 10 and the playback device 20 can be wireless transceiver devices that communicate according to a standard wireless protocol. Among them, the standard wireless protocol can be a Bluetooth protocol, a Wifi protocol, a Lifi protocol, an infrared wireless transmission protocol, etc. In this embodiment, the specific form of its wireless protocol is not limited. In order to specifically illustrate the application scenario of the wireless connection method provided in this embodiment, it can be exemplified that the standard wireless protocol is the Bluetooth protocol. Here, the wireless earphone 10 can be a TWS (True Wireless Stereo) true wireless earphone or a traditional Bluetooth earphone, etc.

[0240] Figure 3 It is a schematic flowchart of an audio processing method shown according to an exemplary embodiment of the present invention. As Figure 3As shown in the figure, the audio processing method provided in this embodiment includes:

[0241] S301. Obtain the original audio signal and generate the audio signal to be presented according to the original audio signal.

[0242] In this step, the playback device obtains the original audio signal and performs preprocessing on the original audio signal, which may include at least one preprocessing program such as decoding, rendering, and re-encoding.

[0243] Optionally, after the playback device obtains the original audio signal, it can decode all or part of the original audio signal to obtain audio content data and audio characteristic information. The audio content data may include, but is not limited to, channel content audio signals; the audio characteristic information may include, but is not limited to, sound field type, sampling rate, bit rate information, etc.

[0244] The original audio signal includes channel-based audio signals, such as AAC / AC3 bitstreams, etc., object-based audio signals, such as ATMOS / MPEG-H bitstreams, etc., scene-based audio signals, such as MPEG-H HOA bitstreams, or audio signals that are any combination of the above three types, such as WANOS bitstreams.

[0245] When the original audio signal is a channel-based audio signal, such as an AAC / AC3 bitstream, etc., the audio bitstream is fully decoded to obtain the audio content signals of each channel and channel characteristic information, such as sound field type, sampling rate, bit rate, etc.

[0246] When the original audio signal is an object-based audio signal, such as an ATMOS / MPEG-H bitstream, etc., only the audio bed is decoded to obtain the audio content signals of each channel and channel characteristic information, such as sound field type, sampling rate, bit rate, etc.

[0247] When the original audio signal is a scene-based audio signal, such as an MPEG-H HOA bitstream, the audio bitstream is fully decoded to obtain the audio content signals of each channel and channel characteristic information, such as sound field type, sampling rate, bit rate, etc.

[0248] When the original audio signal is a bitstream based on the above three types of signals, such as a WANOS bitstream, the audio bitstream is decoded according to the decoding description of the bitstreams of the above three types of signals to obtain the audio content signals of each channel and channel characteristic information, such as sound field type, sampling rate, bit rate, etc.

[0249] Optionally, the playback device may perform rendering processing on the decoded audio content data to obtain a rendered audio signal and metadata. The audio content may include, but is not limited to, audio content signals of channels and audio content signals of objects; the metadata may include, but is not limited to, channel characteristic information such as sound field type, sampling rate, bit rate, etc., three-dimensional spatial information of objects, and rendering metadata of wireless headphones, for example, may include, but is not limited to, sensor metadata and HRTF (Head Related Transfer Function) database.

[0250] Figure 4 It is a schematic diagram of the rendering method included in an audio data rendering module provided by an embodiment of the present invention. In this embodiment, the rendering method, as Figure 4 shown, includes, but is not limited to, any combination of the following rendering methods: HRTF rendering, channel rendering, object rendering, scene rendering, etc.

[0251] Figure 5 It is a schematic flowchart of an HRTF rendering method provided by an embodiment of the present invention. As Figure 5 shown, when the decoded audio signal is a channel signal, the specific steps of this rendering method include:

[0252] S501. Obtain the audio signal based on the channel and basic metadata.

[0253] In this step, the audio signal of the channel is the content signal of the channel, which includes the number of channels, and the basic metadata is the basic information of the channel, including information such as sound field type and sampling rate.

[0254] S502. Construct the spatial position distribution (X1, Y1, Z1) of each channel based on the basic metadata.

[0255] In this step, use the basic metadata to construct the spatial distribution of each channel according to a preset algorithm.

[0256] S503. After receiving the rendering metadata, perform a rotation transformation on the spatial distribution of each channel to obtain the spatial distribution (X2, Y2, Z2) in the new coordinate system, and convert it into spatial polar coordinates (ρ1, α1, β1) centered on the human head.

[0257] In this step, receive the sensor metadata from the sensor in the rendering metadata and perform a rotation transformation on the spatial distribution of each channel. The specific coordinate conversion method can be calculated according to the conversion method between the general Cartesian coordinate system and the polar coordinate system, and will not be elaborated here.

[0258] S504. Based on polar coordinates, select the filter coefficients HRTF(i) corresponding to the angles from the HRTF database, and filter the audio signal based on the sound channels to obtain the filtered audio data.

[0259] In this step, according to the distance and angle information in the polar coordinates (ρ1, α1, β1), select the corresponding filter array HRTF(i) from the HRTF database data, and then filter the audio signals of each sound channel.

[0260] S505. Perform downmix processing on the filtered audio signal to obtain the binaural signal after HRTF virtualization.

[0261] In this step, by performing downmix processing on the filtered audio signal, the audio signals of the left and right wireless earphones, that is, the binaural signals, can be obtained.

[0262] It should be noted that the sensor metadata can be provided by a combination of a gyroscope sensor, a geomagnetic device, and an accelerometer; the HRTF database can be based on, but not limited to, other sensor metadata on the wireless earphones, such as a head size sensor, or can be based on a front-end device with a camera or photographing function to perform intelligent recognition of the human head, and then perform personalized processing and adjustment according to the physical characteristics of the listener's head, ears, etc. to achieve a personalized effect; the HRTF database can be pre-stored in the wireless earphones, or new HRTF databases can be imported into it later through wired or wireless means to update the HRTF database to achieve the above-mentioned personalized purpose.

[0263] It should also be noted that due to the limited accuracy of the HRTF database, interpolation can be considered to obtain the HRTF data group corresponding to the angle during calculation; in addition, subsequent processing steps can be further added after S505, including but not limited to equalization (EQ), delay, reverberation, etc.

[0264] Figure 6 This is a schematic flowchart of another HRTF rendering method provided by an embodiment of the present invention. As Figure 6 shown, when the decoded audio signal is an object signal, the specific steps of this rendering method include:

[0265] S601. Obtain the object-based audio signal and the spatial coordinates (X3, Y3, Z3) of the object.

[0266] S602. After receiving the rendering metadata, perform a rotation transformation on the spatial distribution of each sound channel to obtain the spatial distribution (X4, Y4, Z4) in the new coordinate system, and convert it into spatial polar coordinates (ρ2, α2, β2) centered on the human head.

[0267] S603. Based on polar coordinates, select the filter coefficient HRTF(k) corresponding to the angle from the HRTF database, and filter the object-based audio signal to obtain the filtered audio data.

[0268] S604. Perform downmix processing on the filtered audio data to obtain the binaural signal after HRTF virtualization.

[0269] The steps and noun concepts of S601 - S604 are similar to those of S501 - S505 and can be understood by comparison, so they will not be elaborated here.

[0270] For Figure 4 the channel rendering shown, the playback device can perform rendering processing on all or part of the channel audio signals, and the processing methods include but are not limited to downmixing of the number of channels (such as downmixing from 7.1 to 5.1), downmixing of the channel dimension (such as downmixing from 5.1.4 to 5.1), etc.

[0271] For Figure 4 the object rendering shown, the playback device can perform rendering processing on all or part of the input object audio signals, and render the object audio content to the specified position and the specified number of channels according to the metadata of the object, so that it becomes a channel audio signal.

[0272] For Figure 4 the scene rendering shown, the playback device can perform rendering processing on all or part of the input scene audio signals, and render the scene audio signals to the specified output channels according to the specified number of input channels and output channels, so that it becomes a channel audio signal.

[0273] Further optionally, the playback device can re - encode the rendered audio data and the rendered metadata, and output the encoded audio stream as the audio signal to be presented, and transmit it wirelessly to the wireless headset.

[0274] S302. The playback device sends the audio signal to be presented to the wireless headset through wireless transmission.

[0275] In this step, the audio signal to be presented includes the first audio signal and / or the second audio signal, where the first audio signal is the audio signal after rendering processing on the playback device, and the second audio signal is the audio signal to be rendered.

[0276] It should be noted that the first audio signal is the audio signal that has completed the rendering processing on the playback device, while the second audio signal is the signal that the playback device has not performed rendering processing on, and further rendering processing needs to be performed by the headset.

[0277] Specifically, in a possible design, if the audio signal to be presented only includes the first audio signal, the wireless headset directly plays the first audio signal. Because some high-quality sound source data, such as lossless music, already has a high sound quality or already includes the corresponding rendering effect, there is no need for the headset to perform further rendering processing. Furthermore, in some application scenarios, users rarely make violent head movements when using wireless headsets, and the demand for rendering is not high, so there is no need for wireless headsets to perform rendering.

[0278] In a possible design, if the audio signal to be imaged includes a second audio signal, the wireless headset needs to execute S303 to render the second audio signal.

[0279] It should be noted that the purpose of rendering processing is to make the sound present stereo surround sound and panoramic sound effects, increase the spatial sense of the sound, and simulate the effect of people's sense of sound direction, such as being able to distinguish the direction of a car, and whether the car is approaching or moving away at high speed.

[0280] Furthermore, in a possible design, the wireless headset receives the audio signal to be presented sent by the playback device through wireless transmission. When the audio signal to be presented is a compressed code stream, the wireless headset decodes the audio signal to be presented to obtain the first audio signal and / or the second audio signal. That is, the audio signal to be presented needs to be decoded to obtain the first audio signal and / or the second audio signal.

[0281] It should be noted that the decoded first audio signal or second audio signal contains audio content data and audio characteristic information, wherein the audio content data may include but is not limited to the channel content audio signal; the audio characteristic information may include but is not limited to the sound field type, sampling rate, bit rate information, etc.

[0282] It should also be noted that the wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, LIFI visible light communication. Those skilled in the art can select a specific wireless transmission method according to actual conditions, not limited to the above, or select a combination of several wireless transmission methods to achieve the function of information interaction between the playback device and the wireless headset.

[0283] S303: If the audio signal to be presented includes the second audio signal, render the second audio signal to obtain a third audio signal.

[0284] In this step, the audio signal to be presented includes the second audio signal, which means that the audio signal to be presented only includes the second audio signal, or the audio signal to be presented includes both the first audio signal and the second audio signal.

[0285] Figure 7 This is a schematic diagram of the data stream for audio signal rendering on the wireless headphone side provided by the embodiments of the present invention. As Figure 7 shown, the audio signal 71 to be presented includes at least one of the first audio signal 721 and the second audio signal 722, and the second audio signal 722 must be rendered by the wireless headphones to be played as the subsequent playback audio 74 or a part of the subsequent playback audio 74.

[0286] It should be noted that the rendering processing of the playback device and the wireless headphones in this embodiment includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0287] When the wireless headphones are traditional wireless Bluetooth headphones, that is, the two headphones are connected by wire and share relevant sensors, processing units, etc. At this time, the rendering method is as follows:

[0288] The second audio signal includes audio content data and audio characteristic information. Render the audio content to obtain the rendered audio signal and metadata. The audio content may include, but is not limited to, the audio content signal of the channel and the audio content signal of the object; the metadata may include, but is not limited to, channel characteristic information, such as sound field type, sampling rate, bit rate, etc., and the three-dimensional spatial information of the object, as well as the rendering metadata on the wireless headphone side, for example, may include, but is not limited to, sensor metadata and HRTF database.

[0289] The specific rendering process is the same as the rendering principle of the playback device, and reference can be made to Figure 5 and Figure 6 the HRTF rendering shown, and other rendering methods of the playback device introduced in S302.

[0290] Optionally, the rendering process of the second audio signal to obtain the third audio signal includes:

[0291] Render the second audio signal according to the rendering metadata to obtain the third audio signal, where the rendering metadata includes the first metadata and the second metadata, the first metadata is the metadata on the playback device side, and the second metadata is the metadata on the wireless headphone side.

[0292] The so-called metadata is information that describes the attributes of data. The first metadata is used to represent the current motion state of the playback device, the signal transmission intensity of the playback device, the signal propagation direction, the distance between the playback device and the wireless earphones or the relative motion state, etc.; the second metadata is used to represent the motion state of the wireless earphones. For example, when a person's head swings or shakes, it will cause the wireless earphones to move accordingly. And the second metadata can also include information such as the relative motion distance, relative motion speed, and acceleration between the left and right wireless earphones. The first metadata and the second metadata together provide a rendering basis for achieving high-quality surround sound or panoramic sound effects. For example, when a user is playing a first-person shooter game using a virtual reality device, while turning the head left and right to observe, the user also needs to listen for whether an enemy is approaching, or determine the position of the enemy through the attached gunfight sounds. In order to more realistically render the ambient sound at this time, it is necessary to combine the second metadata of the wireless earphones and the first metadata of the playback device worn on the user or the playback device placed in the room, and provide it to the wireless earphones and / or the playback device to comprehensively render the original audio data to achieve a realistic and high-quality sound playback effect.

[0293] In a possible implementation manner, the first metadata includes first sensor metadata, where the first sensor metadata is used to characterize the motion characteristics of the playback device; and / or,

[0294] The second metadata includes second sensor metadata and a head-related transfer function (HRTF) database, where the second sensor metadata is used to characterize the motion characteristics of the wireless earphones.

[0295] Specifically, the first metadata can be detected by a first sensor. The first sensor can be located on the playback device, on the wireless earphones, or on other objects worn on the user, such as a smart bracelet or a smart watch. As Figure 5 shown, in the audio signal rendering stage of the playback device, the first metadata is the Figure 5 sensor metadata in, and in the audio signal rendering stage of the wireless earphones, the second sensor metadata is the Figure 5 sensor metadata in, and the head-related transfer function (HRTF) database is the Figure 5 HRTF database data in. That is, the first metadata is used for the rendering of the playback device, and the second metadata is used for the rendering of the wireless earphones.

[0296] Optionally, the first sensor metadata is obtained through a first sensor, and the first sensor includes at least one of a gyroscope sensor, a head size sensor, a ranging sensor, a geomagnetic sensor, and an acceleration sensor; and / or,

[0297] The second sensor metadata is obtained by a second sensor, and the second sensor includes at least one of a gyroscope sensor, a head size sensor, a ranging sensor, a geomagnetic sensor, and an acceleration sensor.

[0298] In a possible design, the wireless earphones include a first wireless earphone and a second wireless earphone;

[0299] The second sensor is provided in the first wireless earphone or the second wireless earphone; or,

[0300] The second sensor is provided in both the first wireless earphone and the second wireless earphone. After the second sensor metadata is obtained by the first wireless earphone and the second wireless earphone respectively, the second sensor metadata is synchronized with each other.

[0301] S304. Perform subsequent audio playback according to the first audio signal and / or the third audio signal.

[0302] In this step, the wireless earphones play the first audio signal and / or the third audio signal. Specifically, when only the first audio signal is included, that is, the part of the audio signal to be presented transmitted by the playback device that does not need to be rendered in the wireless earphones, it is directly played; when only the third audio signal is included, that is, all the audio signals to be presented transmitted by the playback device need to be rendered in the wireless earphones to obtain the third audio signal, and then played by the wireless earphones; when both the first audio signal and the third audio signal are included, the wireless earphones need to combine the two according to a preset combination algorithm, and then play the combined audio signal. In the present invention, the combination algorithm is not limited, and those skilled in the art can select a suitable implementation manner of the combination algorithm according to the specific application scenario.

[0303] This embodiment provides an audio processing method. First, the wireless earphone end receives the audio signal to be presented sent by the playback device through a wireless transmission method. The audio signal to be presented includes the audio signal after rendering processing by the playback device, that is, the first audio signal, and the audio signal to be rendered, that is, the second audio signal. Then, if the audio signal to be presented includes the second audio signal, the wireless earphone end performs rendering processing on the second audio signal to obtain the third audio signal. Finally, the wireless earphone end performs subsequent audio playback according to the first audio signal and / or the third audio signal. Thus, the technical effect that the wireless earphones can present high-quality surround sound and panoramic sound effects is achieved.

[0304] Figure 8 It is a schematic flowchart of another audio processing method provided by an embodiment of the present invention. As Figure 8 shown, the specific steps of this method include:

[0305] S801. Obtain the original audio signal.

[0306] In this step, the playback device obtains the original audio signal from a resource library such as an internal memory, a database, or the Internet.

[0307] S802. The wireless earphone sends an indication signal to the playback device through a wireless transmission method.

[0308] In this step, the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the audio signal to be presented. The role of the indication signal is to indicate the rendering processing ability of the wireless earphone. For example, when the battery power of the wireless earphone itself is relatively sufficient, its processing ability is relatively strong. Then, during the handshake stage between the wireless earphone and the playback device, that is, during the establishment of a wireless connection stage, the playback device is instructed to allocate a relatively high proportion of rendering tasks to the wireless earphone; when the battery power carried by the wireless earphone itself is less, its processing ability is weaker, or in order to keep the wireless earphone working for a longer time, that is, in the power-saving mode, at this time, the wireless earphone instructs the playback device to allocate a lower proportion of rendering tasks, or not to allocate rendering tasks to the wireless earphone.

[0309] In a possible design, the wireless earphone sends the performance parameters of the wireless earphone through a wireless transmission method. After receiving the performance parameters of the wireless earphone, the playback device obtains the indication signal by querying the mapping relationship table between the performance parameters and the indication signal, or calculates the indication signal according to the performance parameters using a preset algorithm.

[0310] S803. Render the original audio signal according to the indication signal according to a corresponding preset processing method to obtain the audio signal to be presented.

[0311] In a possible design, the indication signal includes an identification code;

[0312] Among them, if the identification code is the first field, the playback device does not render the original audio signal, the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the wireless earphone renders all of the original audio signal;

[0313] If the identification code is the second field, the playback device renders all of the original audio signal, the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the wireless earphone does not render the original audio signal;

[0314] If the identification code is the third field, the playback device renders a part of the original audio signal, which includes the first audio signal and the second audio signal, and the wireless earphone renders the remaining part of the original audio signal.

[0315] The indication information can be sent from the wireless earphone to the playback device when the wireless earphone is first connected to the playback device, so that the processing resources of the playback device or the wireless earphone do not need to be consumed subsequently.

[0316] It can be understood that the indication information can also be triggered and transmitted periodically to facilitate changes according to different playback contents, so that the sound quality of the wireless earphone can be dynamically adjusted.

[0317] The indication information can also be triggered and transmitted according to user instructions received by sensors in the wireless earphone.

[0318] To facilitate the description of the function of the indication signal, the following will be combined with Figure 9 for illustration.

[0319] Figure 9 FIG. is a schematic diagram of the data link of the audio processing signal provided by the embodiment of the present invention in the playback device and the wireless earphone. As Figure 9 shown, starting from obtaining the original audio signal S0 from the playback device to the playback device outputting the signal S3 to be presented, the function of the indication signal is to guide the data flow direction of the original audio signal S0.

[0320] The original audio signal S0 includes a fourth audio signal S01 and / or a fifth audio signal S02. Among them, the fourth audio signal S01 is used to generate the first audio signal S40 after processing, and the fifth audio signal S02 is used to generate the second audio signal S41;

[0321] After the playback device obtains the original audio signal S0, it decodes the fourth audio signal S01 to obtain a sixth audio signal S1, and the sixth audio signal S1 includes a seventh audio signal S11 and / or an eighth audio signal S12;

[0322] Render the seventh audio signal S11 to obtain a ninth audio signal S2;

[0323] Encode the eighth audio signal S12 and the ninth audio signal S2 to obtain a tenth audio signal S30, and the audio signal to be presented includes the fifth audio signal S02 and the tenth audio signal S30;

[0324] Among them, rendering the seventh audio signal S11 includes:

[0325] Render the seventh audio signal S11 according to the rendering metadata to obtain the ninth audio signal S2, where the rendering metadata includes first metadata D3 and second metadata D5, the first metadata D3 is the metadata of the playback device side, and the second metadata D5 is the metadata of the wireless headset side.

[0326] In Figure 9 In the audio signal transmission link shown, there can be multiple data links from the original audio signal to the subsequent played audio, or there can be only one data link. The indication signal and / or the original audio signal determine the specific usage of the data link.

[0327] S804. The playback device sends the audio signal to be presented to the wireless headset through wireless transmission.

[0328] S805. If the audio signal to be presented includes the second audio signal, render the second audio signal to obtain the third audio signal.

[0329] S806. Perform subsequent audio playback according to the first audio signal and / or the third audio signal

[0330] In this embodiment, steps S804 - S805 are similar to S302 - S304 of the Figure 3 audio processing method shown and will not be elaborated here.

[0331] This embodiment provides an audio processing method. First, the wireless headset side receives the audio signal to be presented sent by the playback device through wireless transmission. The audio signal to be presented includes the audio signal after rendering processing by the playback device, i.e., the first audio signal, and the audio signal to be rendered, i.e., the second audio signal. Then, if the audio signal to be presented includes the second audio signal, the wireless headset side renders the second audio signal to obtain the third audio signal. Finally, the wireless headset side performs subsequent audio playback according to the first audio signal and / or the third audio signal. Thus, the technical effect that the wireless headset can present high-quality surround sound and panoramic sound effects is achieved.

[0332] Figure 10 It is a schematic flowchart of another audio processing method provided by an embodiment of the present invention. As Figure 10 shown, the specific steps of this method include:

[0333] S1001. Obtain the original audio signal and generate the audio signal to be presented according to the original audio signal.

[0334] In this step, the playback device acquires the original audio signal, which may include lossless music, game audio, movie audio, etc. Then, the playback device performs at least one of decoding, rendering, and re-encoding on the original audio signal. For possible implementation manners of this step S1001, please refer to the description of the data link distribution of the playback device part in S803, which will not be elaborated here. Figure 9 in the description of the data link distribution of the playback device part, which will not be elaborated here.

[0335] S10021. The first wireless earphone receives the first audio signal to be presented sent by the playback device.

[0336] S10022. The second wireless earphone receives the second audio signal to be presented sent by the playback device.

[0337] In this embodiment, the wireless earphones include a first wireless earphone and a second wireless earphone. Among them, the first wireless earphone and the second wireless earphone are used to establish a wireless connection with the playback device.

[0338] It should be noted that S10021 and S10022 can occur simultaneously, and there is no limitation on the order.

[0339] S10031. The first wireless earphone performs a rendering process on the first audio signal to be presented to obtain a first playback audio signal.

[0340] S10032. The second wireless earphone performs a rendering process on the second audio signal to be presented to obtain a second playback audio signal.

[0341] It should be noted that S10031 and S10032 can occur simultaneously, and there is no limitation on the order.

[0342] Optionally, before S1021, it further includes:

[0343] The first wireless earphone performs a decoding process on the first audio signal to be presented to obtain a first decoded audio signal;

[0344] Correspondingly, the first wireless earphone performing a rendering process on the first audio signal to be presented includes:

[0345] The first wireless earphone performs a rendering process according to the first decoded audio signal and the rendering metadata to obtain the first playback audio signal.

[0346] Before S1022, it further includes:

[0347] The second wireless earphone performs a decoding process on the second audio signal to be presented to obtain a second decoded audio signal;

[0348] Correspondingly, the second wireless earphone performs rendering processing on the second audio signal to be presented, including:

[0349] The second wireless earphone performs rendering processing according to the second decoded audio signal and the rendering metadata to obtain the second playback audio signal.

[0350] Optionally, the rendering metadata includes at least one of the first wireless earphone metadata, the second wireless earphone metadata, and the playback device metadata.

[0351] Optionally, the first wireless earphone metadata includes the first earphone sensor metadata and the head-related transfer function (HRTF) database, where the first earphone sensor metadata is used to characterize the motion characteristics of the first wireless earphone;

[0352] The second wireless earphone metadata includes the second earphone sensor metadata and the head-related transfer function (HRTF) database, where the second earphone sensor metadata is used to characterize the motion characteristics of the second wireless earphone;

[0353] The playback device metadata includes the playback device sensor metadata, where the playback device sensor metadata is used to characterize the motion characteristics of the playback device.

[0354] Optionally, before performing the rendering processing, it further includes:

[0355] The first wireless earphone synchronizes the rendering metadata with the second wireless earphone.

[0356] Optionally, if there is an earphone sensor on the first wireless earphone, there is no earphone sensor on the second wireless earphone, and there is no playback device sensor on the playback device, then the first wireless earphone synchronizes the rendering metadata with the second wireless earphone, including:

[0357] The first wireless earphone sends the first earphone sensor metadata to the second wireless earphone, and the second wireless earphone uses the first earphone sensor metadata as the second earphone sensor metadata.

[0358] If there are earphone sensors on both the first wireless earphone and the second wireless earphone, and there is no playback device sensor on the playback device, then the first wireless earphone synchronizes the rendering metadata with the second wireless earphone, including:

[0359] The first wireless earphone sends the first earphone sensor metadata to the second wireless earphone, and the second wireless earphone sends the second earphone sensor metadata to the first wireless earphone;

[0360] The first wireless earphone and the second wireless earphone respectively determine the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, and a preset numerical algorithm; or,

[0361] The first wireless earphone sends the first earphone sensor metadata to the playback device, and the second wireless earphone sends the second earphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, and a preset numerical algorithm;

[0362] The first wireless earphone and the second wireless earphone respectively receive the rendering metadata.

[0363] In a possible design, a headset sensor is provided on the first wireless earphone, no headset sensor is provided on the second wireless earphone, and a playback device sensor is provided on the playback device. Then, for the first wireless earphone and the second wireless earphone to synchronize the rendering metadata, it includes:

[0364] The first wireless earphone sends the first earphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0365] The first wireless earphone and the second wireless earphone respectively receive the rendering metadata; or,

[0366] The first wireless earphone receives the playback device sensor metadata sent by the playback device;

[0367] The first wireless earphone determines the rendering metadata according to the first earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0368] The first wireless earphone sends the rendering metadata to the second wireless earphone.

[0369] In another possible design, if headset sensors are provided on both the first wireless earphone and the second wireless earphone, and a playback device sensor is provided on the playback device, then for the first wireless earphone and the second wireless earphone to synchronize the rendering metadata, it includes:

[0370] The first wireless earphone sends the first earphone sensor metadata to the playback device, and the second wireless earphone sends the second earphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0371] The first wireless earphone and the second wireless earphone respectively receive the rendering metadata; or,

[0372] The first wireless earphone sends the first earphone sensor metadata to the second wireless earphone, and the second wireless earphone sends the second earphone sensor metadata to the first wireless earphone;

[0373] The first wireless earphone and the second wireless earphone respectively receive the playback device sensor metadata;

[0374] The first wireless earphone and the second wireless earphone respectively determine the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm.

[0375] Specifically, when the wireless earphone is a TWS true wireless earphone, that is, the two earphones are separated and coupled wirelessly, and the two earphones can each have their own processing units and sensors, etc. Then the first wireless earphone is the left earphone, and the second wireless earphone is the right earphone. At this time, the synchronous rendering method of the first wireless earphone and the second wireless earphone is as follows:

[0376] Figure 11 It is a schematic diagram of the rendering process of the TWS true wireless earphone provided by the embodiment of the present invention regarding channel information.

[0377] For the introduction of the steps of S1101 to S1110, refer to Figure 4 the HRTF rendering method shown, which will not be elaborated here. It should be noted that the sensor metadata of the first wireless earphone and the second wireless earphone can cooperate with each other to adjust the data synchronization of the two earphones to achieve a better sound effect.

[0378] S10041. The first wireless earphone plays the first playback audio signal.

[0379] S10042. The second wireless earphone plays the second playback audio signal.

[0380] It should be noted that S10041 and S10042 can occur simultaneously, and the order is not limited.

[0381] In a possible design, the audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

[0382] It should be noted that the rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0383] It should also be noted that the wireless transmission methods include: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

[0384] In addition, in a possible design, a playback device can also be connected to multiple pairs of wireless earphones simultaneously. At this time, the audio information rendering allocation for multiple pairs of wireless earphones can still be performed in the manner of the above-mentioned embodiments, and the rendering division ratio between different playback devices and wireless earphones can be correspondingly matched according to the processing capabilities of different wireless earphones. Optionally, multiple pairs of wireless earphones can also comprehensively schedule the rendering processing resources among the pairs of wireless earphones by the playback device. That is, for wireless earphones with relatively weak processing capabilities, other wireless earphones with strong processing capabilities connected to the same playback device can be called to assist in audio information rendering.

[0385] The audio processing method provided in this embodiment first receives the first audio signal to be presented and the second signal to be presented sent by the playback device through wireless transmission methods at the first wireless earphone and the second wireless earphone ends respectively, then performs rendering processing respectively to obtain the first playback audio signal and the second playback audio signal, and finally the first wireless earphone and the second wireless earphone play the corresponding playback audio signals respectively. Thereby, the technical effect of reducing the delay caused by the rendering data interaction between the wireless earphone and the playback device and improving the headphone sound effect is achieved.

[0386] Figure 12 It is a schematic structural diagram of an audio processing device provided in an embodiment of the present invention. As Figure 12 shown, the audio processing device 1200 provided in this embodiment includes:

[0387] An acquisition module, configured to receive the audio signal to be presented sent by the playback device through a wireless transmission method, where the audio signal to be presented includes a first audio signal and / or a second audio signal, and wherein the first audio signal is the audio signal after rendering processing at the playback device, and the second audio signal is the audio signal to be rendered;

[0388] A rendering module, configured to perform rendering processing on the second audio signal when the audio signal to be presented includes the second audio signal to obtain a third audio signal;

[0389] A playback module for subsequent audio playback based on the first audio signal and / or the third audio signal.

[0390] In a possible design, before the receiving module receives the to-be-rendered audio signal sent by the playback device through wireless transmission, it further includes:

[0391] A sending module for sending an indication signal to the playback device through wireless transmission, where the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the to-be-rendered audio signal.

[0392] In a possible design, before the sending module sends an indication signal to the playback device through wireless transmission, it further includes:

[0393] The obtaining module is further configured to obtain performance parameters of the wireless headset and determine the indication signal according to the performance parameters.

[0394] In a possible design, before the sending module sends an indication signal to the playback device through wireless transmission, it further includes:

[0395] The obtaining module is further configured to receive audio characteristic information sent by the playback device, where the audio characteristic information includes characteristic parameters of the original audio signal input to the playback device, and the characteristic parameters include at least one of a bitstream format, a channel parameter, an object parameter, and a scene component parameter.

[0396] In a possible design, the indication signal includes an identification code;

[0397] Wherein, if the identification code is a first field, the playback device does not render the original audio signal, the to-be-rendered audio signal includes the second audio signal and does not include the first audio signal, and the wireless headset fully renders the original audio signal;

[0398] If the identification code is a second field, the playback device fully renders the original audio signal, the to-be-rendered audio signal includes the first audio signal and does not include the second audio signal, and the wireless headset does not render the original audio signal;

[0399] If the identification code is a third field, the playback device partially renders the original audio signal, which includes the first audio signal and the second audio signal, and the wireless headset renders the remaining part of the original audio signal.

[0400] In a possible design, it is characterized in that, after the obtaining module is used to receive the audio signal to be presented sent by the playback device through wireless transmission, it further includes:

[0401] A decoding module, configured to perform decoding processing on the audio signal to be presented to obtain the first audio signal and / or the second audio signal.

[0402] In a possible design, the rendering module, configured to perform rendering processing on the second audio signal to obtain a third audio signal, includes:

[0403] The rendering module is configured to perform rendering processing on the second audio signal according to rendering metadata to obtain the third audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless headset end.

[0404] In a possible design, the first metadata includes first sensor module metadata, where the first sensor module metadata is used to characterize the motion characteristics of the playback device; and / or,

[0405] The second metadata includes second sensor module metadata and a head-related transfer function (HRTF) database, where the second sensor module metadata is used to characterize the motion characteristics of the wireless headset.

[0406] In a possible design, the headset sensor metadata is obtained through a first sensor module, and the first sensor module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module; and / or,

[0407] The playback device sensor metadata is obtained through a second sensor module, and the second sensor module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module.

[0408] In a possible design, the audio processing device includes a first audio processing device and a second audio processing device;

[0409] The second sensor sub-module is provided in the first audio processing device or the second audio processing device; or,

[0410] The second sensor sub-module is provided in both the first audio processing device and the second audio processing device. After the obtaining modules of the first audio processing device and the second audio processing device obtain the playback device sensor metadata, it further includes:

[0411] A synchronization module for synchronizing the sensor metadata of the playback device with each other.

[0412] In a possible design, the first audio processing device includes:

[0413] A first receiving module for receiving a first audio signal to be presented sent by the playback device;

[0414] A first rendering module for performing rendering processing on the first audio signal to be presented to obtain a first playback audio signal;

[0415] A first playback module for playing the first playback audio signal;

[0416] The second audio processing device includes:

[0417] A second receiving module for receiving a second audio signal to be presented sent by the playback device;

[0418] A second rendering module for performing rendering processing on the second audio signal to be presented to obtain a second playback audio signal;

[0419] A second playback module for playing the second playback audio signal.

[0420] In a possible design, the first audio processing device further includes:

[0421] A first decoding module for performing decoding processing on the first audio signal to be presented to obtain a first decoded audio signal;

[0422] The first rendering module is specifically configured to: perform rendering processing according to the first decoded audio signal and the rendering metadata to obtain the first playback audio signal;

[0423] The second audio processing device further includes:

[0424] A second decoding module for performing decoding processing on the second audio signal to be presented to obtain a second decoded audio signal;

[0425] The second rendering module is specifically configured to: perform rendering processing according to the second decoded audio signal and the rendering metadata to obtain the second playback audio signal.

[0426] In a possible design, the rendering metadata includes at least one of first wireless headphone metadata, second wireless headphone metadata, and playback device metadata.

[0427] In a possible design, the first wireless headphone metadata includes first headphone sensor metadata and a head-related transfer function (HRTF) database, where the first headphone sensor metadata is used to characterize the motion characteristics of the first wireless headphone;

[0428] The second wireless headphone metadata includes second headphone sensor metadata and a head-related transfer function (HRTF) database, where the second headphone sensor metadata is used to characterize the motion characteristics of the second wireless headphone;

[0429] The playback device metadata includes playback device sensor metadata, where the playback device sensor metadata is used to characterize the motion characteristics of the playback device.

[0430] In a possible design, the first audio processing device further includes:

[0431] A first synchronization module for synchronizing the rendering metadata with the second wireless headphone; and / or,

[0432] The second audio processing device further includes:

[0433] A second synchronization module for synchronizing the rendering metadata with the first wireless headphone.

[0434] In a possible design, the first synchronization module is specifically configured to: send the first headphone sensor metadata to the second wireless headphone, so that the second synchronization module uses the first headphone sensor metadata as the second headphone sensor metadata.

[0435] In a possible design, the first synchronization module is specifically configured to:

[0436] Send the first headphone sensor metadata;

[0437] Receive the second headphone sensor metadata;

[0438] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm;

[0439] The second synchronization module is specifically configured to:

[0440] Send the second headphone sensor metadata;

[0441] Receive the first headphone sensor metadata;

[0442] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm; or,

[0443] The first synchronization module is specifically configured to:

[0444] Send the first headphone sensor metadata;

[0445] Receive the rendering metadata;

[0446] The second synchronization module is specifically configured to:

[0447] Send the second headphone sensor metadata;

[0448] Receive the rendering metadata.

[0449] In a possible design, the first synchronization module is specifically configured to:

[0450] Receive playback device sensor metadata;

[0451] Determine the rendering metadata according to the first headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0452] Send the rendering metadata.

[0453] In a possible design, the first synchronization module is specifically configured to:

[0454] Send the first headphone sensor metadata;

[0455] Receive the second headphone sensor metadata;

[0456] Receive the playback device sensor metadata;

[0457] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm;

[0458] The second synchronization module is specifically configured to:

[0459] Send the second headphone sensor metadata;

[0460] Receive the first headphone sensor metadata;

[0461] Receive the playback device sensor metadata;

[0462] Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm.

[0463] Optionally, the audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

[0464] Optionally, the rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0465] Optionally, the wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

[0466] It should be noted that Figure 12 The audio processing device provided in the illustrated embodiment can execute the method corresponding to the wireless headphone end provided in any of the above method embodiments. The specific implementation principles, technical features, professional term explanations, and technical effects are similar and will not be elaborated here.

[0467] Figure 13 FIG. is a schematic structural diagram of another audio processing device provided in an embodiment of the present invention. As Figure 13 shown, the audio processing device 1300 provided in this embodiment includes:

[0468] An acquisition module, configured to receive an original audio signal and generate a to-be-rendered audio signal according to the original audio signal. The to-be-rendered audio signal includes a first audio signal and / or a second audio signal, where the first audio signal is an audio signal after rendering processing by a playback device, and the second audio signal is an audio signal to be rendered;

[0469] A sending module, configured to send the to-be-rendered audio signal to a wireless headphone through a wireless transmission method.

[0470] In a possible design, before the sending module is configured to send the to-be-rendered audio signal to the wireless headphone through a wireless transmission method, it includes:

[0471] The acquisition module is further configured to receive an indication signal sent by the wireless headphone through the wireless transmission method. The indication signal is used to instruct the playback device to perform rendering on the original audio signal according to a corresponding preset processing method to obtain the to-be-rendered audio signal.

[0472] In a possible design, before the sending module is configured to send the to-be-rendered audio signal to the wireless headphone through a wireless transmission method, it further includes:

[0473] The acquisition module is further configured to receive performance parameters of the wireless headphone through the wireless transmission method, and determine an indication signal according to the performance parameters. The indication signal is used to instruct the playback device to perform rendering on the original audio signal according to a corresponding preset processing method to obtain the to-be-rendered audio signal.

[0474] In a possible design, the obtaining module is further configured to receive performance parameters of the wireless headset through the wireless transmission manner, and determine an indication signal according to the performance parameters, including:

[0475] The obtaining module is further configured to obtain characteristic parameters of the original audio signal, where the characteristic parameters include at least one of a bitstream format, a channel parameter, an object parameter, and a scene component parameter;

[0476] The obtaining module is further configured to determine the indication signal according to the characteristic parameters and the performance parameters.

[0477] Optionally, the indication signal includes an identification code;

[0478] Wherein, if the identification code is a first field, the playback device does not render the original audio signal, then the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the wireless headset renders the original audio signal completely;

[0479] If the identification code is a second field, the playback device renders the original audio signal completely, then the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the wireless headset does not render the original audio signal;

[0480] If the identification code is a third field, the playback device renders the original audio signal partially, then the audio signal includes the first audio signal and the second audio signal, and the wireless headset renders the remaining part of the original audio signal.

[0481] Optionally, the original audio signal includes a fourth audio signal and / or a fifth audio signal, where the fourth audio signal is used to generate the first audio signal after processing, and the fifth audio signal is used to generate the second audio signal;

[0482] Correspondingly, after the obtaining module is configured to obtain the original audio signal, it further includes:

[0483] A decoding module, configured to perform decoding processing on the fourth audio signal to obtain a sixth audio signal, where the sixth audio signal includes a seventh audio signal and / or an eighth audio signal;

[0484] A rendering module, configured to perform rendering processing on the seventh audio signal to obtain a ninth audio signal;

[0485] An encoding module, configured to encode the eighth audio signal and the ninth audio signal to obtain a tenth audio signal, and the audio signal to be presented includes the fifth audio signal and the tenth audio signal.

[0486] In a possible design, the rendering module is configured to perform rendering processing on the seventh audio signal, including:

[0487] The rendering module is configured to perform rendering processing on the seventh audio signal according to rendering metadata to obtain the ninth audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless headset end.

[0488] In a possible design, the first metadata includes first sensor sub-module metadata, where the first sensor sub-module metadata is used to characterize the motion characteristics of the playback device; and / or,

[0489] The second metadata includes second sensor sub-module metadata and a head-related transfer function (HRTF) database, where the sensor sub-module metadata is used to characterize the motion characteristics of the wireless headset.

[0490] In a possible design, the first sensor sub-module metadata is obtained through a first sensor sub-module, and the first sensor sub-module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module; and / or,

[0491] The second sensor sub-module metadata is obtained through a second sensor sub-module, and the second sensor sub-module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module.

[0492] Optionally, the audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

[0493] Optionally, the rendering processing includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

[0494] Optionally, the wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

[0495] It should be noted that Figure 13 The audio processing device provided in the illustrated embodiment can execute the method corresponding to the playback device end provided in any of the above method embodiments. The specific implementation principles, technical features, professional term explanations, and technical effects are similar and will not be elaborated here.

[0496] Figure 14 The structural schematic diagram of a wireless earphone provided by the present invention. As Figure 14 shown, the electronic device 1400 may include: at least one processor 1401 and a memory 1402. Figure 14 The electronic device shown takes one processor as an example.

[0497] The memory 1402 is used to store programs. Specifically, the program may include program codes, and the program codes include computer operation instructions.

[0498] The memory 1402 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0499] The processor 1401 is used to execute the computer execution instructions stored in the memory 1402 to implement the methods corresponding to the wireless earphone end described in the above method embodiments.

[0500] Among them, the processor 1401 may be a central processing unit (abbreviated as CPU), or a specific integrated circuit (abbreviated as ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0501] Optionally, the memory 1402 may be either independent or integrated with the processor 1401. When the memory 1402 is a device independent of the processor 1401, the electronic device 1400 may further include:

[0502] A bus 1403 for connecting the processor 1401 and the memory 1402. The bus may be an industry standard architecture (abbreviated as ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0503] Optionally, in a specific implementation, if the memory 1402 and the processor 1401 are integrated on a chip, the memory 1402 and the processor 1401 may communicate through an internal interface.

[0504] Figure 15 This is a schematic structural diagram of another playback device provided by the present invention. As Figure 15 shown, the electronic device 1500 may include: at least one processor 1501 and a memory 1502. Figure 15 The electronic device shown takes one processor as an example.

[0505] The memory 1502 is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions.

[0506] The memory 1502 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0507] The processor 1501 is used to execute the computer execution instructions stored in the memory 502 to implement the methods corresponding to the playback device side described in the above method embodiments.

[0508] Among them, the processor 1501 may be a central processing unit (CPU for short), or an application specific integrated circuit (ASIC for short), or one or more integrated circuits configured to implement the embodiments of the present application.

[0509] Optionally, the memory 1502 may be either independent or integrated with the processor 1501. When the memory 1502 is a device independent of the processor 1501, the electronic device 1500 may further include:

[0510] A bus 1503 for connecting the processor 1501 and the memory 1502. The bus may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0511] Optionally, in a specific implementation, if the memory 1502 and the processor 1501 are integrated on a chip, the memory 1502 and the processor 1501 may communicate through an internal interface.

[0512] The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as USB flash drives, external hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs. Specifically, the computer-readable storage medium stores program instructions for the methods corresponding to the wireless headphone ends in the above embodiments.

[0513] The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as USB flash drives, external hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs. Specifically, the computer-readable storage medium stores program instructions for the methods corresponding to the playback device ends in the above embodiments.

[0514] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An audio processing method, characterized in that, Applied to wireless earphones, the method includes: Receiving, by means of wireless transmission, a to-be-presented audio signal sent by a playback device, where the to-be-presented audio signal includes a first audio signal and / or a second audio signal, and wherein the first audio signal is an audio signal after being rendered and processed by the playback device, and the second audio signal is an audio signal to be rendered; If the to-be-presented audio signal includes the second audio signal, then rendering the second audio signal to obtain a third audio signal; Performing subsequent audio playback according to the first audio signal and / or the third audio signal; Before receiving, by means of wireless transmission, the to-be-presented audio signal sent by the playback device, it includes: Sending, by means of wireless transmission, an indication signal to the playback device, where the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the to-be-presented audio signal; the indication signal includes an identification code; Wherein, if the identification code is a first field, the playback device does not render the original audio signal, the to-be-presented audio signal includes the second audio signal and does not include the first audio signal, and the wireless earphones render all of the original audio signal; If the identification code is a second field, the playback device renders all of the original audio signal, the to-be-presented audio signal includes the first audio signal and does not include the second audio signal, and the wireless earphones do not render the original audio signal; If the identification code is a third field, the playback device performs partial rendering on the original audio signal, the to-be-presented audio signal includes the first audio signal and the second audio signal, and the wireless earphones render the remaining part of the original audio signal.

2. The audio processing method according to claim 1, wherein Before sending, by means of wireless transmission, the indication signal to the playback device, it further includes: Obtaining performance parameters of the wireless earphones and determining the indication signal according to the performance parameters.

3. The audio processing method according to claim 2, characterized in that, Before sending, by means of wireless transmission, the indication signal to the playback device, it further includes: Receiving audio characteristic information sent by the playback device, where the audio characteristic information includes characteristic parameters of the original audio signal input to the playback device, and the characteristic parameters include at least one of: bitstream format, channel parameters, object parameters, and scene component parameters.

4. The audio processing method according to any one of claims 1-3, characterized in that, After receiving, by means of wireless transmission, the to-be-presented audio signal sent by the playback device, it further includes: Performing decoding processing on the to-be-presented audio signal to obtain the first audio signal and / or the second audio signal.

5. The audio processing method according to any one of claims 1-3, characterized in that The rendering the second audio signal to obtain a third audio signal includes: Rendering the second audio signal according to rendering metadata to obtain the third audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless earphones end.

6. The audio processing method according to claim 5, wherein The first metadata includes playback device sensor metadata, where the playback device sensor metadata is used to characterize the motion characteristics of the playback device; and / or, The second metadata includes headphone sensor metadata and a head-related transfer function (HRTF) database, where the headphone sensor metadata is used to characterize the motion characteristics of the wireless headphones.

7. The audio processing method according to claim 6, characterized in that, The headphone sensor metadata is obtained through a headphone sensor, and the headphone sensor includes at least one of a gyroscope sensor, a head size sensor, a distance measurement sensor, a geomagnetic sensor, and an acceleration sensor; and / or, The playback device sensor metadata is obtained through a playback device sensor, and the playback device sensor includes at least one of a gyroscope sensor, a head size sensor, a distance measurement sensor, a geomagnetic sensor, and an acceleration sensor.

8. The audio processing method according to claim 7, wherein The wireless headphones include a first wireless headphone and a second wireless headphone; The headphone sensor is provided in the first wireless headphone or the second wireless headphone; or, If the headphone sensor is provided in both the first wireless headphone and the second wireless headphone, after the headphone sensor metadata is respectively obtained by the first wireless headphone and the second wireless headphone, the headphone sensor metadata is mutually synchronized.

9. The audio processing method according to claim 8, wherein The first wireless headphone and the second wireless headphone are used to establish a wireless connection with the playback device; Receiving the audio signal to be presented sent by the playback device through a wireless transmission method includes: The first wireless headphone receives a first audio signal to be presented sent by the playback device, and the second wireless headphone receives a second audio signal to be presented sent by the playback device; Correspondingly, the rendering process in the wireless headphones includes: The first wireless headphone performs a rendering process on the first audio signal to be presented to obtain a first playback audio signal, and the second wireless headphone performs a rendering process on the second audio signal to be presented to obtain a second playback audio signal; The first wireless headphone plays the first playback audio signal, and the second wireless headphone plays the second playback audio signal.

10. The audio processing method according to claim 9, wherein Before the first wireless headphone performs a rendering process on the first audio signal to be presented, it further includes: The first wireless headphone performs a decoding process on the first audio signal to be presented to obtain a first decoded audio signal; Correspondingly, the first wireless headphone performing a rendering process on the first audio signal to be presented includes: The first wireless headphone performs a rendering process according to the first decoded audio signal and the rendering metadata to obtain the first playback audio signal; and Before the second wireless headphone performs a rendering process on the second audio signal to be presented, it further includes: The second wireless headphone performs a decoding process on the second audio signal to be presented to obtain a second decoded audio signal; Correspondingly, the second wireless headphone performing a rendering process on the second audio signal to be presented includes: The second wireless headphone performs a rendering process according to the second decoded audio signal and the rendering metadata to obtain the second playback audio signal.

11. The audio processing method according to claim 10, characterized in that, The rendering metadata includes at least one of first wireless headphone metadata, second wireless headphone metadata, and playback device metadata.

12. The audio processing method according to claim 11, characterized in that, The first wireless headphone metadata includes first headphone sensor metadata and a head-related transfer function (HRTF) database, wherein the first headphone sensor metadata is used to characterize the motion characteristics of the first wireless headphone. The second wireless headphone metadata includes second headphone sensor metadata and a head-related transfer function (HRTF) database, wherein the second headphone sensor metadata is used to characterize the motion characteristics of the second wireless headphone. The playback device metadata includes playback device sensor metadata, wherein the playback device sensor metadata is used to characterize the motion characteristics of the playback device.

13. The audio processing method according to claim 12, wherein Before performing the rendering process, it further includes: The first wireless headphone and the second wireless headphone synchronize the rendering metadata.

14. The audio processing method according to claim 13, wherein If the first wireless headphone is provided with a headphone sensor, the second wireless headphone is not provided with a headphone sensor, and the playback device is not provided with a playback device sensor, then the first wireless headphone and the second wireless headphone synchronizing the rendering metadata includes: The first wireless headphone sends the first headphone sensor metadata to the second wireless headphone, and the second wireless headphone uses the first headphone sensor metadata as the second headphone sensor metadata.

15. The audio processing method according to claim 13, wherein If both the first wireless headphone and the second wireless headphone are provided with headphone sensors, and the playback device is not provided with a playback device sensor, then the first wireless headphone and the second wireless headphone synchronizing the rendering metadata includes: The first wireless headphone sends the first headphone sensor metadata to the second wireless headphone, and the second wireless headphone sends the second headphone sensor metadata to the first wireless headphone; The first wireless headphone and the second wireless headphone respectively determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm; or, The first wireless headphone sends the first headphone sensor metadata to the playback device, and the second wireless headphone sends the second headphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm; The first wireless headphone and the second wireless headphone respectively receive the rendering metadata.

16. The audio processing method according to claim 13, wherein If the first wireless headphone is provided with a headphone sensor, the second wireless headphone is not provided with a headphone sensor, and the playback device is provided with a playback device sensor, then the first wireless headphone and the second wireless headphone synchronizing the rendering metadata includes: The first wireless headphone sends the first headphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm; The first wireless headphone and the second wireless headphone respectively receive the rendering metadata; or, The first wireless earphone receives the playback device sensor metadata sent by the playback device; The first wireless earphone determines the rendering metadata according to the first earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm; The first wireless earphone sends the rendering metadata to the second wireless earphone.

17. The audio processing method according to claim 13, characterized in that If both the first wireless earphone and the second wireless earphone are provided with earphone sensors, and the playback device is provided with a playback device sensor, then the first wireless earphone and the second wireless earphone synchronize the rendering metadata, including: The first wireless earphone sends the first earphone sensor metadata to the playback device, and the second wireless earphone sends the second earphone sensor metadata to the playback device, so that the playback device determines the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm; The first wireless earphone and the second wireless earphone respectively receive the rendering metadata; or, The first wireless earphone sends the first earphone sensor metadata to the second wireless earphone, and the second wireless earphone sends the second earphone sensor metadata to the first wireless earphone; The first wireless earphone and the second wireless earphone respectively receive the playback device sensor metadata; The first wireless earphone and the second wireless earphone respectively determine the rendering metadata according to the first earphone sensor metadata, the second earphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm.

18. The audio processing method according to any one of claims 1-3, characterized in that The audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

19. The audio processing method according to any one of claims 1-3, characterized in that, The rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

20. The audio processing method according to any one of claims 1-3, characterized in that The wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, LIFI visible light communication.

21. An audio processing method, characterized in that, Applied to a playback device, the method includes: Obtain an original audio signal, and generate an audio signal to be presented according to the original audio signal. The audio signal to be presented includes a first audio signal and / or a second audio signal, where the first audio signal is the audio signal after rendering processing on the playback device, and the second audio signal is the audio signal to be rendered; Send the audio signal to be presented to the wireless earphone through a wireless transmission method; Before sending the audio signal to be presented to the wireless earphone through the wireless transmission method, it includes: Receive an indication signal sent by the wireless earphone through a wireless transmission method. The indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the audio signal to be presented; the indication signal includes an identification code; Wherein, if the identification code is the first field, the playback device does not render the original audio signal, the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the wireless headset renders the entire original audio signal; If the identification code is the second field, the playback device renders the entire original audio signal, the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the wireless headset does not render the original audio signal; If the identification code is the third field, the playback device partially renders the original audio signal, the audio signal to be presented includes the first audio signal and the second audio signal, and the wireless headset renders the remaining part of the original audio signal.

22. The audio processing method according to claim 21, wherein Before sending the audio signal to be presented to the wireless headset via wireless transmission, it further includes: Receiving the performance parameters of the wireless headset via the wireless transmission method, and determining an indication signal according to the performance parameters, where the indication signal is used to instruct the playback device to render the original audio signal according to the corresponding preset processing method to obtain the audio signal to be presented.

23. The audio processing method according to claim 22, wherein The receiving the performance parameters of the wireless headset via the wireless transmission method and determining the indication signal includes: Obtaining the characteristic parameters of the original audio signal, where the characteristic parameters include at least one of bitstream format, channel parameters, object parameters, and scene component parameters; Determining the indication signal according to the characteristic parameters and the performance parameters.

24. The audio processing method according to any one of claims 21-23, characterized in that, The original audio signal includes a fourth audio signal and / or a fifth audio signal, wherein the fourth audio signal is used to generate the first audio signal after processing, and the fifth audio signal is used to generate the second audio signal; Correspondingly, after obtaining the original audio signal, it further includes: Performing decoding processing on the fourth audio signal to obtain a sixth audio signal, where the sixth audio signal includes a seventh audio signal and / or an eighth audio signal; Performing rendering processing on the seventh audio signal to obtain a ninth audio signal; Encoding the eighth audio signal and the ninth audio signal to obtain a tenth audio signal, and the audio signal to be presented includes the fifth audio signal and the tenth audio signal.

25. The audio processing method according to claim 24, wherein The performing rendering processing on the seventh audio signal includes: Performing rendering processing on the seventh audio signal according to the rendering metadata to obtain the ninth audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is the metadata at the playback device end, and the second metadata is the metadata at the wireless headset end.

26. The audio processing method according to claim 25, characterized in that The first metadata includes playback device sensor metadata, where the playback device sensor metadata is used to characterize the motion characteristics of the playback device; and / or, The second metadata includes headset sensor metadata and a head-related transfer function (HRTF) database, where the headset sensor metadata is used to characterize the motion characteristics of the wireless headset.

27. The audio processing method according to claim 26, wherein The headphone sensor metadata is obtained by a headphone sensor, which includes at least one of a gyroscope sensor, a head size sensor, a ranging sensor, a geomagnetic sensor, and an acceleration sensor; and / or, The playback device sensor metadata is obtained by a playback device sensor, which includes at least one of a gyroscope sensor, a head size sensor, a ranging sensor, a geomagnetic sensor, and an acceleration sensor.

28. The audio processing method according to any one of claims 21-23, characterized in that, The audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

29. The audio processing method according to any one of claims 21-23, characterized in that The rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

30. The audio processing method according to any one of claims 21-23, characterized in that, The wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

31. An audio processing device, characterized in that, Including: An acquisition module, configured to receive, via a wireless transmission method, an audio signal to be presented sent by a playback device, where the audio signal to be presented includes a first audio signal and / or a second audio signal. Among them, the first audio signal is an audio signal after rendering processing on the playback device, and the second audio signal is an audio signal to be rendered; A rendering module, configured to, when the audio signal to be presented includes the second audio signal, perform rendering processing on the second audio signal to obtain a third audio signal; A playback module, configured to perform subsequent audio playback according to the first audio signal and / or the third audio signal; A sending module, configured to, before the acquisition module receives, via a wireless transmission method, the audio signal to be presented sent by the playback device, send an indication signal to the playback device via a wireless transmission method, where the indication signal is used to instruct the playback device to render an original audio signal according to a corresponding preset processing method to obtain the audio signal to be presented; the indication signal includes an identification code; Among them, if the identification code is a first field, the playback device has not rendered the original audio signal, the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the audio processing device performs all rendering on the original audio signal; If the identification code is a second field, the playback device performs all rendering on the original audio signal, the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the audio processing device does not perform rendering on the original audio signal; If the identification code is a third field, the playback device performs partial rendering on the original audio signal, the audio signal to be presented includes the first audio signal and the second audio signal, and the audio processing device performs rendering on the remaining part of the original audio signal.

32. The audio processing device according to claim 31, wherein The sending module, before sending the indication signal to the playback device via a wireless transmission method, further includes: The acquisition module is further configured to acquire performance parameters of the wireless headset and determine the indication signal according to the performance parameters.

33. The audio processing device according to claim 32, wherein The sending module, before sending the indication signal to the playback device via a wireless transmission method, further includes: The obtaining module is further configured to receive audio characteristic information sent by the playback device, where the audio characteristic information includes characteristic parameters of the original audio signal input to the playback device, and the characteristic parameters include at least one of a bitstream format, a channel parameter, an object parameter, and a scene component parameter.

34. The audio processing device according to any one of claims 31-33, characterized in that After the obtaining module, which is configured to receive the to-be-presented audio signal sent by the playback device through a wireless transmission method, further includes: A decoding module, configured to perform decoding processing on the to-be-presented audio signal to obtain the first audio signal and / or the second audio signal.

35. The audio processing device according to any one of claims 31 to 33, characterized in that, The rendering module is configured to perform rendering processing on the second audio signal to obtain a third audio signal, including: The rendering module is configured to perform rendering processing on the second audio signal according to rendering metadata to obtain the third audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless headphone end.

36. The audio processing device according to claim 35, wherein, The first metadata includes first sensor module metadata, where the first sensor module metadata is used to characterize the motion characteristics of the playback device; and / or, The second metadata includes second sensor module metadata and a head-related transfer function (HRTF) database, where the second sensor module metadata is used to characterize the motion characteristics of the wireless headphone.

37. The audio processing device according to claim 36, characterized in that, The second sensor module metadata is obtained through a second sensor module, and the second sensor module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module; and / or, The first sensor module metadata is obtained through a first sensor module, and the first sensor module includes at least one of a gyroscope sensor sub-module, a head size sensor sub-module, a ranging sensor sub-module, a geomagnetic sensor sub-module, and an acceleration sensor sub-module.

38. The audio processing device according to claim 37, characterized in that, The audio processing device includes a first audio processing device and a second audio processing device; A second sensor sub-module is provided in the first audio processing device or the second audio processing device; or, A second sensor sub-module is provided in both the first audio processing device and the second audio processing device. After the obtaining modules of the first audio processing device and the second audio processing device obtain the first sensor module metadata, further includes: A synchronization module, configured to synchronize the first sensor module metadata with each other.

39. The audio processing device according to claim 38, characterized in that, The first audio processing device includes: A first receiving module, configured to receive a first to-be-presented audio signal sent by the playback device; A first rendering module, configured to perform rendering processing on the first to-be-presented audio signal to obtain a first playback audio signal; A first playback module, configured to play the first playback audio signal; The second audio processing device includes: A second receiving module, configured to receive a second to-be-presented audio signal sent by the playback device; A second rendering module, configured to perform rendering processing on the second to-be-presented audio signal to obtain a second playback audio signal; A second playback module for playing the second playback audio signal.

40. The audio processing apparatus according to claim 39, characterized in that, The first audio processing device further includes: A first decoding module for decoding the first audio signal to be presented to obtain a first decoded audio signal; The first rendering module is specifically configured to: perform rendering processing according to the first decoded audio signal and rendering metadata to obtain the first playback audio signal; The second audio processing device further includes: A second decoding module for decoding the second audio signal to be presented to obtain a second decoded audio signal; The second rendering module is specifically configured to: perform rendering processing according to the second decoded audio signal and rendering metadata to obtain the second playback audio signal.

41. The audio processing device according to claim 40, characterized in that, The rendering metadata includes at least one of first wireless headphone metadata, second wireless headphone metadata, and playback device metadata.

42. The audio processing device according to claim 41, wherein The first wireless headphone metadata includes first headphone sensor metadata and a head-related transfer function (HRTF) database, where the first headphone sensor metadata is used to characterize the movement characteristics of the first wireless headphone; The second wireless headphone metadata includes second headphone sensor metadata and a head-related transfer function (HRTF) database, where the second headphone sensor metadata is used to characterize the movement characteristics of the second wireless headphone; The playback device metadata includes playback device sensor metadata, where the playback device sensor metadata is used to characterize the movement characteristics of the playback device.

43. The audio processing apparatus according to claim 42, wherein The first audio processing device further includes: A first synchronization module for synchronizing the rendering metadata with the second wireless headphone; and / or, The second audio processing device further includes: A second synchronization module for synchronizing the rendering metadata with the first wireless headphone.

44. The audio processing apparatus according to claim 43, wherein The first synchronization module is specifically configured to: send the first headphone sensor metadata to the second wireless headphone, and the second synchronization module uses the first headphone sensor metadata as the second headphone sensor metadata.

45. The audio processing device according to claim 43, characterized in that, The first synchronization module is specifically configured to: Send the first headphone sensor metadata; Receive the second headphone sensor metadata; Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm; The second synchronization module is specifically configured to: Send the second headphone sensor metadata; Receive the first headphone sensor metadata; Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, and a preset numerical algorithm; or, The first synchronization module is specifically configured to: Send the first headphone sensor metadata; Receive the rendering metadata; The second synchronization module is specifically configured to: Send the second headphone sensor metadata; Receive the rendering metadata.

46. The audio processing apparatus according to claim 43, wherein The first synchronization module is specifically configured to: Receive the playback device sensor metadata; Determine the rendering metadata according to the first headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm; Send the rendering metadata.

47. The audio processing device according to claim 43, characterized in that, The first synchronization module is specifically configured to: Send the first headphone sensor metadata; Receive the second headphone sensor metadata; Receive the playback device sensor metadata; Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm; The second synchronization module is specifically configured to: Send the second headphone sensor metadata; Receive the first headphone sensor metadata; Receive the playback device sensor metadata; Determine the rendering metadata according to the first headphone sensor metadata, the second headphone sensor metadata, the playback device sensor metadata, and a preset numerical algorithm.

48. The audio processing device according to any one of claims 31-33, characterized in that, The audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

49. The audio processing device according to any one of claims 31-33, characterized in that, The rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

50. The audio processing device according to any one of claims 31-33, characterized in that, The wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

51. An audio processing device, characterized in that, It includes: An acquisition module, configured to receive an original audio signal and generate an audio signal to be presented according to the original audio signal. The audio signal to be presented includes a first audio signal and / or a second audio signal, where the first audio signal is an audio signal after rendering by a playback device, and the second audio signal is an audio signal to be rendered; A sending module, configured to send the audio signal to be presented to a wireless headphone through a wireless transmission method; The acquisition module is configured to, before the sending module sends the audio signal to be presented to the wireless headphone through the wireless transmission method, receive an indication signal sent by the wireless headphone through the wireless transmission method. The indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the audio signal to be presented; the indication signal includes an identification code; Wherein, if the identification code is a first field, the playback device does not render the original audio signal, the audio signal to be presented includes the second audio signal and does not include the first audio signal, and the audio processing device performs full rendering on the original audio signal; If the identification code is a second field, the playback device performs full rendering on the original audio signal, the audio signal to be presented includes the first audio signal and does not include the second audio signal, and the audio processing device does not render the original audio signal; If the identification code is a third field, the playback device performs partial rendering on the original audio signal, the audio signal to be presented includes the first audio signal and the second audio signal, and the audio processing device renders the remaining part of the original audio signal.

52. The audio processing device according to claim 51, characterized in that, Before the sending module, which is configured to send the audio signal to be presented to the wireless headphone through the wireless transmission method, further includes: The obtaining module is further configured to receive performance parameters of the wireless earphone by the wireless transmission method, and determine an indication signal according to the performance parameters, where the indication signal is used to instruct the playback device to render the original audio signal according to a corresponding preset processing method to obtain the to-be-presented audio signal.

53. The audio processing device according to claim 52, characterized in that, The obtaining module is further configured to receive performance parameters of the wireless earphone by the wireless transmission method, and determine an indication signal according to the performance parameters, including: The obtaining module is further configured to obtain characteristic parameters of the original audio signal, where the characteristic parameters include at least one of a bitstream format, a channel parameter, an object parameter, and a scene component parameter; The obtaining module is further configured to determine the indication signal according to the characteristic parameters and the performance parameters.

54. The audio processing device according to any one of claims 51-53, characterized in that, The original audio signal includes a fourth audio signal and / or a fifth audio signal, where the fourth audio signal is used to generate the first audio signal after being processed, and the fifth audio signal is used to generate the second audio signal; Correspondingly, after the obtaining module is configured to obtain the original audio signal, it further includes: A decoding module, configured to perform decoding processing on the fourth audio signal to obtain a sixth audio signal, where the sixth audio signal includes a seventh audio signal and / or an eighth audio signal; A rendering module, configured to perform rendering processing on the seventh audio signal to obtain a ninth audio signal; An encoding module, configured to encode the eighth audio signal and the ninth audio signal to obtain a tenth audio signal, where the to-be-presented audio signal includes the fifth audio signal and the tenth audio signal.

55. The audio processing apparatus according to claim 54, wherein, The rendering module is configured to perform rendering processing on the seventh audio signal, including: The rendering module is configured to perform rendering processing on the seventh audio signal according to rendering metadata to obtain the ninth audio signal, where the rendering metadata includes first metadata and second metadata, the first metadata is metadata at the playback device end, and the second metadata is metadata at the wireless earphone end.

56. The audio processing device according to claim 55, characterized in that, The first metadata includes first sensing sub-module metadata, where the first sensing sub-module metadata is used to characterize the motion characteristics of the playback device; and / or, The second metadata includes second sensing sub-module metadata and a head-related transfer function (HRTF) database, where the second sensing sub-module metadata is used to characterize the motion characteristics of the wireless earphone.

57. The audio processing device according to claim 56, characterized in that, The first sensing sub-module metadata is obtained by a first sensing sub-module, and the first sensing sub-module includes at least one of a gyroscope sensing sub-module, a head size sensing sub-module, a ranging sensing sub-module, a geomagnetic sensing sub-module, and an acceleration sensing sub-module; and / or, The second sensing sub-module metadata is obtained by a second sensing sub-module, and the second sensing sub-module includes at least one of a gyroscope sensing sub-module, a head size sensing sub-module, a ranging sensing sub-module, a geomagnetic sensing sub-module, and an acceleration sensing sub-module.

58. The audio processing device according to any one of claims 51 to 53, characterized in that The audio signal to be presented includes at least one of a channel-based audio signal, an object-based audio signal, and a scene-based audio signal.

59. The audio processing device according to any one of claims 51-53, characterized in that, The rendering process includes at least one of binaural virtual rendering, channel signal rendering, object signal rendering, and scene signal rendering.

60. The audio processing device according to any one of claims 51 to 53, characterized in that, The wireless transmission method includes: Bluetooth communication, infrared communication, WIFI communication, and LIFI visible light communication.

61. An audio processing system, characterized in that, Includes: The audio processing device according to claim 31 and the audio processing device according to claim 51.

62. A wireless earphone, characterized in that, Includes: A processor; And A memory for storing a computer program of the processor; Wherein, the processor is configured to implement the audio processing method according to any one of claims 1-20 by executing the computer program.

63. A playback device, characterized in that, Includes: A processor; And A memory for storing a computer program of the processor; Wherein, the processor is configured to implement the audio processing method according to any one of claims 21-30 by executing the computer program.

64. An audio processing system, characterized in that, Includes: The wireless earphone according to claim 62 and the playback device according to claim 63.

65. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the audio processing method according to any one of claims 1-20.

66. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the audio processing method according to any one of claims 21-30.

Images