An audio processing method and an audio playback device

Abstract

The application provides an audio processing method and an audio playing device, relates to the technical field of signal processing and earphones, and is used for reducing the dependence on an audio sending device and improving the effect of spatial audio. The method is applied to an audio playing device, the audio playing device comprises earphones and an earphone case, the earphones and the earphone case are both wirelessly connected with an audio sending device, the earphones and the earphone case are wirelessly connected, and the method comprises the following steps: the earphone case receives an audio signal from the audio sending device; the earphone case performs first rendering on the audio signal to obtain a first spatial audio signal; the earphone case sends the first spatial audio signal to the earphones; and the earphones play a target audio signal, wherein the target audio signal comprises the first spatial audio signal. Since the earphone case has a larger volume than the earphones, the earphone case is used for rendering, the related algorithm in the spatial audio technology does not need to be simplified, the effect of the first spatial audio is improved, and the dependence on the audio sending device is reduced.

Description

Technical Field

[0001] This application relates to signal processing technology and the field of headphones, and more particularly to an audio processing method and an audio playback device. Background Technology

[0002] With the development of smart mobile devices, headphones have become an essential item for people's daily audio listening. Wireless headphones, due to their convenience, are increasingly favored by the market and have even gradually become the mainstream headphone product. Consequently, people's demands for sound quality are also increasing, not only pursuing lossless sound quality but also demanding a greater sense of spatial sound. Currently, spatial audio technology is used to eliminate the in-head effect in audio signals to enhance the spatial sense of audio signals, thus obtaining spatial audio. Spatial audio technology: headphones use the head-related transfer function (HRTF) to simulate the auditory effect of a sound field in the real world. In-head effect: refers to the unnatural effect that users perceive when listening to stereo audio with headphones, as if the sound image is located inside their head. Summary of the Invention

[0003] This application provides an audio processing method and an audio playback device to reduce dependence on audio transmission devices and improve the effect of spatial audio.

[0004] In a first aspect, an audio processing method is provided, applied to an audio playback device, the audio playback device including headphones and a charging case, both headphones and the charging case being wirelessly connected to an audio transmitting device, the method comprising: the charging case receiving an audio signal from the audio transmitting device; the charging case performing a first rendering on the audio signal to obtain a first spatial audio signal; the charging case transmitting the first spatial audio signal to the headphones; and the headphones playing a target audio signal, the target audio signal including the first spatial audio signal.

[0005] In the above technical solution, the earphone case receives an audio signal from an audio transmitting device and performs a first rendering on the audio signal to obtain a first spatial audio signal. The earphone case then sends the first spatial audio signal to the earphones, and the earphones play a target audio signal, which includes the first spatial audio signal. Since the earphones and earphone case are compatible, users typically carry the earphone case with them when using the earphones in practical applications. Furthermore, the earphone case is relatively large compared to the earphones. Using the earphone case for the first rendering of the audio signal eliminates the need to simplify the relevant algorithms in spatial audio technology, thus not reducing the quality of the first spatial audio signal. Moreover, in this process, the earphone case performs the first rendering of the audio signal independently of the audio transmitting device, reducing dependence on the audio transmitting device.

[0006] Secondly, an audio processing method is provided, applied to an audio playback device. The audio playback device includes headphones and an earphone case, both of which are wirelessly connected to an audio transmitting device. The method includes: both headphones and the earphone case receiving audio signals from the audio transmitting device; the earphone case performing a first rendering on the audio signals to obtain a first spatial audio signal; the earphone case sending the first spatial audio signal to the headphones; the headphones performing a second rendering on the audio signals to obtain a second spatial audio signal; and the headphones playing a target audio signal, the target audio signal including the first spatial audio signal and the second spatial audio signal.

[0007] In the above technical solution, the headphones and the headphone case work together to render the audio signal, thus improving the speed of audio signal rendering and reducing rendering latency.

[0008] In any possible implementation of the second aspect, the first rendering is reverberant rendering and direct sound rendering, and the second rendering is direct sound rendering. In the above possible implementations, the headphones and the headphone charging case use different rendering methods to render the audio signal separately. Compared to using the headphone charging case to complete the entire audio signal rendering task, this improves the speed of audio signal rendering and reduces rendering latency.

[0009] In any possible implementation of the second aspect, the method further includes: delaying the first spatial audio signal in the headphones to obtain a delayed spatial audio signal; and determining the target audio signal based on the delayed spatial audio signal and the second spatial audio signal. In the above possible implementations, the headphones time-align the first spatial audio signal and the second spatial audio signal, facilitating the playback of the target audio signal and improving the quality of the target audio signal.

[0010] In any possible implementation of the second aspect, the delay duration of the first spatial audio signal is related to the rendering method of the first rendering. In the above possible implementations, the headphone compartment can use different rendering methods to render the audio signal to obtain the first spatial audio signal, increasing selectivity.

[0011] In any possible implementation of the second aspect, the delay duration of the first spatial audio signal is related to the transmission delay, which is the delay between the earphone receiving the audio signal and receiving the first spatial audio signal. The above possible implementations, considering the delay between the earphone receiving the audio signal and receiving the first spatial audio signal, time-align the first and second spatial audio signals, thereby improving the quality of the target audio signal.

[0012] In any possible implementation of the second aspect, the first spatial audio signal is reverberant sound, and the second spatial audio signal is direct sound. Direct sound refers to the sound directly transmitted to the user from the target audio signal played by the headphones, while reverberant sound refers to the sound transmitted to the user from the target audio signal played by the headphones after reflection from the environment. In the above possible implementations, the headphone charging case renders the audio signal with reverberation, and the headphones render the audio signal with direct sound. Compared to using the headphone charging case to complete the entire audio signal rendering task, this improves the speed of audio signal rendering and reduces rendering latency.

[0013] Thirdly, an audio playback device is provided, the device including headphones and a charging case, both of which are wirelessly connected to an audio transmitting device; the charging case is used to receive audio signals from the audio transmitting device; the charging case is used to perform a first rendering on the audio signals to obtain a first spatial audio signal; the charging case is also used to send the first spatial audio signal to the headphones; the headphones are also used to play a target audio signal, the target audio signal including the first spatial audio signal.

[0014] Fourthly, an audio playback device is provided, comprising headphones and an earphone charging case, both of which are wirelessly connected to an audio transmitting device; both headphones and the earphone charging case receive audio signals from the audio transmitting device; the earphone charging case performs a first rendering on the audio signals to obtain a first spatial audio signal; the earphone charging case sends the first spatial audio signal to the headphones; the headphones perform a second rendering on the audio signals to obtain a second spatial audio signal; the headphones play a target audio signal, the target audio signal including the first spatial audio signal and the second spatial audio signal.

[0015] In any possible implementation of the fourth aspect, the first rendering is reverberant rendering and direct sound rendering, and the second rendering is direct sound rendering.

[0016] In any possible implementation of the fourth aspect, the headphones are further configured to delay the first spatial audio signal to obtain a delayed spatial audio signal; the headphones are further configured to determine the target audio signal based on the delayed spatial audio signal and the second spatial audio signal.

[0017] In any possible implementation of the fourth aspect, the delay duration of the first spatial audio signal is related to the rendering method of the first rendering.

[0018] In any possible implementation of the fourth aspect, the delay duration of the first spatial audio signal is related to the transmission delay, which is the delay between when the headphones receive the audio signal and when they receive the first spatial audio signal.

[0019] In any possible implementation of the fourth aspect, the first spatial audio signal is reverberant sound and the second spatial audio signal is direct sound.

[0020] Fifthly, a computer-readable storage medium is provided, wherein a computer program or instructions are stored therein, which, when executed, implement the audio processing method provided by the first aspect or any possible implementation thereof.

[0021] In a sixth aspect, a computer program product is provided, comprising: a computer program, also referred to as code or instructions, which, when run, causes a computer to perform an audio processing method as provided in the first aspect or any possible implementation thereof.

[0022] Understandably, the beneficial effects that can be achieved by the third to sixth aspects mentioned above can be referred to in relation to the beneficial effects provided by the first aspect or any possible implementation of the first aspect and the second aspect or any possible implementation of the second aspect, which will not be repeated here. Attached Figure Description

[0023] Figure 1 A schematic diagram illustrating an audio signal rendering method provided in an embodiment of this application;

[0024] Figure 2 A schematic diagram illustrating another audio signal rendering provided in an embodiment of this application;

[0025] Figure 3 A schematic diagram of an audio playback device provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of the structure of an earphone case provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the structure of an earphone provided in an embodiment of this application;

[0028] Figure 6 A flowchart illustrating an audio processing method provided in an embodiment of this application;

[0029] Figure 7 A flowchart illustrating another audio processing method provided in an embodiment of this application;

[0030] Figure 8 A schematic diagram illustrating an audio processing method provided in an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of another audio processing method provided in an embodiment of this application. Detailed Implementation

[0032] The following sections will discuss the fabrication and use of various embodiments in detail. However, it should be understood that many applicable inventive concepts provided in this application can be implemented in a variety of specific environments. The specific embodiments discussed are merely illustrative of specific ways of implementing and using this application and technology, and do not limit the scope of this application.

[0033] Unless otherwise defined, all technical terms used herein have the same meaning as commonly known to one of ordinary skill in the art.

[0034] Each circuit or other component may be described or referred to as "for" performing one or more tasks. In this context, "for" is used to imply a structure by indicating that the circuit / component includes a structure (e.g., a circuit system) that performs one or more tasks during operation. Therefore, even when the specified circuit / component is currently inoperable (e.g., not turned on), it can still be referred to as "for performing that task." Circuits / components used with the term "for" include hardware, such as circuits that perform operations.

[0035] The technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, a, b, and c; where a, b, and c can be single or multiple.

[0036] The embodiments of this application use terms such as "first" and "second" to distinguish objects with similar names, functions, or roles. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order.

[0037] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0038] Before introducing the embodiments of this application, we will first introduce and explain the relevant knowledge of spatial audio.

[0039] With the development of smart mobile devices, headphones have become an essential item for people to listen to audio in their daily lives. Wireless headphones, due to their convenience, are increasingly favored by the market and have even gradually become the mainstream headphone product. Consequently, people's demands for sound quality are also increasing, not only pursuing lossless sound quality but also demanding a greater sense of spatial sound. In one possible implementation, spatial audio technology is used to eliminate the in-head effect in audio signals, thereby enhancing the spatial sense of the audio signal and obtaining spatial audio. Spatial audio technology: headphones use the head-related transfer function (HRTF) to simulate the auditory effect of a sound field in the real world. In-head effect: refers to the unnatural effect that users perceive when listening to stereo audio while wearing headphones, as if the sound image is located inside their head. However, spatial audio technology has certain requirements for computing power and memory.

[0040] In one possible embodiment, such as Figure 1 As shown, an audio transmitting device (such as a mobile phone or other multimedia device) renders the audio signal to obtain spatial audio and sends the spatial audio to headphones, which then play the spatial audio. However, in this embodiment, the headphones can only play spatial audio sent by audio devices compatible with them and cannot play spatial audio sent by audio devices from other brands, thus limiting the usability of the headphones. Figure 1 and the following text Figure 2 Taking a mobile phone as an example, the audio transmission device is an audio device.

[0041] In another possible embodiment, such as Figure 2 As shown, the audio transmitting device sends an audio signal to the headphones. The headphones then render the audio signal to obtain spatial audio and play it. In other words, the headphones render the audio signal and obtain spatial audio independently of the audio transmitting device. However, due to the limited computing power and memory of the headphones, the relevant algorithms in spatial audio technology need to be simplified, which reduces the quality of the spatial audio and the user experience.

[0042] Based on this, this application provides an audio processing method for performing a first rendering of an audio signal through an earphone compartment to obtain a first spatial audio signal. This method renders the audio signal independently of the audio transmitting device, reducing dependence on the audio transmitting device while improving the effect of the first spatial audio signal.

[0043] The audio processing method provided in this application embodiment can be applied to an audio playback device; optionally, the audio playback device can be a true wireless stereo (TWS) headset, which includes an earphone case and earphones. For example, Figure 3 This is a schematic diagram of an audio playback device provided in an embodiment of this application. The audio playback device includes an earphone compartment 11 and earphones 12, which are wirelessly connected. The following is in conjunction with... Figure 4 and Figure 5 The specific structure of the earphone compartment 11 and the earphone 12 will be described and explained.

[0044] The earphone compartment 11 can be used to store earphones 12; the earphone compartment 11 is also used to power the earphones 12. In this embodiment, the earphone compartment 11 can also be used to receive and process audio signals to obtain a first spatial audio signal. For example, the earphone compartment 11 can be used to receive audio signals from an audio transmitting device and process the audio signals to obtain the first audio signal; the earphone compartment 11 is also used to send the first spatial audio signal to the earphones 12. For example, as shown... Figure 4 As shown, the headphone charging case 11 may include a first processor 110, a first memory 111, and a power supply 112. The first processor 110 can receive audio signals from an audio transmitting device and process the audio signals to obtain a first spatial audio signal; the first processor 110 is also used to send the first spatial audio signal to the headphones 12. The first memory 111 can be used to store data, software programs, and modules; it mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, and can also store sound rendering functions, etc.; the data storage area can store data created based on the use of the headphone charging case 11, such as audio data (including the first spatial audio signal). The power supply 112 (e.g., a battery) is logically connected to the first processor 110 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. In practical applications, the headphone charging case 11 can power the headphones 12 through a charging interface.

[0045] The headphones 12 can be used to receive a first spatial audio signal and play a target audio signal, the target audio signal including the first spatial audio signal. In one possible embodiment, the headphones 12 are also used to receive and process audio signals to obtain a second spatial audio signal. For example, the headphones 12 can be used to receive audio signals from an audio transmitting device and process those audio signals to obtain a second audio signal, the target audio signal also including the second spatial audio signal. For example, as... Figure 5As shown, the headset 12 includes a second processor 120, a second memory 121, an audio circuit 122, a speaker 123, and a microphone 124. The second processor 120 can receive audio signals from an audio transmitting device and process these signals to obtain a second spatial audio signal; the second processor 120 determines the target audio signal based on the first and second spatial audio signals. The second memory 121 can store data, software programs, and modules; it mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, and can store sound rendering and sound playback functions, etc. The data storage area can store data created based on the use of the headset 12, such as audio data (including the first spatial audio signal, the second spatial audio signal, and the target audio signal, etc.). The audio circuit 122, speaker 123, and microphone 124 provide an audio interface between the user and the headset 12. The audio circuit 122 can convert the target audio signal into an electrical signal and send it to the speaker 123. The speaker 123 converts the electrical signal into a sound signal and outputs it. On the other hand, the microphone 124 converts the collected sound signal into an electrical signal, which is received by the audio circuit and converted into an audio signal. The audio signal is then output to the second memory 121 for further processing.

[0046] Optionally, either the first processor 110 or the second processor 120 may include one or more processing units, which may include, but are not limited to, a central processing unit (CPU), a network processing unit (NPU), a tensor processing unit (TPU), a data processing unit (DPU), a digital signal processor (DSP), a microcontroller, or a microprocessor. Furthermore, the processor may also include other hardware circuits or accelerators, such as application-specific integrated circuits (ASICs), complex programmable logic devices (CPLDs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. Optionally, the processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, or a combination of a digital signal processor and a microprocessor.

[0047] Optionally, either the first memory 111 or the second memory 121 can be a pool of volatile memory or a pool of non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0048] Although not shown, the earphone compartment 11 and earphones 12 may also include other modules, such as a wireless fidelity (WiFi) module, a Bluetooth module, etc., which will not be described in detail in this embodiment. Those skilled in the art will understand that... Figure 4 and Figure 5 The structure of the earphone compartment 11 and earphone 12 shown does not constitute a limitation on the earphone compartment 11 and earphone 12. They may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0049] Figure 6 This is a flowchart illustrating an audio processing method provided in an embodiment of this application. The method can be applied to an audio playback device, which includes an earphone compartment and earphones, as described above. Figure 3 The audio playback device shown has the headphone compartment as described above. Figure 4 The earphone compartment 11 shown above contains the earphones as described above. Figure 5The earphone 12 shown. The method includes the following steps.

[0050] S601: The earphone compartment receives audio signals from the audio transmitting device.

[0051] In practical applications, the charging case and the audio transmitting device are wirelessly connected. The charging case receives audio signals transmitted from the audio transmitting device. Specifically, a first processor in the charging case receives the audio signals from the audio transmitting device; these audio signals are the audio signals to be played. This first processor can be configured as described above. Figure 4 The first processor 110 shown is illustrated. In one possible embodiment, the first processor can be a chip. The audio signal can also be referred to as the raw audio signal.

[0052] Optionally, the audio transmitting device can be an electronic device. For example, the electronic device may include, but is not limited to, mobile phones, tablets, laptops, PDAs, mobile internet devices (MIDs), cameras, smartwatches, smart glasses, smart bracelets, pedometers, set-top boxes, game consoles, printers, mice, keyboards, in-vehicle equipment (e.g., equipment on vehicles such as cars, airplanes, ships, trains, and high-speed trains), virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, smart home devices (e.g., refrigerators, televisions, air conditioners, electricity meters, etc.), smart robots, workshop equipment, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, and flying equipment (e.g., smart robots, hot air balloons, drones, airplanes), etc.

[0053] S602: The headphone case performs the first rendering of the audio signal to obtain the first spatial audio signal.

[0054] Specifically, the first processor in the headphone compartment performs a first rendering of the audio signal to obtain a first spatial audio signal.

[0055] In one possible embodiment, the headphone compartment can perform all rendering tasks of the audio signal (including direct sound rendering and reverberation rendering), in which case the first rendering is direct sound rendering and reverberation rendering. In another possible embodiment, the headphone compartment can also perform partial rendering tasks of the audio signal; for example, the headphone compartment can perform reverberation rendering, in which case the first rendering is reverberation rendering.

[0056] Additionally, direct sound rendering is used to render direct sound, which is the sound that travels directly from the target audio signal played through the headphones to the user. Reverb rendering is used to render reverb sound, which is the sound that travels from the target audio signal played through the headphones to the user after being reflected by the environment.

[0057] The following sections will explain the two scenarios in which the headphone compartment performs the first rendering of the audio signal.

[0058] In one possible embodiment, the headphone compartment performs all the rendering tasks for the audio signal.

[0059] Specifically, the first processor in the earphone case performs direct sound rendering and reverberation rendering on the audio signal to obtain a first spatial audio signal. This first spatial audio signal includes direct sound and reverberation, with the reverberation delay being higher than the direct sound delay. The reverberation delay relative to the direct sound is T0. In this embodiment, the first processor in the earphone case completes all audio signal rendering tasks independently of the audio transmitting device. Compared to the earphones, the earphone case has a larger volume. When using the earphone case to complete all audio signal rendering tasks, there is no need to simplify the relevant algorithms of audio spatial technology. Compared to rendering the audio signal through the earphones, the effect of the first spatial audio signal is improved.

[0060] In another possible embodiment, the headphone compartment performs a partial rendering task of the audio signal. Optionally, the headphone compartment may perform at least a partial rendering of the reverberation in the audio signal, or the headphone compartment may perform both direct sound rendering and reverberation rendering on a portion of the audio signal. Several possible scenarios will be described below.

[0061] In the first possible implementation, the first rendering is reverberation rendering, and the headphone compartment completes the reverberation rendering task for the audio signal. Specifically, the first processor in the headphone compartment performs reverberation rendering on the audio signal to obtain a first spatial audio signal, which is reverberation.

[0062] In the second possible implementation, the first rendering involves direct sound rendering and reverberation rendering, with the headphone compartment performing both tasks on a portion of the audio signal. Specifically, the first processor in the headphone compartment performs direct sound rendering and reverberation rendering on a portion of the audio signal to obtain a first spatial audio signal, which consists of a first sub-direct sound and a first sub-reverberation. The first sub-direct sound is a portion of the direct sound, and the first sub-reverberation is a portion of the reverberation. For ease of understanding, the portion of the audio signal will be referred to as the first sub-audio signal in the following text.

[0063] In a third possible embodiment, the first rendering is a partial rendering within the reverberation rendering process, and the headphone compartment completes a partial reverberation rendering task within the reverberation rendering task of the audio signal. Specifically, the first processor in the headphone compartment performs partial reverberation rendering on the audio signal to obtain a first spatial audio signal, which is a second sub-reverberation, and the second sub-reverberation is a portion of the reverberation. For ease of understanding, the partial rendering within the reverberation rendering process will be referred to as the first sub-reverberation rendering below.

[0064] S603: The earphone case sends the first spatial audio signal to the earphones.

[0065] In practical applications, the earphones and the charging case are wirelessly connected. A first processor in the charging case sends a first spatial audio signal to a second processor in the earphones. This second processor can perform the aforementioned... Figure 5 The second processor 120 shown is a chip.

[0066] S604: The headphones play a target audio signal, which includes a first spatial audio signal.

[0067] In one possible embodiment, when the first spatial audio signal includes direct sound and reverberation, the target audio signal only includes the first audio signal. The second processor in the headphones sends the target audio signal to the audio circuit, which converts the target audio signal into an electrical signal and sends it to the speaker. The speaker then converts the electrical signal back into sound and plays it. The audio circuit can be as described above. Figure 5 The audio circuit 122 shown can be used as described above. Figure 5 The speaker 123 shown.

[0068] In one possible embodiment, when the first spatial audio signal is reverberant, or the first spatial audio signal is a first sub-direct sound and a first sub-reverberant, or only a second sub-reverberant and the first spatial audio signal is a second sub-reverberant, the target audio signal further includes a second spatial audio signal. For example, as shown... Figure 7 As shown, the method provided in this application embodiment further includes:

[0069] S605: The headphones receive audio signals from the audio transmitting device.

[0070] In practical applications, the headphones are wirelessly connected to the audio transmitting device, and the second processor in the headphones receives the audio signal from the audio transmitting device.

[0071] S606: The headphones perform a second rendering of the audio signal to obtain a second spatial audio signal.

[0072] Since the first spatial audio signal can be reverberation, first sub-direct sound and first sub-reverberation, or second sub-reverberation, the specific process of the headphones performing the second rendering of the audio signal in the above three cases will be explained in detail below.

[0073] In a first possible embodiment, the first spatial audio signal is reverberant sound. In this case, the second rendering is direct sound rendering, and the rendering methods of the first and second renderings differ in this embodiment.

[0074] Specifically, the second processor in the earphone performs direct sound rendering on the audio signal to obtain a second spatial audio signal, which is direct sound. In this embodiment, different devices perform different rendering on the same audio signal using different rendering methods, that is, different devices complete different rendering tasks. Compared with using the earphone compartment to complete the entire audio signal rendering task, this improves the speed of audio signal rendering and reduces rendering latency.

[0075] In a second possible embodiment, the first spatial audio signal is a first sub-direct sound and a first sub-reverberation. In this case, the second rendering is direct sound rendering and reverberation rendering, and the rendering method of the first rendering and the rendering method of the second rendering are the same in this embodiment.

[0076] Specifically, the second processor in the headphones performs a second rendering on the remaining audio signals (excluding the first sub-audio signal) to obtain a second spatial audio signal. The second spatial audio signal includes a second sub-direct sound and a second sub-reverberation. The second sub-direct sound is the direct sound excluding the first sub-direct sound, and the second sub-reverberation is the reverberation sound excluding the first sub-reverberation. In this embodiment, different devices use the same rendering method to render different parts of the audio signal. Compared to using the headphone charging case to complete the entire audio signal rendering task, this increases the speed of audio signal rendering and reduces rendering latency.

[0077] In a third possible embodiment, the first spatial audio signal is the second sub-reverberation. The second rendering is the second sub-reverberation rendering, excluding the first sub-reverberation rendering, between direct sound rendering and reverberation rendering. In this embodiment, the rendering methods of the first rendering and the second rendering are different.

[0078] Specifically, the second processor in the headphones performs a second rendering on the audio signal to obtain a second spatial audio signal, which includes direct sound and a third sub-reverberation. The third sub-reverberation is the reverberation sound other than the second sub-reverberation. In this embodiment, the headphones are used to complete more rendering tasks, further reducing rendering latency.

[0079] Furthermore, when the first spatial audio signal is reverberant sound and the second spatial audio signal is direct sound, the method provided in this application embodiment further includes: delaying the first spatial audio signal with headphones to obtain a delayed spatial audio signal, for example, delaying the first spatial audio signal with headphones for a duration of T0-T to obtain a delayed spatial audio signal; determining a target audio signal based on the delayed spatial audio signal and the second spatial audio signal, for example, superimposing the delayed spatial audio signal and the second spatial audio signal with headphones to obtain a target audio signal.

[0080] The delay duration T0-T of the first spatial audio signal is related to the rendering method and transmission delay of the first rendering. For example, T0 varies depending on the rendering method of the first rendering; when the first rendering is reverberation rendering, the headphone compartment performs reverberation rendering while the headphones perform direct sound rendering, and T0 can be 10ms; when the first rendering is first sub-reverberation rendering (i.e., the headphone compartment completes part of the reverberation rendering task for the audio signal), the headphone compartment performs part of the reverberation rendering task while the headphones perform direct sound rendering and reverberation rendering excluding the first sub-reverberation rendering, and T0 can be 50ms. The transmission delay is the delay between the headphones receiving the audio signal and receiving the first spatial audio signal.

[0081] The following is combined Figure 8 and Figure 9 The audio processing method provided in the embodiments of this application will be described and explained. Figure 8 and Figure 9 The following explanation uses a mobile phone as an example to illustrate the audio transmission device.

[0082] For example, Figure 8This is a schematic diagram of an audio processing method provided in an embodiment of this application. The audio processing method provided in this application embodiment may include the following steps: S801: The mobile phone sends an audio signal to the earphone compartment. S802: The earphone compartment performs a first rendering on the audio signal to obtain a first spatial audio signal. The first spatial audio signal includes reverberation and direct sound. S803: The earphone compartment sends the first spatial audio signal to the earphones. S804: The earphones play the first spatial audio signal.

[0083] For example, Figure 9 This is a schematic diagram of another audio processing method provided in an embodiment of this application. The audio processing method provided in this application embodiment may include the following steps: S901a: The mobile phone sends an audio signal to the earphone compartment. S901b: The mobile phone sends an audio signal to the earphone. S902: The earphone compartment performs a first rendering on the audio signal to obtain a first spatial audio signal. The first spatial audio signal includes at least a portion of reverberation. S903: The earphone compartment sends the first spatial audio signal to the earphone. S904: The earphone performs a second rendering on the audio signal to obtain a second spatial audio signal. The second spatial audio signal is direct sound, or the second spatial audio signal is direct sound and at least a portion of reverberation. S905: The earphone determines a target audio signal based on the first spatial audio signal and the second spatial audio signal and plays it.

[0084] In one possible embodiment, a similar audio processing method can also be used between the mobile phone and the wearable audio device. For example, taking a mobile phone and smart glasses as an example, the mobile phone performs a first rendering of the audio signal and sends the resulting first spatial audio signal to the smart glasses; the smart glasses perform a second rendering of the audio signal to obtain a second spatial audio signal, which was sent by the mobile phone; the smart glasses determine the target audio signal based on the first and second spatial audio signals and play it. Similarly, smartwatches and smart glasses can also use a similar audio processing method as those used with mobile phones and smart glasses.

[0085] In this embodiment, the earphone case receives an audio signal from an audio transmitting device and performs a first rendering on the audio signal to obtain a first spatial audio signal. The earphone case then sends the first spatial audio signal to the earphones, and the earphones play a target audio signal, which includes the first spatial audio signal. Since the earphones and earphone case are compatible, users typically carry the earphone case with them when using the earphones in practical applications. Furthermore, the earphone case is relatively large compared to the earphones. Using the earphone case for the first rendering of the audio signal eliminates the need to simplify the relevant algorithms in spatial audio technology, thus maintaining the quality of the first spatial audio signal. Moreover, in this process, the earphone case performs the first rendering of the audio signal independently of the audio transmitting device, reducing dependence on the audio transmitting device.

[0086] The foregoing primarily describes the solutions provided in this application from the perspective of audio playback devices. It is understood that, in order to achieve the aforementioned functions, the audio playback device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0087] This application embodiment can divide the audio playback device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the division of each functional module according to its own function as an example.

[0088] When using an integrated unit, the audio playback device may include an earphone compartment and earphones, and the audio playback device may be as described above. Figure 3 The audio playback device shown can have the headphone jack as described above. Figure 4 The earphone case shown can be used for the earphones described above. Figure 5 The headphones shown. All relevant content of each step involved in the above method embodiments can be referenced in the functional description of audio playback devices, and will not be repeated here.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed.

[0090] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. This readable storage medium may include various media capable of storing program code, such as a USB flash drive, external hard drive, read-only memory, random access memory, magnetic disk, or optical disk. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product.

[0092] In another embodiment of this application, a computer-readable storage medium is also provided, which stores a computer program or instructions that, when executed, cause a device to perform the steps described in the method embodiments above.

[0093] In another embodiment of this application, a computer program product is also provided, which includes a computer program, also referred to as code or instructions, which, when run, causes a device to perform the steps in the above method embodiments.

[0094] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An audio processing method, characterized in that, An audio playback device, comprising headphones and a charging case, wherein the charging case is wirelessly connected to an audio transmitting device, and the headphones and the charging case are wirelessly connected, the method comprising: The earphone compartment receives audio signals from the audio transmitting device; The earphone compartment performs a first rendering on the audio signal to obtain a first spatial audio signal; The earphone compartment sends the first spatial audio signal to the earphones; The headphones play a target audio signal, which includes the first spatial audio signal.

2. An audio processing method, characterized in that, An audio playback device, comprising headphones and a charging case, wherein both the charging case and the headphones are wirelessly connected to an audio transmitting device, and the headphones and the charging case are wirelessly connected, the method comprising: Both the earphones and the earphone case receive audio signals from the audio transmitting device; The earphone compartment performs a first rendering on the audio signal to obtain a first spatial audio signal; The earphone compartment sends the first spatial audio signal to the earphones; The headphones perform a second rendering on the audio signal to obtain a second spatial audio signal; The headphones play a target audio signal, which includes the first spatial audio signal and the second spatial audio signal.

3. The method according to claim 2, characterized in that, The first rendering consists of reverberant rendering and direct sound rendering, while the second rendering consists of direct sound rendering.

4. The method according to claim 2 or 3, characterized in that, The method further includes: The earphone delays the first spatial audio signal to obtain a delayed spatial audio signal; The headphones determine the target audio signal based on the delayed spatial audio signal and the second spatial audio signal.

5. The method according to claim 4, characterized in that, The delay duration of the first spatial audio signal is related to the rendering method of the first rendering.

6. The method according to claim 4 or 5, characterized in that, The delay duration of the first spatial audio signal is related to the transmission delay, which is the delay between when the earphone receives the audio signal and when it receives the first spatial audio signal.

7. The method according to any one of claims 2-6, characterized in that, The first spatial audio signal is reverberant sound, and the second spatial audio signal is direct sound.

8. An audio playback device, characterized in that, The device includes earphones and an earphone case, both of which are wirelessly connected to an audio transmitting device. The earphone compartment is used to receive audio signals from the audio transmitting device; The earphone compartment is used to perform a first rendering of the audio signal to obtain a first spatial audio signal; The earphone compartment is also used to send the first spatial audio signal to the earphones; The headphones are also used to play a target audio signal, which includes the first spatial audio signal.

9. An audio playback device, characterized in that, The device includes earphones and an earphone case, both of which are wirelessly connected to an audio transmitting device. Both the earphones and the earphone case receive audio signals from the audio transmitting device; The earphone compartment performs a first rendering on the audio signal to obtain a first spatial audio signal; The earphone compartment sends the first spatial audio signal to the earphones; The headphones perform a second rendering on the audio signal to obtain a second spatial audio signal; The headphones play a target audio signal, which includes the first spatial audio signal and the second spatial audio signal.

10. The device according to claim 9, characterized in that, The first rendering consists of reverberant rendering and direct sound rendering, while the second rendering consists of direct sound rendering.

11. The device according to claim 9 or 10, characterized in that, The earphone is also used to delay the first spatial audio signal to obtain a delayed spatial audio signal; The headphones are also used to determine the target audio signal based on the delayed spatial audio signal and the second spatial audio signal.

12. The device according to claim 11, characterized in that, The delay duration of the first spatial audio signal is related to the rendering method of the first rendering.

13. The device according to claim 11 or 12, characterized in that, The delay duration of the first spatial audio signal is related to the transmission delay, which is the delay between when the earphone receives the audio signal and when it receives the first spatial audio signal.

14. The device according to any one of claims 9-13, characterized in that, The first spatial audio signal is reverberant sound, and the second spatial audio signal is direct sound.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on an audio playback device, cause the audio playback device to perform the audio processing method as described in any one of claims 1-7.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an audio playback device, causes the audio playback device to perform the audio processing method as described in any one of claims 1-7.

Images