Audio processing method and device and storage medium

Abstract

The invention relates to an audio processing method and device and a storage medium. The method comprises the following steps: sending first rotation information of the wearable device to the electronic device; receiving coded data sent by the electronic equipment; decoding the coded data to obtain rendering space audio data and first rotation information; acquiring second rotation information of the wearable device; and based on the first rotation information and the second rotation information, performing rendering processing on the rendering space audio data to obtain target space audio data. On one hand, the spatial audio data and the first rotation information are directly transmitted and rendered through coding, the two are not delayed, secondary delay caused by returning of the first rotation information in other schemes is avoided, and then the sense of delay brought by audio rendering to a user is reduced; and on the other hand, audio rendering can be carried out at the two ends of the electronic equipment and the wearable equipment at the same time, the accuracy and the space sense of audio rendering can be improved, and then the texture of the generated target space audio data is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of information technology, and in particular to an audio processing method, device and storage medium. Background Art

[0002] With the advancement of science and technology and the development of diversified electronic devices, wearable devices have emerged. In order to realize the diversification of wearable device functions, audio playback modules can be configured for wearable devices. In order to improve the user's auditory experience and enable the user to feel the source and location of the sound, the sound can be processed and adjusted so that the audio played by the wearable device presents a specific spatial effect, that is, spatial audio is generated.

[0003] In related technologies, audio rendering can be performed on an electronic device (e.g., a mobile phone), and the rendered spatial audio is sent to a wearable device (e.g., headphones) for playback, which causes the spatial sense of the sound to lag significantly behind the actual activity of the wearable device, causing a noticeable sense of delay for the user. In addition, audio rendering can also be performed on the wearable device, but due to the chip size and power consumption limitations of wearable devices, the computing power is relatively small, resulting in insufficient accuracy in audio rendering, insufficient spatial sense, and difficulty in performing complex tuning processing, which will also affect the sound quality. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an audio processing method, device and storage medium to overcome the delay caused to the user, as well as the problems of low accuracy, insufficient spatial sense and poor sound quality of the rendered audio.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an audio processing method, including:

[0006] Sending first rotation information of the wearable device to the electronic device;

[0007] Receiving encoded data sent by the electronic device; wherein the encoded data is obtained by the electronic device encoding the rendered spatial audio data and the first rotation information, and the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information;

[0008] Decoding the encoded data to obtain the rendering spatial audio data and the first rotation information;

[0009] Acquire second rotation information of the wearable device;

[0010] Based on the first rotation information and the second rotation information, the rendering spatial audio data is rendered to obtain target spatial audio data.

[0011] In some embodiments, the rendering spatial audio data is rendered based on the first rotation information and the second rotation information to obtain target spatial audio data, including:

[0012] determining a perception bias parameter based on the first rotation information and the second rotation information;

[0013] The rendering spatial audio data is rendered based on the perception deviation parameter to obtain the target spatial audio data.

[0014] In some embodiments, the perception deviation parameter includes: a delay deviation parameter and / or a sound pressure deviation parameter; and rendering the rendering spatial audio data based on the perception deviation parameter to obtain the target spatial audio data includes:

[0015] Delay processing is performed on the rendered spatial audio data based on the delay deviation parameter, and / or

[0016] The sound pressure of the rendering spatial audio data is adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data.

[0017] In some embodiments, determining the perception deviation parameter based on the first rotation information and the second rotation information includes:

[0018] Determine the compensation rotation information based on the first rotation information and the second rotation information, and determine the delay deviation parameter based on the compensation rotation information and a preset correlation coefficient; and / or

[0019] A first sound pressure parameter corresponding to the first rotation information and a second sound pressure parameter corresponding to the second rotation information are acquired, and the sound pressure deviation parameter is determined based on the first sound pressure parameter and the second sound pressure parameter.

[0020] In some embodiments, determining the compensating rotation information based on the first rotation information and the second rotation information includes:

[0021] Performing weighted processing on the first rotation information based on a first weight coefficient to obtain first weighted rotation information;

[0022] performing weighted processing on the second rotation information based on a second weight coefficient to obtain second weighted rotation information;

[0023] The compensating rotation information is determined based on a difference between the first weighted rotation information and the second rotation information.

[0024] In some embodiments, the decoding of the encoded data to obtain the rendered spatial audio data and the first rotation information includes:

[0025] Decoding the encoded data based on a first decoder to obtain pre-encoded data and the first rotation information;

[0026] The pre-encoded data is decoded based on a second decoder to obtain the rendered spatial audio data.

[0027] According to a second aspect of an embodiment of the present disclosure, there is provided an audio processing method, including:

[0028] Receiving first rotation information sent by the wearable device;

[0029] Rendering the initial audio data based on the first rotation information to obtain rendered spatial audio data;

[0030] Encoding the rendering space audio data and the first rotation information to obtain encoded data;

[0031] The encoded data is sent to the wearable device, so that the wearable device performs rendering processing on the rendering spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data.

[0032] In some embodiments, the rendering of the initial audio data based on the first rotation information to obtain rendered spatial audio data includes:

[0033] Based on the first rotation information and the spatial transfer function, obtaining a related impulse response corresponding to the first rotation information;

[0034] The initial audio data and the relevant impulse response are convolved to obtain the rendered spatial audio data.

[0035] In some embodiments, encoding the rendering spatial audio data and the first rotation information to obtain encoded data includes:

[0036] Encoding the rendering spatial audio data based on a first encoder to obtain pre-encoded data;

[0037] The pre-encoded data and the first rotation information are encoded based on a second encoder to obtain the encoded data.

[0038] According to a third aspect of an embodiment of the present disclosure, there is provided an audio processing device, including:

[0039] A first sending module, configured to send first rotation information of the wearable device to the electronic device;

[0040] A first receiving module is configured to receive encoded data sent by the electronic device; wherein the encoded data is obtained by the electronic device encoding the rendered spatial audio data and the first rotation information, and the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information;

[0041] A decoding module, configured to decode the encoded data to obtain the rendering spatial audio data and the first rotation information;

[0042] An acquisition module, configured to acquire second rotation information of the wearable device;

[0043] The first rendering module is configured to perform rendering processing on the rendering spatial audio data based on the first rotation information and the second rotation information to obtain target spatial audio data.

[0044] In some embodiments, the first rendering module is configured as follows:

[0045] determining a perception bias parameter based on the first rotation information and the second rotation information;

[0046] The rendering spatial audio data is rendered based on the perception deviation parameter to obtain the target spatial audio data.

[0047] In some embodiments, the perception deviation parameter includes: a delay deviation parameter and / or a sound pressure deviation parameter; the first rendering module is configured to:

[0048] Delay processing is performed on the rendered spatial audio data based on the delay deviation parameter, and / or

[0049] The sound pressure of the rendering spatial audio data is adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data.

[0050] In some embodiments, the first rendering module is configured as follows:

[0051] Determine the compensation rotation information based on the first rotation information and the second rotation information, and determine the delay deviation parameter based on the compensation rotation information and a preset correlation coefficient; and / or

[0052] A first sound pressure parameter corresponding to the first rotation information and a second sound pressure parameter corresponding to the second rotation information are acquired, and the sound pressure deviation parameter is determined based on the first sound pressure parameter and the second sound pressure parameter.

[0053] In some embodiments, the first rendering module is configured as follows:

[0054] Performing weighted processing on the first rotation information based on a first weight coefficient to obtain first weighted rotation information;

[0055] performing weighted processing on the second rotation information based on a second weight coefficient to obtain second weighted rotation information;

[0056] The compensating rotation information is determined based on a difference between the first weighted rotation information and the second rotation information.

[0057] In some embodiments, the decoding module is configured as follows:

[0058] Decoding the encoded data based on a first decoder to obtain pre-encoded data and the first rotation information;

[0059] The pre-encoded data is decoded based on a second decoder to obtain the rendered spatial audio data.

[0060] According to a fourth aspect of an embodiment of the present disclosure, there is provided an audio processing device, including:

[0061] A second receiving module is configured to receive first rotation information sent by the wearable device;

[0062] A second rendering module is configured to render the initial audio data based on the first rotation information to obtain rendered spatial audio data;

[0063] an encoding module, configured to encode the rendering spatial audio data and the first rotation information to obtain encoded data;

[0064] The second sending module is configured to send the encoded data to the wearable device, so that the wearable device renders the rendering spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data.

[0065] In some embodiments, the second rendering module is configured as follows:

[0066] Based on the first rotation information and the spatial transfer function, obtaining a related impulse response corresponding to the first rotation information;

[0067] The initial audio data and the relevant impulse response are convolved to obtain the rendered spatial audio data.

[0068] In some embodiments, the encoding module is configured as follows:

[0069] Encoding the rendering spatial audio data based on a first encoder to obtain pre-encoded data;

[0070] The pre-encoded data and the first rotation information are encoded based on a second encoder to obtain the encoded data.

[0071] According to a fifth aspect of an embodiment of the present disclosure, there is provided an audio processing device, including:

[0072] processor;

[0073] a memory configured to store processor-executable instructions;

[0074] Wherein, the processor is configured to: implement the steps in any one of the audio processing methods in the first aspect and the second aspect when executed.

[0075] According to the sixth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an audio processing device, the device is enabled to perform any one of the audio processing methods of the first and second aspects described above.

[0076] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0077] In an embodiment of the present disclosure, first rotation information of the wearable device is sent to an electronic device, and encoded data sent by the electronic device is received, the encoded data is decoded to obtain rendered spatial audio data and the first rotation information, and based on the first rotation information and the acquired second rotation information, the rendered spatial audio data is rendered to obtain target spatial audio data, wherein the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information.

[0078] The technical solution disclosed in the present invention, on the one hand, adopts encoding to directly transmit the rendered spatial audio data and the first rotation information, and there will be no delay between the two, thus avoiding the secondary delay caused by returning the first rotation information in other solutions, thereby reducing the sense of delay brought to the user by the audio rendering; on the other hand, it can perform audio rendering on both the electronic device and the wearable device at the same time, which can improve the accuracy and spatial sense of audio rendering, thereby improving the texture of the generated target spatial audio data.

[0079] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0081] Figure 1 The following is a flow chart of an audio processing method according to an exemplary embodiment. Figure 1 .

[0082] Figure 2A The first scene of wearing headphones to listen to audio is shown according to an exemplary embodiment.

[0083] Figure 2B The second scenario of wearing headphones to listen to audio is shown according to an exemplary embodiment.

[0084] Figure 3 It is a schematic diagram of a data structure of encoded data according to an exemplary embodiment.

[0085] Figure 4 FIG. 2 is a second flowchart of an audio processing method according to an exemplary embodiment.

[0086] Figure 5 The following is a flow chart of an audio processing method according to an exemplary embodiment. Figure 3 .

[0087] Figure 6 The present invention is a block diagram of an audio processing device according to an exemplary embodiment.

[0088] Figure 7 The present invention is a block diagram of an audio processing device according to an exemplary embodiment.

[0089] Figure 8 is a hardware structure block diagram of an audio processing device 800 according to an exemplary embodiment. DETAILED DESCRIPTION

[0090] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0091] Figure 1 The following is a flow chart of an audio processing method according to an exemplary embodiment. Figure 1 ,like Figure 1 As shown, the method mainly includes the following steps:

[0092] In step 101, first rotation information of the wearable device is sent to an electronic device;

[0093] In step 102, receiving encoded data sent by the electronic device; wherein the encoded data is obtained by the electronic device encoding the rendered spatial audio data and the first rotation information, and the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information;

[0094] In step 103, the encoded data is decoded to obtain the rendered spatial audio data and the first rotation information;

[0095] In step 104, second rotation information of the wearable device is obtained;

[0096] In step 105, the rendering spatial audio data is rendered based on the first rotation information and the second rotation information to obtain target spatial audio data.

[0097] Here, the method can be applied to electronic devices. The electronic devices may include terminal devices. The terminal devices may include mobile terminals and fixed terminals, such as mobile phones, tablet computers, PDAs, laptop computers, desktop computers, wearable devices, smart speakers, televisions, and vehicle-mounted terminals.

[0098] Wearable devices include: portable devices that are worn directly on the user or integrated into the user's accessories. For example, they include: wearable devices supported by the wrist (including watches, wristbands, etc.); wearable devices supported by the foot (including shoes, socks, etc.), wearable devices supported by the head (including headphones, glasses, helmets, headbands, etc.). They can also include: smart clothing, school bags, crutches, accessories, etc.

[0099] In some embodiments, the wearable device may have an audio playback function. Since the wearable device can be directly worn on the user, when the user's body rotates, the wearable device will also rotate with the user's body.

[0100] Take the case of a wearable device such as headphones. When the user's head turns, the headphones will also turn with the user's head. Figure 2A FIG. 1 is a diagram showing a scene 1 of wearing headphones to listen to audio according to an exemplary embodiment. Figure 2A As shown, the audio played by the headphones will also rotate with the rotation of the head; Figure 2B FIG. 2 is a diagram showing a second scenario of wearing headphones to listen to audio according to an exemplary embodiment. Figure 2B As shown, the audio played in the headphones will maintain the original playback direction.

[0101] In the embodiment of the present disclosure, when the wearable device rotates, the first rotation information of the wearable device can be obtained and sent to the electronic device. Here, the first rotation information may include: information generated when the wearable device rotates, for example, it may include the rotation angle and rotation direction of the wearable device. In some embodiments, a detection module (for example, an angle detection module, an orientation detection module, etc.) can be set in the wearable device. When the wearable device is detected to rotate, the detection module can sense the first rotation information of the wearable device.

[0102] Taking the first rotation information as the first rotation angle as an example, an angular motion detection module can be set in the wearable device, and the first rotation angle of the wearable device is determined by the angular motion detection module, wherein the angular motion detection module may include: an angular velocity sensor (gyroscope). Taking the first rotation information as the first rotation orientation as an example, an orientation detection module can be set in the wearable device, and the first rotation orientation of the wearable device is determined by the orientation detection module, wherein the orientation detection module may include: an orientation sensor.

[0103] In an embodiment of the present disclosure, after obtaining the first rotation information of the wearable device, the first rotation information can be sent to the electronic device. In some embodiments, a communication connection between the wearable device and the electronic device can be established. For example, a wired connection between the wearable device and the electronic device can be established, wherein the wired connection can include: a data line connection; a wireless connection between the wearable device and the electronic device can also be established, wherein the wireless connection can include: a short-range wireless connection (e.g., a Bluetooth connection) and a long-range wireless connection (e.g., a Wi-Fi connection). For example, Bluetooth communication can be established between the electronic device and the wearable device.

[0104] In some embodiments, the electronic device is a device that outputs initial audio data, and the wearable device is a device that outputs target spatial audio data, wherein the target spatial audio data is spatial audio obtained by rendering the initial audio data.

[0105] After receiving the first rotation information, the electronic device may render the initial audio data based on the first rotation information to obtain rendered spatial audio data. In some embodiments, the initial audio data may be rendered using a spatial transfer function, for example, the initial audio data may be rendered using a head-related transfer function (HRTF) to obtain rendered spatial audio data. When constructing the head-related transfer function, the relevant impulse responses corresponding to each angle in the space may be collected in advance, and a mapping relationship between each angle and the relevant impulse responses corresponding to each angle may be established to construct a data set.

[0106] In the embodiment of the present disclosure, after obtaining the first rotation information, the electronic device can obtain the relevant impulse response corresponding to the first rotation information from the data set corresponding to the spatial transfer function based on the first rotation information, and then perform convolution processing on the initial audio data and the relevant impulse response to obtain rendered spatial audio data.

[0107] Taking the first rotation information being the first rotation angle as an example, a related impulse response corresponding to the first rotation angle can be obtained based on the first rotation angle, and the initial audio data and the related impulse response can be convolved to obtain rendered spatial audio data. Here, the rendered spatial audio data is spatial audio obtained by rendering the initial audio data via an electronic device.

[0108] After obtaining the rendered spatial audio data, the electronic device may encode the rendered spatial audio data and the first rotation information to obtain encoded data.

[0109] In some embodiments, the rendering spatial audio data may be first encoded based on an encoding algorithm to obtain pre-encoded data, and then the pre-encoded data and the first rotation information may be repackaged to obtain encoded data. In some embodiments, the rendering spatial audio data may be encoded based on a first encoder to obtain pre-encoded data, and the pre-encoded data and the first rotation information may be encoded based on a second encoder to obtain encoded data.

[0110] In some embodiments, the type of the first encoder and the type of the second encoder may be the same, or the type of the first encoder and the type of the second encoder may be different. In some embodiments, the first encoder may be an audio encoder for encoding audio data. For example, the rendering spatial audio data may be input into the first encoder, and then the rendering spatial audio data may be encoded to obtain pre-encoded data. In some embodiments, the second encoder may be a data encoder for integrating different types of data into a data packet. For example, the pre-encoded data and the first rotation information may be input into the second encoder for encoding to obtain encoded data. The encoded data may be a data packet including the pre-encoded data and the first rotation information.

[0111] For example, an audio encoder (including a Bluetooth audio encoder) can be used to complete the encoding and packaging of the rendering spatial audio data to obtain pre-coded data (enc_data), and then a data encoder can be used to perform secondary packaging of the pre-coded data (enc_data) and the first rotation information (angle_data) according to a preset packaging format to obtain coded data. After obtaining the coded data, the electronic device can send the coded data to the wearable device.

[0112] In some embodiments, the encoded data may consist of three parts: Figure 3 is a schematic diagram of a data structure of coded data according to an exemplary embodiment, such as Figure 3 As shown, the first part of the coded data is the frame header part, which can also be called the encapsulation frame header. In the process of encoding the pre-coded data and the first rotation information, the first rotation information can be set in the frame header part, and the synchronization code can be set in the frame header part; the second part is the data part, which is used to place the pre-coded data; the third part is the frame tail part, which can also be called the encapsulation frame tail, which is used to set the check code, wherein the check code is used to check the coded data. For example, the integrity of the coded data can be checked.

[0113] In the disclosed embodiment, when the precoded data and the first rotation information are repackaged according to a preset packaging format, a check code may be set in the data frame for verification during decoding, thereby reducing the possibility of decoding failure due to data loss during transmission.

[0114] Correspondingly, after receiving the encoded data, the wearable device can decode the encoded data. For example, before the Bluetooth audio decoder processes the encoded data, the encoded data can be parsed and processed first, and after the encoded data is verified based on the check code and determined to be normal, the pre-encoded data and the first rotation information are obtained, and then the pre-encoded data is input into the Bluetooth audio decoder for decoding, thereby obtaining the rendered spatial audio data, wherein the first rotation information may include: angle information for spatial audio rendering.

[0115] In other embodiments, the rendering spatial audio data and the first rotation information may be directly encoded by an encoding algorithm to obtain encoded data. For example, the rendering spatial audio data and the first rotation information may be encoded directly based on an audio encoding protocol. Correspondingly, after receiving the encoded data, the wearable device may decode the encoded data. For example, the encoded data may be directly decoded based on an audio decoding protocol to obtain the rendering spatial audio data and the first rotation information.

[0116] In the embodiment of the present disclosure, after obtaining the rendered spatial audio data and the first rotation information, the wearable device can obtain the second rotation information. It can be understood that the first rotation information is the rotation information collected when the wearable device rotates for the first time, and the second rotation information is the rotation information collected when the wearable device rotates currently.

[0117] In the disclosed embodiment, when the rotation of the wearable device is detected, the first rotation information can be collected and the audio rendering mechanism can be triggered, so that the electronic device can perform transmission processing on the initial audio data. After the wearable device obtains the rendered spatial audio data, the second rotation information can be collected, and the current rotation information of the wearable device can be determined more accurately. The rendered spatial audio data is then re-rendered based on the first rotation information and the second rotation information to obtain the target spatial audio data, which can ensure that the obtained target spatial audio data is closer to the current rotation information of the wearable device, thereby ensuring the accuracy and real-time nature of the obtained target spatial audio data.

[0118] In the disclosed embodiment, the wearable device sends the acquired first rotation information to the electronic device, receives the encoded data sent by the electronic device, decodes the encoded data to obtain rendered spatial audio data and the first rotation information, and renders the rendered spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data, wherein the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information.

[0119] The technical solution disclosed in the present invention, on the one hand, adopts encoding to directly transmit the rendered spatial audio data and the first rotation information, and there will be no delay between the two, thus avoiding the secondary delay caused by returning the first rotation information in other solutions, thereby reducing the sense of delay brought to the user by the audio rendering; on the other hand, it can perform audio rendering on both the electronic device and the wearable device at the same time, which can improve the accuracy and spatial sense of audio rendering, thereby improving the texture of the generated target spatial audio data.

[0120] In some embodiments, the rendering spatial audio data is rendered based on the first rotation information and the second rotation information to obtain target spatial audio data, including:

[0121] determining a perception bias parameter based on the first rotation information and the second rotation information;

[0122] The rendering spatial audio data is rendered based on the perception deviation parameter to obtain the target spatial audio data.

[0123] Here, the perception deviation parameter may be a deviation parameter used to characterize the audio generated when the wearable device rotates. In the disclosed embodiment, after the wearable device obtains the rendered spatial audio data, the perception deviation parameter may be determined based on the first rotation information and the second rotation information, and the rendered spatial audio data may be rendered based on the perception deviation parameter to obtain the target spatial audio data.

[0124] In the disclosed embodiment, the rendering spatial audio data can be rendered based on the perception deviation parameters generated by the audio when the wearable device rotates to obtain the target spatial audio data. The secondary rendering of the audio data can be achieved. At the same time, the first rotation information obtained during the first rotation is combined with the second rotation information currently obtained, so that the obtained target spatial audio data can correspond to the current rotation state of the wearable device, thereby making the obtained target spatial audio data have a stronger sense of space.

[0125] In some embodiments, the perception deviation parameter includes: a delay deviation parameter and / or a sound pressure deviation parameter. The determining of the perception deviation parameter based on the first rotation information and the second rotation information includes:

[0126] Determine the compensation rotation information based on the first rotation information and the second rotation information, and determine the delay deviation parameter based on the compensation rotation information and a preset correlation coefficient; and / or

[0127] A first sound pressure parameter corresponding to the first rotation information and a second sound pressure parameter corresponding to the second rotation information are acquired, and the sound pressure deviation parameter is determined based on the first sound pressure parameter and the second sound pressure parameter.

[0128] Here, the delay deviation parameter may include a time parameter related to the rotation information. For example, it may include: Interaural Time Difference (ITD). The sound pressure deviation parameter may include a sound pressure parameter related to the rotation information. For example, it may include: Interaural Level Difference (ILD).

[0129] In some embodiments, the compensating rotation information may be determined based on the difference between the first rotation information and the second rotation information. For example, if the first rotation information is a first rotation angle, the second rotation information is a second rotation angle, and the compensating rotation information is a compensating rotation angle, the difference between the first rotation angle and the second rotation angle may be determined as the compensating rotation angle.

[0130] In some other embodiments, determining the compensating rotation information based on the first rotation information and the second rotation information includes:

[0131] Performing weighted processing on the first rotation information based on a first weight coefficient to obtain first weighted rotation information;

[0132] performing weighted processing on the second rotation information based on a second weight coefficient to obtain second weighted rotation information;

[0133] The compensating rotation information is determined based on a difference between the first weighted rotation information and the second rotation information.

[0134] In some embodiments, the first weight coefficient and the second weight coefficient may be set as needed, for example, based on experience or historical data, etc., which is not specifically limited here.

[0135] In some embodiments, the first rotation information may be a first rotation angle, the second rotation information may be a second rotation angle, and the compensation rotation information may be a compensation rotation angle. In this way, the first rotation angle may be weighted based on the first weight coefficient to obtain a first weighted rotation angle, and the second rotation angle may be weighted based on the second weight coefficient to obtain a second weighted rotation angle, and then the compensation rotation angle may be determined based on the difference between the first weighted rotation angle and the second rotation angle. For example, the difference between the first weighted rotation angle and the second rotation angle may be directly determined as the compensation rotation angle. For another example, the difference between the first weighted rotation angle and the second rotation angle may be weighted, and the result of the weighted processing may be determined as the compensation rotation angle, etc.

[0136] In the disclosed embodiment, the first rotation information can be weighted based on the first weight coefficient, and the second rotation information can be weighted based on the second weight coefficient, and the compensation rotation information can be obtained based on the difference between the weighted first weighted rotation information and the weighted second weighted rotation information. In this way, the proportion of the first rotation information and the second rotation information in the compensation rotation information can be adjusted by adjusting the first weight coefficient and the second weight coefficient, that is, the user can adjust the proportion of the first rotation information and the second rotation information in the audio rendering process as needed, thereby improving the flexibility of audio rendering.

[0137] In the disclosed embodiment, after obtaining the compensation rotation information, the delay deviation parameter can be determined based on the compensation rotation information and the preset correlation coefficient. The preset correlation coefficient can be set as needed or obtained based on experiments. For example, it can be determined based on the propagation speed of sound and the radius of the human head during the test. In some embodiments, the calculation formula of the delay deviation parameter is as follows:

[0138]

[0139]

[0140] In formulas (1) and (2), ITD represents the delay deviation parameter, λ represents the preset correlation coefficient, θ represents the compensation rotation information, c represents the speed of sound, and a represents the radius of the human head.

[0141] In some embodiments, a mapping relationship between rotation information and sound pressure parameters can be preset, so that when the rotation information is determined, the corresponding sound pressure parameters can be determined according to the rotation information and the mapping relationship. Taking the rotation information as the rotation angle as an example, during the test, the impulse response of all rotation angles corresponding to the HRTF can be convolved with the white noise signal w to obtain the rendered white noise signal c corresponding to the rotation angle, and then the sound pressure parameters corresponding to each rotation angle are determined based on the rendered white noise signal c, and a mapping relationship between each rotation angle and the sound pressure parameter is established.

[0142] In some embodiments, the calculation formula of the sound pressure deviation parameter is as follows:

[0143]

[0144] In formula (3), ILD represents the sound pressure deviation parameter, represents the first sound pressure parameter, represents the second sound pressure parameter.

[0145] The sound pressure parameter can be determined based on the rotation information. In some embodiments, the sound pressure parameter is calculated as follows:

[0146]

[0147] In formula (4), represents the sound pressure parameter, Indicates rendering white noise, where m is the length of the audio data of c.

[0148] In some embodiments, the impulse responses of all rotation angles corresponding to the HRTF may be convolved with the white noise signal w to obtain a rendered white noise signal c corresponding to the rotation angle. The calculation formula of the rendered white noise signal is as follows:

[0149]

[0150] In formula (5), represents the rendered white noise signal, w represents the initial white noise signal, φ n Indicates the rotation angle corresponding to the HRTF.

[0151] In the disclosed embodiment, the delay deviation parameter can be determined based on the compensating rotation information determined by the first rotation information and the second rotation information, and the sound pressure deviation parameter can be determined based on the first sound pressure parameter corresponding to the first rotation information and the second sound pressure parameter corresponding to the second rotation information. The first rotation information and the second rotation information can be fully considered in the audio rendering process. In this way, even if the wearable device rotates multiple times during the audio rendering process, it will not affect the spatial sense of the target space audio data finally output.

[0152] In some embodiments, rendering the rendering spatial audio data based on the perception deviation parameter to obtain the target spatial audio data includes:

[0153] Delay processing is performed on the rendered spatial audio data based on the delay deviation parameter, and / or

[0154] The sound pressure of the rendering spatial audio data is adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data.

[0155] Here, taking the perception deviation parameter as the delay deviation parameter as an example, the rendering spatial audio data can be delayed based on the delay deviation parameter. In some embodiments, delaying the rendering spatial audio data can include: delaying the rendering spatial audio data corresponding to the farther channel, for example, if the left ear is farther from the sound source, the rendering spatial audio data of the channel corresponding to the left ear can be delayed.

[0156] Taking the perception deviation parameter as a sound pressure deviation parameter as an example, the sound pressure of the rendered spatial audio data can be adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data. Taking the perception deviation parameter as an example, including a delay deviation parameter and a sound pressure deviation parameter, the rendered spatial audio data can be delayed based on the delay deviation parameter, and the sound pressure of the rendered spatial audio data can be adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data.

[0157] In the embodiment of the present disclosure, a delay deviation parameter and / or a sound pressure deviation parameter can be determined based on the first rotation information, and the rendering spatial audio data can be delayed based on the delay deviation parameter, and / or the sound pressure of the rendering spatial audio data can be adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data.

[0158] In some embodiments, the decoding of the encoded data to obtain the rendered spatial audio data and the first rotation information includes:

[0159] Decoding the encoded data based on a first decoder to obtain pre-encoded data and the first rotation information;

[0160] The pre-encoded data is decoded based on a second decoder to obtain the rendered spatial audio data.

[0161] In some embodiments, the type of the first decoder and the type of the second decoder may be the same, or the type of the first decoder and the type of the second decoder may be different. In some embodiments, the encoded data may be a data packet including pre-encoded data and first rotation information, and the first decoder may be a data decoder for parsing the data packet. After receiving the encoded data, the wearable device may parse the encoded data based on the first decoder to obtain the pre-encoded data and the first rotation information. The second decoder may be an audio decoder for decoding audio data. For example, the pre-encoded data may be input into the second decoder, and then the pre-encoded data may be decoded to obtain rendered spatial audio data.

[0162] Since the audio decoder does not have the function of parsing data packets, in the embodiments of the present disclosure, different decoders can be set according to different data types to achieve parsing of different types of data, and different types of data can be placed in the same data packet for transmission, thereby reducing the possibility of delays caused by sending data separately.

[0163] Figure 4 is a flowchart 2 of an audio processing method according to an exemplary embodiment. Figure 4 As shown, the method is applied to an electronic device, and the method mainly comprises the following steps:

[0164] In step 401, first rotation information sent by a wearable device is received;

[0165] In step 402, the initial audio data is rendered based on the first rotation information to obtain rendered spatial audio data;

[0166] In step 403, the rendering spatial audio data and the first rotation information are encoded to obtain encoded data;

[0167] In step 404, the encoded data is sent to the wearable device, so that the wearable device performs rendering processing on the rendering spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data.

[0168] Here, the method can be applied to electronic devices. The electronic devices may include terminal devices. The terminal devices may include mobile terminals and fixed terminals, such as mobile phones, tablet computers, PDAs, laptop computers, desktop computers, wearable devices, smart speakers, televisions, and vehicle-mounted terminals.

[0169] Wearable devices include: portable devices that are worn directly on the user or integrated into the user's accessories. For example, they include: wearable devices supported by the wrist (including watches, wristbands, etc.); wearable devices supported by the foot (including shoes, socks, etc.), wearable devices supported by the head (including headphones, glasses, helmets, headbands, etc.). They can also include: smart clothing, school bags, crutches, accessories, etc.

[0170] In the embodiment of the present disclosure, when the wearable device rotates, the first rotation information of the wearable device can be obtained and sent to the electronic device. Here, the first rotation information may include: information generated when the wearable device rotates, for example, it may include the rotation angle and rotation direction of the wearable device. In some embodiments, a detection module can be set in the wearable device, and when the wearable device is detected to rotate, the detection module can sense the first rotation information of the wearable device.

[0171] Taking the first rotation information as the first rotation angle as an example, an angular motion detection module can be set in the wearable device, and the first rotation angle of the wearable device is determined by the angular motion detection module, wherein the angular motion detection module may include: an angular velocity sensor (gyroscope). Taking the first rotation information as the first rotation orientation as an example, an orientation detection module can be set in the wearable device, and the first rotation orientation of the wearable device is determined by the orientation detection module, wherein the orientation detection module may include: an orientation sensor.

[0172] In the embodiment of the present disclosure, after acquiring the first rotation information, the wearable device may send the first rotation information to the electronic device. In some embodiments, a communication connection between the wearable device and the electronic device may be established. For example, a wired connection may be established between the wearable device and the electronic device, wherein the wired connection may include: a data line connection; a wireless connection may also be established between the wearable device and the electronic device, wherein the wireless connection may include: a short-range wireless connection (e.g., a Bluetooth connection) and a long-range wireless connection (e.g., a Wi-Fi connection). For example, Bluetooth communication may be established between the electronic device and the wearable device.

[0173] In some embodiments, the electronic device is a device that outputs initial audio data, and the wearable device is a device that outputs target spatial audio data, wherein the target spatial audio data is spatial audio obtained by rendering the initial audio data.

[0174] After receiving the first rotation information, the electronic device may render the initial audio data based on the first rotation information to obtain rendered spatial audio data.

[0175] In some embodiments, the rendering of the initial audio data based on the first rotation information to obtain rendered spatial audio data includes:

[0176] Based on the first rotation information and the spatial transfer function, obtaining a related impulse response corresponding to the first rotation information;

[0177] The initial audio data and the relevant impulse response are convolved to obtain the rendered spatial audio data.

[0178] Here, after receiving the first rotation information, the electronic device may render the initial audio data based on the first rotation information to obtain rendered spatial audio data. In some embodiments, the initial audio data may be rendered using a spatial transfer function, for example, the initial audio data may be rendered using an HRTF to obtain rendered spatial audio data. When constructing a head-related transfer function, the relevant impulse responses corresponding to each angle in the space may be collected in advance, and a mapping relationship between each angle and the relevant impulse responses corresponding to each angle may be established to construct a data set.

[0179] In the embodiment of the present disclosure, after obtaining the first rotation information, the electronic device can obtain the relevant impulse response corresponding to the first rotation information from the data set corresponding to the spatial transfer function based on the first rotation information, and then perform convolution processing on the initial audio data and the relevant impulse response to obtain rendered spatial audio data.

[0180] Taking the first rotation information being the first rotation angle as an example, a related impulse response corresponding to the first rotation angle can be obtained based on the first rotation angle, and the initial audio data and the related impulse response can be convolved to obtain rendered spatial audio data. Here, the rendered spatial audio data is spatial audio obtained by rendering the initial audio data via an electronic device.

[0181] In some embodiments, the rendering spatial audio data may be first encoded based on an encoding algorithm to obtain pre-encoded data, and then the pre-encoded data and the first rotation information may be repackaged to obtain encoded data.

[0182] In some embodiments, encoding the rendering spatial audio data and the first rotation information to obtain encoded data includes:

[0183] Encoding the rendering spatial audio data based on a first encoder to obtain pre-encoded data;

[0184] The pre-encoded data and the first rotation information are encoded based on a second encoder to obtain the encoded data.

[0185] In some embodiments, the type of the first encoder and the type of the second encoder may be the same, or the type of the first encoder and the type of the second encoder may be different. In some embodiments, the first encoder may be an audio encoder for encoding audio data. For example, the rendering spatial audio data may be input into the first encoder, and then the rendering spatial audio data may be encoded to obtain pre-encoded data. In some embodiments, the second encoder may be a data encoder for integrating different types of data into a data packet. For example, the pre-encoded data and the first rotation information may be input into the second encoder for encoding to obtain encoded data. The encoded data may be a data packet including the pre-encoded data and the first rotation information.

[0186] For example, an audio encoder (including a Bluetooth audio encoder) can be used to complete the encoding and packaging of the rendering spatial audio data to obtain pre-coded data (enc_data), and then a data encoder can be used to perform secondary packaging of the pre-coded data (enc_data) and the first rotation information (angle_data) according to a preset packaging format to obtain coded data. After obtaining the coded data, the electronic device can send the coded data to the wearable device.

[0187] Correspondingly, after receiving the encoded data, the wearable device can decode the encoded data. For example, before the Bluetooth audio decoder processes the encoded data, the encoded data can be parsed and processed first, and after the encoded data is verified based on the check code and determined to be normal, the pre-encoded data and the first rotation information are obtained, and then the pre-encoded data is input into the Bluetooth audio decoder for decoding, thereby obtaining the rendered spatial audio data, wherein the first rotation information may include: angle information for spatial audio rendering.

[0188] In other embodiments, the rendering spatial audio data and the first rotation information may be directly encoded by an encoding algorithm to obtain encoded data. For example, the rendering spatial audio data and the first rotation information may be encoded directly based on an audio encoding protocol. Correspondingly, after receiving the encoded data, the wearable device may decode the encoded data. For example, the encoded data may be directly decoded based on an audio decoding protocol to obtain the rendering spatial audio data and the first rotation information.

[0189] In the disclosed embodiment, the wearable device sends the acquired first rotation information to the electronic device, receives the encoded data sent by the electronic device, decodes the encoded data to obtain rendered spatial audio data and the first rotation information, and renders the rendered spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data, wherein the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information.

[0190] The technical solution disclosed in the present invention, on the one hand, adopts encoding to directly transmit the rendered spatial audio data and the first rotation information, and there will be no delay between the two, thus avoiding the secondary delay caused by returning the first rotation information in other solutions, thereby reducing the sense of delay brought to the user by the audio rendering; on the other hand, it can perform audio rendering on both the electronic device and the wearable device at the same time, which can improve the accuracy and spatial sense of audio rendering, thereby improving the texture of the generated target spatial audio data.

[0191] Figure 5 The following is a flow chart of an audio processing method according to an exemplary embodiment. Figure 3 ,like Figure 5 As shown, when the wearable device 501 rotates, the wearable device 501 can obtain the first rotation information by using the motion detection module, and send the first rotation information to the electronic device 502 by wireless communication. After receiving the first rotation information, the electronic device 502 can render the initial audio data based on the first rotation information by using the spatial transfer function to obtain the rendered spatial audio data, and input the rendered spatial audio data and the first rotation information into the encoder to obtain the encoded data, and send the encoded data to the wearable device 501 by wireless communication. After receiving the encoded data, the wearable device 501 can input the encoded data into the decoder, decode to obtain the first rotation information and the rendered spatial audio data, and determine the perception deviation parameter based on the first rotation information and the obtained second rotation information, and then perform audio rendering on the rendered spatial audio data based on the perception deviation parameter to obtain the target spatial audio data, and output the target spatial audio data to the user.

[0192] The technical solution disclosed in the present invention, on the one hand, adopts encoding to directly transmit the rendered spatial audio data and the first rotation information, and there will be no delay between the two, thus avoiding the secondary delay caused by returning the first rotation information in other solutions, thereby reducing the sense of delay brought to the user by the audio rendering; on the other hand, it can perform audio rendering on both the electronic device and the wearable device at the same time, which can improve the accuracy and spatial sense of audio rendering, thereby improving the texture of the generated target spatial audio data.

[0193] Furthermore, in the embodiments of the present disclosure, since the HRTF rendering of the first rotation information (e.g., the first rotation angle) is performed in the electronic device (e.g., a mobile phone), the externalization sense and the positioning of the first rotation information are guaranteed. At the same time, after the time (e.g., ITD) and sound pressure (e.g., ILD) rendering is performed in the wearable device (e.g., headphones), the orientation can be closer to the second rotation information, avoiding obvious delays. At the same time, the rendered spatial audio data and the first rotation information are directly transmitted in an encoded manner, and there will be no delay between the two, avoiding the secondary delay caused by other solutions returning the first rotation information, thereby reducing the sense of delay brought to the user by audio rendering. The technical solution of the present disclosure can ensure that the delay of the head tracking function is small enough, the tracking accuracy is higher, the sound quality is better, and the sense of space is better without increasing the power consumption and computing power of the wearable device and modifying the system delay.

[0194] Figure 6 FIG. 1 is a block diagram of an audio processing device according to an exemplary embodiment. Figure 6 As shown, the audio processing device 600 mainly includes:

[0195] A first sending module 601 is configured to send first rotation information of the wearable device to an electronic device;

[0196] A first receiving module 602 is configured to receive encoded data sent by the electronic device; wherein the encoded data is obtained by the electronic device encoding the rendered spatial audio data and the first rotation information, and the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information;

[0197] A decoding module 603 is configured to decode the encoded data to obtain the rendering spatial audio data and the first rotation information;

[0198] An acquisition module 604 is configured to acquire second rotation information of the wearable device;

[0199] The first rendering module 605 is configured to perform rendering processing on the rendering spatial audio data based on the first rotation information and the second rotation information to obtain target spatial audio data.

[0200] In some embodiments, the first rendering module 605 is configured to:

[0201] determining a perception bias parameter based on the first rotation information and the second rotation information;

[0202] The rendering spatial audio data is rendered based on the perception deviation parameter to obtain the target spatial audio data.

[0203] In some embodiments, the perception deviation parameter includes: a delay deviation parameter and / or a sound pressure deviation parameter; the first rendering module 605 is configured to:

[0204] Delay processing is performed on the rendered spatial audio data based on the delay deviation parameter, and / or

[0205] The sound pressure of the rendering spatial audio data is adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data.

[0206] In some embodiments, the first rendering module 605 is configured to:

[0207] Determine the compensation rotation information based on the first rotation information and the second rotation information, and determine the delay deviation parameter based on the compensation rotation information and a preset correlation coefficient; and / or

[0208] A first sound pressure parameter corresponding to the first rotation information and a second sound pressure parameter corresponding to the second rotation information are acquired, and the sound pressure deviation parameter is determined based on the first sound pressure parameter and the second sound pressure parameter.

[0209] In some embodiments, the first rendering module 605 is configured to:

[0210] Performing weighted processing on the first rotation information based on a first weight coefficient to obtain first weighted rotation information;

[0211] performing weighted processing on the second rotation information based on a second weight coefficient to obtain second weighted rotation information;

[0212] The compensating rotation information is determined based on a difference between the first weighted rotation information and the second rotation information.

[0213] In some embodiments, the decoding module 603 is configured to:

[0214] Decoding the encoded data based on a first decoder to obtain pre-encoded data and the first rotation information;

[0215] The pre-encoded data is decoded based on a second decoder to obtain the rendered spatial audio data.

[0216] Figure 7 FIG. 1 is a block diagram of an audio processing device according to an exemplary embodiment. Figure 7 As shown, the audio processing device 700 mainly includes:

[0217] The second receiving module 701 is configured to receive first rotation information sent by the wearable device;

[0218] A second rendering module 702 is configured to render the initial audio data based on the first rotation information to obtain rendered spatial audio data;

[0219] An encoding module 703 is configured to encode the rendering spatial audio data and the first rotation information to obtain encoded data;

[0220] The second sending module 704 is configured to send the encoded data to the wearable device, so that the wearable device renders the rendering spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data.

[0221] In some embodiments, the second rendering module 702 is configured to:

[0222] Based on the first rotation information and the spatial transfer function, obtaining a related impulse response corresponding to the first rotation information;

[0223] The initial audio data and the relevant impulse response are convolved to obtain the rendering spatial audio data.

[0224] In some embodiments, the encoding module 703 is configured to:

[0225] Encoding the rendering spatial audio data based on a first encoder to obtain pre-encoded data;

[0226] The pre-encoded data and the first rotation information are encoded based on a second encoder to obtain the encoded data.

[0227] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0228] Figure 8 8 is a hardware structure block diagram of an audio processing device 800 according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, a wearable device, etc.

[0229] Reference Figure 8 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0230] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0231] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0232] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0233] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0234] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0235] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0236] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800 and the temperature change of the device 800. The sensor assembly 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0237] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0238] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0239] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0240] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an audio processing device, enables the audio processing device to perform an audio processing method, the method comprising:

[0241] Sending first rotation information of the wearable device to the electronic device;

[0242] Receiving encoded data sent by the electronic device; wherein the encoded data is obtained by the electronic device encoding the rendered spatial audio data and the first rotation information, and the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information;

[0243] Decoding the encoded data to obtain the rendering spatial audio data and the first rotation information;

[0244] Acquire second rotation information of the wearable device;

[0245] Based on the first rotation information and the second rotation information, the rendering spatial audio data is rendered to obtain target spatial audio data.

[0246] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an audio processing device, enables the audio processing device to perform an audio processing method, the method comprising:

[0247] Receiving first rotation information sent by the wearable device;

[0248] Rendering the initial audio data based on the first rotation information to obtain rendered spatial audio data;

[0249] Encoding the rendering space audio data and the first rotation information to obtain encoded data;

[0250] The encoded data is sent to the wearable device, so that the wearable device performs rendering processing on the rendering spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data.

[0251] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0252] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio processing method, characterized in that: include: Sending first rotation information of the wearable device to the electronic device; Receiving encoded data sent by the electronic device; wherein the encoded data is obtained by the electronic device encoding the rendered spatial audio and the first rotation information, and the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information; Decoding the encoded data to obtain the rendering spatial audio data and the first rotation information; Acquire second rotation information of the wearable device; Based on the first rotation information and the second rotation information, the rendering spatial audio data is rendered to obtain target spatial audio data.

2. The method according to claim 1, characterized in that: The rendering spatial audio data is rendered based on the first rotation information and the second rotation information to obtain target spatial audio data, including: determining a perception bias parameter based on the first rotation information and the second rotation information; The rendering spatial audio data is rendered based on the perception deviation parameter to obtain the target spatial audio data.

3. The method according to claim 2, characterized in that The perception deviation parameter includes: a delay deviation parameter and / or a sound pressure deviation parameter; and the rendering spatial audio data is rendered based on the perception deviation parameter to obtain the target spatial audio data, including: Delay processing is performed on the rendered spatial audio data based on the delay deviation parameter, and / or The sound pressure of the rendering spatial audio data is adjusted based on the sound pressure deviation parameter to obtain the target spatial audio data.

4. The method according to claim 3, characterized in that The determining of the perception deviation parameter based on the first rotation information and the second rotation information includes: Determine the compensation rotation information based on the first rotation information and the second rotation information, and determine the delay deviation parameter based on the compensation rotation information and a preset correlation coefficient; and / or A first sound pressure parameter corresponding to the first rotation information and a second sound pressure parameter corresponding to the second rotation information are acquired, and the sound pressure deviation parameter is determined based on the first sound pressure parameter and the second sound pressure parameter.

5. The method according to claim 2, characterized in that: The determining the compensation rotation information based on the first rotation information and the second rotation information includes: Performing weighted processing on the first rotation information based on a first weight coefficient to obtain first weighted rotation information; performing weighted processing on the second rotation information based on a second weight coefficient to obtain second weighted rotation information; The compensating rotation information is determined based on a difference between the first weighted rotation information and the second rotation information.

6. The method according to any one of claims 1 to 5, characterized in that: The decoding process of the encoded data to obtain the rendering space audio data and the first rotation information includes: Decoding the encoded data based on a first decoder to obtain pre-encoded data and the first rotation information; The pre-encoded data is decoded based on a second decoder to obtain the rendered spatial audio data.

7. An audio processing method, characterized in that: include: Receiving first rotation information sent by the wearable device; Rendering the initial audio data based on the first rotation information to obtain rendered spatial audio data; Encoding the rendering space audio data and the first rotation information to obtain encoded data; The encoded data is sent to the wearable device, so that the wearable device performs rendering processing on the rendering spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data.

8. The method according to claim 7, characterized in that The rendering of the initial audio data based on the first rotation information to obtain rendered spatial audio data includes: Based on the first rotation information and the spatial transfer function, obtaining a related impulse response corresponding to the first rotation information; The initial audio data and the relevant impulse response are convolved to obtain the rendered spatial audio data.

9. The method according to claim 7 or 8, characterized in that: The step of encoding the rendering space audio data and the first rotation information to obtain encoded data includes: Encoding the rendering spatial audio data based on a first encoder to obtain pre-encoded data; The pre-encoded data and the first rotation information are encoded based on a second encoder to obtain the encoded data.

10. An audio processing device, characterized in that: include: A first sending module, configured to send first rotation information of the wearable device to the electronic device; A first receiving module is configured to receive encoded data sent by the electronic device; wherein the encoded data is obtained by the electronic device encoding the rendered spatial audio data and the first rotation information, and the rendered spatial audio data is obtained by the electronic device rendering the initial audio data based on the first rotation information; A decoding module, configured to decode the encoded data to obtain the rendering spatial audio data and the first rotation information; An acquisition module, configured to acquire second rotation information of the wearable device; The first rendering module is configured to perform rendering processing on the rendering spatial audio data based on the first rotation information and the second rotation information to obtain target spatial audio data.

11. An audio processing device, characterized in that: include: A second receiving module is configured to receive first rotation information sent by the wearable device; A second rendering module is configured to render the initial audio data based on the first rotation information to obtain rendered spatial audio data; an encoding module, configured to encode the rendering spatial audio data and the first rotation information to obtain encoded data; The second sending module is configured to send the encoded data to the wearable device, so that the wearable device renders the rendering spatial audio data based on the first rotation information and the acquired second rotation information to obtain target spatial audio data.

12. An audio processing device, characterized in that: include: processor; a memory configured to store processor-executable instructions; Wherein, the processor is configured to: implement the steps in any one of the audio processing methods in claims 1 to 9 when executed.

13. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an audio processing device, the device is enabled to perform any one of the audio processing methods of claims 1 to 9.

Images