Audio processing method and apparatus, audio device, audio system, and storage medium

Abstract

Embodiments of the present disclosure provide an audio processing method and apparatus, an audio device, an audio system, and a storage medium. The audio processing method comprises: acquiring first-type sound channel data and second-type sound channel data of a first audio, wherein the first-type sound channel data is used for output by a first-type audio device, and the second-type sound channel data is used for output by a second-type audio device; and acquiring third-type sound channel data of the first audio, wherein the third-type sound channel data is used for output by a device that has a better output effect for the third-type sound channel data than the first-type audio device and the second-type audio device.

Description

Audio processing method and device, audio equipment, audio system and storage medium Technical Field

[0001] The present disclosure relates to the field of audio technology, and in particular to an audio processing method and apparatus, an audio device, an audio system, and a storage medium. Background Art

[0002] With the development of communication technology, the demand for immersive audio playback is increasing. Conventional multimedia devices can only support stereo playback. How to achieve immersive audio playback effects is a problem that continues to be solved in related technologies.

[0003] Summary of the Invention

[0004] The embodiments of the present disclosure provide a technical solution for achieving an immersive audio playback effect.

[0005] Embodiments of the present disclosure provide an audio processing method and apparatus, an audio device, and a storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, an audio processing method is provided, including: obtaining first-category channel data and second-category channel data of a first audio; wherein the first-category channel data is used for output by a first-category audio device; and the second-category channel data is used for output by a second-category audio device.

[0007] According to a second aspect of an embodiment of the present disclosure, there is provided an audio device, comprising:

[0008] The processing module is configured to obtain first-category channel data and second-category channel data of the first audio; wherein the first-category channel data is used for output of the first-category audio device; and the second-category channel data is used for output of the second-category audio device.

[0009] According to a third aspect of an embodiment of the present disclosure, there is provided an audio processing system, comprising: a first type of audio device configured to output first type of channel data of a first audio;

[0010] The second type of audio device is configured to output second type of channel data of the first audio.

[0011] According to a fourth aspect of the embodiments of the present disclosure,

[0012] An audio device is provided, wherein the audio device comprises:

[0013] one or more processors;

[0014] The processor is used to call instructions to enable the audio device to execute the audio processing method provided by any technical solution of the first aspect.

[0015] According to a fifth aspect of an embodiment of the present disclosure, a storage medium is provided, wherein the storage medium stores instructions, which, when executed on an audio device, enable the audio device to execute the audio processing method provided by any technical solution of the first aspect.

[0016] The technical solution provided by the embodiment of the present disclosure uses two types of audio devices to output different types of channel data respectively, so that the immersive effect of the first audio output can be achieved with the help of multiple devices.

[0017] It should be understood that the foregoing general description and the following detailed description are merely exemplary and explanatory and are not restrictive of the embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0019] FIG1 is a schematic diagram showing an architecture of an audio system according to an exemplary embodiment;

[0020] FIG2 is a schematic flow chart showing an audio processing method according to an exemplary embodiment;

[0021] FIG3 is a schematic diagram of a process for matching sound field reproduction capabilities according to an exemplary embodiment;

[0022] FIG4 is a schematic diagram showing a process of matching a sound field reproduction capability according to an exemplary embodiment;

[0023] FIG5 is a schematic diagram of a process for matching sound field reproduction capabilities according to an exemplary embodiment;

[0024] FIG6 is a schematic flow chart showing an audio processing method according to an exemplary embodiment;

[0025] FIG7 is a schematic structural diagram of an audio device according to an exemplary embodiment;

[0026] FIG8A is a schematic structural diagram of an audio device according to an exemplary embodiment;

[0027] FIG8B is a schematic structural diagram of a chip according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure provide an audio processing method and apparatus, an audio device, an audio system, and a storage medium.

[0029] In a first aspect, an embodiment of the present disclosure provides an audio processing method, wherein the method includes: obtaining first-category channel data and second-category channel data of a first audio; wherein the first-category channel data is used for output of a first-category audio device; and the second-category channel data is used for output of a second-category audio device.

[0030] In some embodiments, a first type of audio device and a second type of audio device are used to output first type of channel data and second type of channel data respectively, thereby improving the audio output effect.

[0031] In conjunction with some embodiments of the first aspect,

[0032] The channel data of the first audio further includes third-category channel data; wherein a center frequency of the third-category channel data is lower than or equal to a center frequency of the first-category channel data and / or a center frequency of the second-category channel data;

[0033] The third-category sound channel data is output by the device having a better output effect on the third-category sound channel data between the first-category audio device and the second-category audio device.

[0034] Based on the above embodiment, for the third-category sound channel data with a low center frequency, the first-category audio device or the second-category audio device with better effect will be selected for output, thereby ensuring the output effect of the low-frequency component.

[0035] In combination with some embodiments of the first aspect, the first type of audio device is a speaker device; and the second type of audio device is a head-mounted audio device.

[0036] Based on the above solution, the immersive audio effect can be achieved by combining the speaker device and the head-mounted audio device.

[0037] In conjunction with some embodiments of the first aspect, the first type of channel data includes at least one of the following:

[0038] Front channel data;

[0039] Surround channel data;

[0040] Height channel data.

[0041] Based on the above solution, these channel data are provided to the first type of audio device for output, and the output effect of these channel data can be improved by using a high-power speaker.

[0042] In combination with some embodiments of the first aspect, the second type of channel data includes: binaural audio data.

[0043] Based on the above solution, a head-mounted audio device is used to output binaural audio data, thereby forming a good playback effect at the listening ear.

[0044] In conjunction with some embodiments of the first aspect, the method further includes:

[0045] The sound field reproduction capabilities of the first-category audio device and the second-category audio device are matched.

[0046] Based on the above solution, by matching the sound field reproduction capabilities of the first type of audio equipment and the second type of audio equipment, the overall effect of the audio output can be further improved, thereby achieving an immersive audio effect.

[0047] In conjunction with some embodiments of the first aspect, the method further includes:

[0048] Determining a first transfer parameter of the first type of audio device; wherein the first transfer parameter is used to determine a change in a sound wave when the audio output of the first type of audio device reaches a target object;

[0049] Determining a second transfer parameter of the second type of audio device; the second transfer parameter is used to indicate a change in a sound wave when the second type of audio device outputs audio that reaches the target object;

[0050] The matching of the sound field reproduction capabilities of the first-category audio device and the second-category audio device includes:

[0051] The sound field reproduction capabilities of the first type of audio device and the second type of audio device are matched according to the first transfer parameter and the second transfer parameter.

[0052] Based on the above solution, the first transfer parameter and the second transfer parameter are determined, and the sound field reproduction capabilities of the first type of audio device and the second type of audio device are matched based on the first transfer parameter and the second transfer parameter.

[0053] In conjunction with some embodiments of the first aspect, determining the first transfer parameter of the first-category audio device includes:

[0054] Collecting the first-category sound channel data output by the first-category audio device to obtain a second audio;

[0055] The first transfer parameter of the first type of audio device is obtained according to the first audio and the second audio.

[0056] Based on the above solution, the second type of audio device is used to collect the audio output by the first type of audio device, thereby simply obtaining the first transfer parameter, thereby realizing dynamic determination of the first transfer parameter of the first type of audio device.

[0057] In conjunction with some embodiments of the first aspect, determining the first transfer parameter of the first-category audio device includes:

[0058] Get the spatial parameters of the target space where the target object is located;

[0059] Performing space modeling based on the space parameters and the location of the first type of audio device;

[0060] The first transfer parameter is determined based on the spatial modeling.

[0061] Based on the above scheme, the first transfer parameter of the first type of audio equipment is determined by space modeling, which can be conveniently used in a known space.

[0062] In conjunction with some embodiments of the first aspect, matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device based on the first transfer parameter and the second transfer parameter includes:

[0063] determining target parameters for matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device based on the first transfer parameter, the second transfer parameter, device parameters, and output parameters; the device parameters including device parameters of the first-category audio device and / or the second-category audio device; and the output parameters including audio output parameters of the first-category audio device and / or the second-category audio device;

[0064] The output parameters of the first-category audio device and / or the second-category audio device are adjusted according to the target parameters to achieve matching of sound field reproduction capabilities of the first-category audio device and the second-category audio device.

[0065] Based on the above solution, sound field matching between the first-category audio device and the second-category audio device is ultimately achieved by adjusting the output parameters of the first-category audio device and / or the second-category audio device based on the first transfer parameter, the second transfer parameter, the device parameter, and the output parameter. In conjunction with some embodiments of the first aspect, matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device includes:

[0066] Collecting the first-category sound channel data output by the first-category audio device to obtain a second audio;

[0067] determining target parameters for matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device based on the first audio, the second audio, device parameters, and output parameters; the device parameters including device parameters of the first-category audio device and / or the second-category audio device; and the output parameters including output parameters of the audio of the first-category audio device and / or the second-category audio device;

[0068] The output parameters of the first type of audio device and / or the second type of audio device are adjusted according to the target parameters.

[0069] Based on the above solution, target parameters for matching the sound field reproduction capabilities of the first and second audio devices are directly determined according to the first audio, the second audio, the device parameters and the output parameters, which has the characteristic of simple implementation.

[0070] In conjunction with some embodiments of the first aspect, matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device includes:

[0071] Get the spatial parameters of the target space where the target object is located;

[0072] Performing space modeling according to the space parameters and the location of the first type of audio device;

[0073] Determining target parameters for matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device based on the spatial modeling, device parameters, and output parameters; the device parameters including device parameters of the first-category audio device and / or the second-category audio device; and the output parameters including audio output parameters of the first-category audio device and / or the second-category audio device.

[0074] The output parameters of the first type of audio device and / or the second type of audio device are adjusted according to the target parameters to achieve matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device.

[0075] Based on the above scheme, the target parameters are directly determined according to the space modeling, device parameters and output parameters, thereby easily achieving the matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device.

[0076] In a second aspect, an embodiment of the present disclosure provides an audio device, comprising:

[0077] The processing module is configured to obtain first-category channel data and second-category channel data of the first audio; wherein the first-category channel data is used for output of the first-category audio device; and the second-category channel data is used for output of the second-category audio device.

[0078] In a third aspect, an embodiment of the present disclosure provides an audio processing system, comprising: a first type of audio device configured to output first type of channel data of a first audio;

[0079] The second type of audio device is configured to output second type of channel data of the first audio.

[0080] In conjunction with some embodiments of the third aspect, the channel data of the first audio further includes third-category channel data; wherein a center frequency of the third-category channel data is lower than or equal to a center frequency of the first-category channel data and / or a center frequency of the second-category channel data;

[0081] The audio device with better output effect of the third category audio data among the first category audio device and the second category audio device is further configured to output the third category audio data.

[0082] According to a fourth aspect, an audio device is provided, wherein the audio device includes:

[0083] one or more processors;

[0084] The processor is configured to call instructions to enable the audio device to execute the audio processing method according to any one of claims 1 to 12.

[0085] In a fifth aspect, an embodiment of the present disclosure provides a storage medium, wherein the storage medium stores instructions, which, when executed on an audio device, enable the audio device to execute the audio processing method described in the optional implementation manner of the first aspect.

[0086] In a sixth aspect, an embodiment of the present disclosure provides a program product. When the program product is executed by an audio device, the device executes the audio processing method described in the optional implementation manner of the first aspect.

[0087] In a seventh aspect, an embodiment of the present disclosure provides a computer program, which, when executed on a computer, enables the computer to execute the audio processing method described in the optional implementation of the first aspect, the second aspect, or the third aspect.

[0088] It is understandable that the above-mentioned first-category audio device, second-category audio device, audio processing system, storage medium, program product, and computer program are all used to execute the method provided by the embodiments of the present disclosure. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods and will not be repeated here.

[0089] The present disclosure provides an audio processing method and apparatus, an audio device, an audio system, and a storage medium. In some embodiments, the terms "audio processing method" and "audio output method" are interchangeable, as are "information processing method" and "data processing method"; "information indicating device" and "information processing device" are interchangeable, as are "information transmission device"; and "audio system" and "information processing system" are interchangeable.

[0090] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0091] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0092] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0093] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when articles such as "a", "an", "the" in English are used in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0094] In the embodiments of the present disclosure, “plurality” refers to two or more.

[0095] In some embodiments, the terms "at least one," "one or more," "a plurality of," "multiple," etc. may be used interchangeably.

[0096] In some embodiments, descriptions such as "at least one of A and B," "A and / or B," "in one case A, in another case B," or "in one case A, in another case B" may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); and in some embodiments, A and B (both A and B are executed). The same applies when there are more branches such as A, B, and C.

[0097] In some embodiments, "A or B" and other descriptions may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The above is also applicable when there are more branches such as A, B, C, etc.

[0098] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0099] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0100] In some embodiments, terms such as "...", "determine...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0101] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0102] In some embodiments, devices, etc. can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. Terms such as "device", "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", and "subject" can be used interchangeably.

[0103] In some embodiments, an audio processing system is provided, which may include: a first type of audio device and a second type of audio device.

[0104] In some embodiments, the first type of audio device and the second type of audio device may be different types of audio devices.

[0105] In some embodiments, the first type of audio device may have audio capture and / or audio output capabilities.

[0106] In other embodiments, the second type of audio device may have audio acquisition and / or audio output capabilities.

[0107] In some embodiments, the output power of the first type of audio device may be higher than the output capability of the second type of audio device.

[0108] In some embodiments, the first type of audio device may have a speaker.

[0109] In some embodiments, the second type of audio device may be a head-mounted audio device, which may include but is not limited to an in-ear audio device, and / or a near-ear audio device.

[0110] In some embodiments, the first type of audio device and the second type of audio device can form an audio processing system that can be used to simultaneously output different channel data of the same audio, or simultaneously output the same audio device, thereby improving the output effect of the audio device. For example, using this audio processing system, an immersive audio effect can be achieved.

[0111] In other embodiments, the audio processing system may include: a control device.

[0112] Audio processing system 100 Audio processing system 100. Connections may be established between various devices in the audio processing system 100. The connections may be wired connections and / or wireless connections.

[0113] In some embodiments, the wireless connection may include, but is not limited to, a Bluetooth connection and / or a WiFi connection.

[0114] As shown in FIG2 , an embodiment of the present disclosure relates to an audio processing method, which is used in an audio processing system 100. The method includes:

[0115] S2101: Acquire first-category channel data and second-category channel data of first audio.

[0116] In some embodiments, the first audio may be multi-channel audio.

[0117] In some embodiments, the first audio may be various audios capable of providing stereo and / or surround sound effects and / or immersive sound effects.

[0118] Exemplarily, the first audio may be 5.1 audio and / or 7.1 audio, etc.

[0119] In some embodiments, the first type of audio device obtains the first type of channel data and the second type of channel data of the first audio.

[0120] In some other embodiments, the second type of audio device obtains the first type of channel data and the second type of channel data of the first audio.

[0121] In some other embodiments, the control device obtains the first type of channel data and the second type of channel data of the first audio.

[0122] In some embodiments, the first type of audio device, the second type of audio device and / or the control device separates the first audio to obtain first type of channel data and second type of channel data.

[0123] In some embodiments, the first type of channel data and the second type of channel data are data of different channels.

[0124] In some embodiments, the first type of audio device and the second type of audio device may be different types of audio devices.

[0125] In some embodiments, the first type of audio device may be an audio device having audio output capabilities.

[0126] In other embodiments, the first type of audio device may also be an audio device with audio acquisition capability.

[0127] In some embodiments, the second type of audio device may be an audio device having audio output capability.

[0128] In some embodiments, the second type of audio device may be an audio device with audio acquisition capability.

[0129] In some embodiments, the first type of audio device and the second type of audio device may be different types of audio devices.

[0130] In some embodiments, the first type of audio device may be an audio device having a speaker.

[0131] Exemplarily, the first category of audio devices may be devices such as smart TVs and / or smart speakers.

[0132] In other embodiments, the second type of audio device may be a head-mounted audio device, specifically an in-ear audio device and / or a near-ear audio device.

[0133] Exemplarily, the second category of audio devices may include: Bluetooth headphones and / or smart glasses, etc.

[0134] In some embodiments, the first type of channel data includes at least one of the following:

[0135] Front channel data;

[0136] Surround channel data;

[0137] Height channel data.

[0138] In some embodiments, the front channel data may be front channel data, specifically including one of the following channel data:

[0139] Left front channel (LF) data;

[0140] Centre Front (CF) data;

[0141] Right front channel (RF) data.

[0142] In some embodiments, the surround channel data may be audio data of the surround channels. For example, the surround channels may be plane channels other than the designated channels. For example, the designated channels may include but are not limited to the aforementioned front channels.

[0143] In some embodiments, the surround channel data may be channel data formed using a plurality of audio output components (eg, speakers).

[0144] Exemplarily, the surround sound channel data may include at least one of the following:

[0145] Left surround channel (Left Surround, LS) data;

[0146] Right surround channel (Right Surround, RS) data;

[0147] Left rear channel (LR) data;

[0148] Right rear channel (RR) data.

[0149] In some embodiments, the height channel data is audio data of a height channel. Typically, the height channel may be a channel located above the head of a target object (eg, an audience).

[0150] In some embodiments, the height channel data may include, but is not limited to, one of the following:

[0151] Left front height channel (Left Top Front, Ltf) data;

[0152] Right front height channel (Right Top Front, Rtf) data;

[0153] Left Top Rear (Ltr) data;

[0154] Right Top Rear (Ltr) data

[0155] Centre Top (CT) data.

[0156] In some embodiments, the second-category channel data may be audio data of various channels other than the first-category channel data.

[0157] In some embodiments, the second type of channel data includes: binaural audio data.

[0158] In some embodiments, the binaural audio data may be binaural audio data obtained by downscaling using stereo or immersive audio.

[0159] In some embodiments, a head-related transfer function (HRTF) is convolved according to the position of the target object to obtain binaural audio data of the left and right channels of the head-mounted device.

[0160] In some embodiments, the channel data of the first audio may further include a third type of channel data.

[0161] In some embodiments, the third type of channel data may be simply referred to as bass channel data.

[0162] In some embodiments, the center frequency of the third type of channel data is lower than or equal to the center frequency of the first type of channel data and / or the center frequency of the second type of channel data.

[0163] In some embodiments, the center frequency may be a median value of a frequency range obtained after the channel data of the first audio is converted into the frequency domain.

[0164] In some other embodiments, the average frequency of the third type of channel data is lower than or equal to the center frequency of the first type of channel data and / or the average frequency of the second type of channel data.

[0165] In some other embodiments, the lowest frequency of the third type of channel data is lower than or equal to the center frequency of the first type of channel data and / or the lowest frequency of the second type of channel data.

[0166] In some embodiments, the third type of channel data may have frequency overlapping portions with the first type of channel data and the second type of channel data.

[0167] In some other embodiments, the third type of channel data may be channel data labeled as a bass channel in the first audio. For example, channel 1 in 5.1 audio is the bass channel. For another example, channel 1 in 7.1 audio is the bass channel.

[0168] S2102: Determine whether the first type of audio device or the second type of audio device outputs the third type of channel data.

[0169] In some embodiments, the audio device that outputs the third type of channel data is selected based on the low-frequency output effects of the first type of audio device and the second type of audio device.

[0170] In some embodiments, the audio device that outputs the third type of channel data is selected based on the low frequency output capabilities of the first type of audio device and the second type of audio device.

[0171] The low-frequency output capability may be reflected by at least one of the following parameters:

[0172] The lower limit of output frequency supported by Class 1 and Class 2 audio devices;

[0173] transient response of Class I and Class II audio equipment;

[0174] frequency responses of Class I and Class II audio devices;

[0175] Harmonic distortion for Class I and Class II audio equipment;

[0176] Non-harmonic distortion for Class I and Class II audio devices.

[0177] Exemplarily, an audio device that supports a lower output frequency limit is selected as the output device for the third type of channel data.

[0178] Exemplarily, an audio device with better frequency response is selected as the output device for the third type of channel data.

[0179] As another example, an audio device with better transient response is selected as the output device for the third type of channel data.

[0180] Also illustratively, an audio device with better transient frequency response in the bass band is selected as the output device for the third type of channel data.

[0181] Also illustratively, an audio device with low harmonic distortion is selected as the output device for the third type of channel data.

[0182] As another example, an audio device with smaller harmonic distortion in the bass frequency band is selected as the output device for the third type of channel data.

[0183] Also exemplarily, an audio device with small non-harmonic distortion is selected as the output device of the third type of channel data.

[0184] As another example, an audio device with smaller non-harmonic distortion in the bass frequency band is selected as the output device for the third type of channel data.

[0185] It is worth noting that S2102 is an optional step. For example, the third-category sound channel data is sent to the first-category audio device and the second-category audio device at the same time, and the first-category audio device and the second-category audio device simultaneously output the third-category sound channel data.

[0186] S2103: Match the sound field reproduction capabilities of the first-category audio device and the second-category audio device.

[0187] In some embodiments, before the first type of audio device and the second type of audio device output the first audio, the sound field reproduction capabilities of the first type of audio device and the second type of audio device are matched.

[0188] In other embodiments, during the process in which the first type of audio device and the second type of audio device output the first audio, the sound field reproduction capabilities of the first type of audio device and the second type of audio device are matched.

[0189] In some embodiments, the matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device is performed by the first type of audio device.

[0190] In other embodiments, the second type of audio device performs the matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device.

[0191] In some other embodiments, the control device performs the matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device.

[0192] That is, in some embodiments, there is no fixed execution order between S2103, S2101, and S2102. In some embodiments, S2103 may be executed before S2101, or S2103 may be executed during the execution of S2103 and S2102.

[0193] The following are several options for matching the sound field reproduction capabilities of Class 1 audio devices and Class 2 audio devices:

[0194] Option 1:

[0195] As shown in FIG3 , matching the sound field reproduction capability may include:

[0196] S3101: Determine a first transmission parameter of the first type of audio device.

[0197] In some embodiments, the first transfer parameter is used to determine (or reflect) the change in sound waves when the audio output by the first type of audio device reaches the target object.

[0198] S3102: Determine a second transmission parameter of the second-category audio device.

[0199] In some embodiments, the second transfer parameter is used to indicate the sound wave changes when the second type of audio device outputs audio to reach the target object.

[0200] S3103: Match the sound field reproduction capabilities of the first-category audio device and the second-category audio device according to the first transfer parameter and the second transfer parameter.

[0201] The first transfer parameter reflects the attenuation and / or final effect of the audio output by each first-category audio device reaching the ears of the listener (target object).

[0202] The second transfer parameter reflects the attenuation and / or final effect of the audio output by the second type of audio device reaching the ears of the listener (target object) after being affected by the wearer's head.

[0203] In some embodiments, the first transfer parameter may include, but is not limited to: a first transfer function.

[0204] In some embodiments, the second transfer parameter may include, but is not limited to: a second transfer function.

[0205] The first transfer function may be used to determine how well the audio output by the first type of audio device reaches the ears of the target subject.

[0206] The second transfer function may be used to determine how well the audio output by the second type of audio device reaches the ears of the target subject.

[0207] Matching is performed by means of sound field reproduction capabilities so that the audio outputs of the first type of audio device and the second type of audio device meet similarity and / or harmony requirements.

[0208] The second transfer function of the second type of audio device can obtain the effect of the output audio reaching the ears of the target object based on the influence of the human brain on the audio output.

[0209] In some embodiments, the first transfer function of the first type of audio device may be obtained in the following manner:

[0210] Method a:

[0211] Recording the audio playback effect of the first type of audio device to obtain a second audio;

[0212] A first transfer function is obtained by function fitting according to the difference between the first audio and the second audio.

[0213] In some embodiments, the second-type audio device has an audio acquisition capability, and the second-type audio device can be used to acquire the audio output of the first-type audio device to obtain the second audio. In this way, the second audio can be acquired and the first transfer function of the first-type audio device can be obtained without the need for external equipment.

[0214] Method b:

[0215] The determining of the first transfer parameter of the first type of audio device includes:

[0216] Get the spatial parameters of the target space where the target object is located;

[0217] Performing space modeling based on the space parameters and the location of the first type of audio device;

[0218] The first transfer parameter is determined based on the spatial modeling.

[0219] In some embodiments, the spatial parameters may describe the shape and / or configuration of the target space, etc.

[0220] In some embodiments, the space parameter may also describe whether the space is an open space and / or a closed space.

[0221] In some embodiments, the space parameter may also be used to indicate the length, width, and / or height of the target space.

[0222] In some embodiments, the spatial parameters determine the propagation path of the audio output by the first type of audio device.

[0223] In some embodiments, space modeling can be performed based on the space parameters and the location of the first type of audio device. For example, the first type of audio device can be located in the target space, or can be located outside the target space and play audio into the target space.

[0224] In summary, in the embodiment of the present disclosure, a control model for sound wave transmission of audio playback is performed based on the spatial parameters and the location of the first type of audio device, thereby obtaining the first transmission parameter.

[0225] Two optional ways of determining the first transfer parameter are provided above, but the specific implementation is not limited to the above examples.

[0226] Matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device according to the first transfer parameter and the second transfer parameter includes:

[0227] determining target parameters for matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device based on the first transfer parameter, the second transfer parameter, device parameters, and output parameters; the device parameters including device parameters of the first-category audio device and / or the second-category audio device; and the output parameters including audio output parameters of the first-category audio device and / or the second-category audio device;

[0228] The output parameters of the first-category audio device and / or the second-category audio device are adjusted according to the target parameters to achieve matching of sound field reproduction capabilities of the first-category audio device and the second-category audio device.

[0229] In some embodiments, device parameters may include, but are not limited to, at least one of the following:

[0230] The device type of the audio device;

[0231] Hardware parameters of the audio device;

[0232] Audio output power;

[0233] Supported audio output range;

[0234] Hardware parameters of the audio output component.

[0235] In some embodiments, the output parameter may be a parameter of the audio output by the first type of audio device and / or the second type of audio device.

[0236] Exemplarily, the output parameter may include: current audio output power of the audio device and / or a frequency response of the audio output.

[0237] In the embodiment of the present disclosure, target parameters for matching the sound field reproduction capabilities of the first type of audio device and the second type of audio device are determined based on the first transfer parameter, the second transfer parameter, the device parameter, and the output parameter.

[0238] After obtaining the target parameters, the output parameters can be adjusted according to the current output parameters of the first and second audio devices so that the adjusted output parameters of the first and second audio devices are equal to the target parameters.

[0239] Option 2:

[0240] As shown in FIG4 , the sound field matching between the first type of audio device and the second type of audio device provided by the embodiment of the present disclosure may include:

[0241] S3201: Collect the first-category sound channel data output by the first-category audio device to obtain a second audio;

[0242] S3202: Determine target parameters for matching sound field reproduction capabilities of the first-category audio device and the second-category audio device based on the first audio, the second audio, device parameters, and output parameters.

[0243] In some embodiments, the device parameters include: device parameters of the first category of audio devices and / or the second category of audio devices.

[0244] In some embodiments, the output parameters include: output parameters of the audio of the first-category audio device and / or the second-category audio device.

[0245] S3203: Adjust output parameters of the first-category audio device and / or the second-category audio device according to the target parameters.

[0246] In the embodiment of the present disclosure, the target parameters for matching the sound field reproduction capabilities of the first type of audio device and the second type of audio device are directly solved based on the collected second audio, first audio, device parameters and output parameters, without obtaining intermediate quantities for conversion.

[0247] After the target parameters are solved, adjustments are made based on the current output parameters of the first type of audio device and the second type of audio device so that the final output parameters of the first type of audio device and / or the second type of audio device are equal to the target parameters, thereby achieving matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device.

[0248] Option 3:

[0249] As shown in FIG5 , matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device includes:

[0250] S3301: Obtaining the spatial parameters of the target space where the target object is located;

[0251] S3302: Perform space modeling based on the space parameters and the location of the first type of audio device;

[0252] S3303: Determine target parameters when the sound field reproduction capabilities of the first type of audio device and the second type of audio device match based on the space modeling, device parameters, and output parameters.

[0253] In some embodiments, the device parameters include: device parameters of the first category of audio devices and / or the second category of audio devices.

[0254] In some municipalities, the output parameters include: output parameters of the audio of the first-category audio device and / or the second-category audio device;

[0255] S3304: Adjust output parameters of the first-category audio device and / or the second-category audio device according to the target parameters to match the sound field reproduction capabilities of the first-category audio device and the second-category audio device.

[0256] In the embodiment of the present disclosure, the target parameters are directly solved based on space modeling, device parameters and output parameters, and the sound field reproduction capabilities of the first and second audio devices are matched by adjusting the output parameters of the first and second audio devices to the target parameters.

[0257] In some embodiments, at least the device that performs sound field reproduction capability matching may be a second-class audio device. The second-class audio device collects the audio output effect of the first-class audio device to obtain a second audio, and adjusts its own output parameters based on the second audio to achieve sound field reproduction capability matching between the first-class audio device and the second-class audio device.

[0258] In some embodiments, any one of the first-class audio device, the second-class audio device and / or the control device separates the channel data of the first audio, and matches the sound field reproduction capability by adjusting one or more output parameters of the first-class audio device and / or the second-class audio device.

[0259] It is worth noting that the channel matching solutions of the above-mentioned solutions 1 to 3 can be used as technical solutions for the current audio system to play channel data to improve the audio playback effect, and can also be executed separately to achieve the matching of the sound field reproduction capabilities of multiple audio devices for audio playback.

[0260] S2104: The first type of audio device outputs the first type of channel data.

[0261] In some embodiments, the first type of audio device outputs first type of channel data and third type of channel data.

[0262] S2105: The second type of audio device outputs the second type of channel data.

[0263] In some embodiments, the second type of audio device outputs the first type of channel data and the third type of channel data.

[0264] It is worth noting that this audio processing method may include S2104 and S2105, meaning that steps S2101 through S2103 are optional. For example, the separation of the channel data and the playback device for the third-category audio data can be performed by a device outside the audio processing system. To achieve high-quality audio output, the first-category audio device and the second-category audio device may have previously matched their sound field reproduction capabilities.

[0265] In some embodiments, the audio processing method may perform steps S2101, S2104, and S2105, meaning that steps S2102 and S2103 are optional. For example, the third-category sound channel data may be transmitted simultaneously to the first-category audio device and the second-category audio device, and output simultaneously by the first-category audio device and the second-category audio device. Sound field reproduction capability matching may be pre-established.

[0266] In some embodiments, the audio processing method may include S2101, and the other steps may be optional. For example, the control device of the audio system separates the channel data and transmits the separated channel data to the first type of audio device and the second type of audio device, and the execution steps are completed for the control device.

[0267] In some embodiments, the audio processing method may include steps S2101 and S2102, with all other steps being optional. For example, the control device of the audio system separates the channel data, determines which audio device is to play the third-category audio data, and transmits the separated channel data to the first-category audio device and the second-category audio device. For the control device, these steps are complete.

[0268] In some embodiments, the audio processing method may include steps S2103 to S2105, with all other steps being optional. For example, the audio system receives separated channel data from a peripheral device, the first and second audio devices in the audio system match their sound field reproduction capabilities, and output their designated channel data, thereby ensuring audio playback quality.

[0269] In some embodiments, the demand for external immersive audio is increasing, but mainstream multimedia devices only support stereo external speakers. One of the main reasons is that the two speakers for stereo playback only need to be arranged in a single line, which can be integrated into most multimedia devices. Multimedia devices in this context may include but are not limited to televisions, speakers, and / or mobile phones.

[0270] Immersive audio often requires more channels, which means more speakers. For example, a classic 5.1 surround sound audio system requires two speakers behind or to the sides of the user in addition to the speakers in front.

[0271] Speakers need to be set up around the user. For example, speakers can be placed in a polygon or circle around the user. For spatial audio (such as 5.1.4), four more speakers need to be placed above the user's head. The speakers need to occupy the space above and to the sides of the head, and the space above and around the user. In addition, the speaker layout has a great impact on the immersive audio effect and has additional layout requirements. The difficulty of arranging speakers further increases the difficulty for ordinary users to use multi-channel audio systems.

[0272] On the other hand, speaker systems are not very compatible. The sound image reconstructed by the speaker system only appears between two adjacent speakers. Generally speaking, audio formats can be divided into mono, stereo, surround sound, and spatial audio. Speaker systems are almost not backward compatible.

[0273] Without the aforementioned multiple speaker locations, headphones are typically used to downmix multi-channel audio to binaural audio. This downmixing reduces the number of channels in the resulting audio data, and when the downmixed audio is played back on a device with fewer channels, the playback quality of the pre-downmixed audio is restored as closely as possible.

[0274] However, headphones have a typical head-in-the-head effect, resulting in a narrower reconstructed sound field that cannot match the sound field reconstructed by a high-fidelity speaker system. Therefore, when users have a high-fidelity speaker system (stereo, surround sound) but want to experience more complex audio formats (such as spatial audio), they have to make a choice: first, use a high-fidelity speaker system to obtain better sound quality but a limited-dimensional sound field, and second, use other technical solutions to obtain a multi-dimensional sound field but limited sound quality.

[0275] Therefore, the disclosed embodiments propose a high-fidelity immersive audio playback solution that is both portable and high-fidelity. One approach is to combine open-ear headsets (such as AR glasses and bone conduction headphones) with multimedia devices (such as mobile phones, TVs, home theaters, and professional audio systems) to form a high-fidelity immersive audio system that supports playback of multiple audio sources in formats such as 5.1, 7.1, and 7.1.4, providing a high-fidelity listening experience for all audio formats.

[0276] Using multiple speakers and placing them around the listener so that sound is emitted from all directions requires multiple playback devices for playback, which places high demands on the number and layout of playback devices, and may not meet high sound quality requirements, resulting in a poor experience.

[0277] You can also use a certain number of speakers and utilize the sound bar principle for high-fidelity playback. The principle of the sound bar is to reflect the sound waves of the corresponding channel through the wall to reach the human ear. Therefore, although the speakers are (generally) in the front, the sound waves of the corresponding channel arrive from other angles.

[0278] However, since loudspeaker directivity is nearly impossible to control, it can be considered omnidirectional, meaning it radiates in all directions. Therefore, sound waves from a particular channel will not only reflect off walls but also reach the ear directly as direct sound. Furthermore, because direct sound has a shorter path, and walls absorb sound, reflected sound waves are easily masked by direct sound waves, making the desired effect virtually impossible. Furthermore, the energy difference in all directions, other than the front and back, is minimal.

[0279] As a real speaker device, smart speakers also have layout requirements and need to occupy additional physical space.

[0280] Therefore, the embodiments of the present disclosure propose an immersive audio distributed playback solution that is both portable and high-fidelity, supporting the combination of open-ear head-mounted devices (hereinafter referred to as head-mounted devices) and stereo multimedia devices (such as mobile phones, televisions, and audio systems) to form a high-fidelity immersive audio system.

[0281] Speaker playback requires speakers placed in the appropriate orientation. Surround sound requires at least rear or side speakers, and 3D spatial audio requires additional speakers at a certain height. In many scenarios, it's difficult to place speakers in the appropriate positions. Downmixing without speakers in the appropriate orientation is virtually impossible to achieve the desired effect.

[0282] Using headphones alone to play back the signal downmixed from multi-channel to binaural audio will have limited playback effects. For example, due to the head-in-the-head effect and the narrowing of the sound field, it is difficult to achieve the high-fidelity effect expected by users.

[0283] As shown in Figure 6, the technical solution of the disclosed embodiment combines a head-mounted device and the speakers of a multimedia device into a new distributed audio system. This enables portable and high-fidelity immersive audio playback. The head-mounted device can simulate channels that are not supported by the actual speaker system (or channels that are poorly downmixed), achieving an effect that exceeds that of using speakers or headphones alone.

[0284] Used to separate audio channels into channels suitable for speaker playback and channels suitable for headset playback.

[0285] Channels are divided into four categories:

[0286] Bass channel, front channels (LF, CF, RF), surround channels (LS, RS, LR, RR are plane channels other than the front channels), height channels. Here, height channels refer to channels arranged at a certain height higher than the plane channels.

[0287] The channels can be separated as follows:

[0288] Used to separate audio channels into channels suitable for speaker playback and channels suitable for headset playback.

[0289] For the bass channel, the bass channel itself in the multi-channel audio (such as the .1 channel of 5.1), and / or the components of all other channels below the crossover point (according to the parameters obtained by the parameter calculation module).

[0290] Low-frequency management may include: processing the bass channel in a certain way and sending it to appropriate equipment based on relevant parameters.

[0291] Generally, the sound system that contains the subwoofer is more suitable for the bass channel.

[0292] The bass channel can be sent to the speaker device and head-mounted device without being processed, or it can be managed at low frequencies to achieve better low-frequency effects.

[0293] For speaker channels, the speaker system plays the channels it supports (as well as some channels suitable for speaker upmixing / downmixing).

[0294] In principle, the speaker system is only suitable for up / down mixing of channels in the same direction (front channels, surround channels, height channels).

[0295] Illustratively, the front channels of the speaker system include LF and RF, and the LF and RF channels can be used to be compatible with the CF channel.

[0296] Audio designed for 7.1 channels can be played by a 5.1 speaker system.

[0297] However, stereo speaker systems (front channels only) are not compatible with 5.1, 7.1, 5.1.4, 7.1.4, and other audio formats that include surround and height channels. Surround and height channels can be provided by headsets.

[0298] Surround sound speaker systems are not compatible with 5.1.4, 7.1.4, or other formats that include height channels. Height channels can be provided by headsets. In this case, the speaker system can play both front and surround channels.

[0299] The channels for head-mounted device playback can be as follows:

[0300] Send channels that aren't supported by the speakers and / or aren't downmixed well to the headset.

[0301] The signal is first downmixed into binaural audio, adapted to the speaker signal, played in the headset, and combined with the sound played by the speakers to reconstruct a higher-fidelity sound field.

[0302] Note: For example, the sound bed and multi-channel ambient sound tracks (generally including channels other than CF) in the object audio can all be played by the head-mounted device to achieve a more integrated sound field effect.

[0303] Binaural signal downmixing may include:

[0304] The corresponding signal is convolved with the HRTF obtained according to its relative position to the listener to obtain the binaural signal of the corresponding channel.

[0305] The binaural signals of all corresponding channels are superimposed to obtain binaural audio data that the head-mounted device needs to play back. This binaural audio data can also be called a binaural signal.

[0306] The sound field reproduction capability matching (or adaptation) of the speakers and headset can be as follows:

[0307] Since the parameters of speakers and head-mounted devices and the paths from the devices to the human ears vary greatly, there will inevitably be significant differences in the listening experience. Generally, the differences between devices can be clearly distinguished.

[0308] In order to make different devices sound similar, the audio of the headset and the signal of the speaker need to be adapted so that there is almost no noticeable difference between the different devices, thus obtaining a continuous and harmonious sound field.

[0309] The adaptation processing module makes the sound waves radiated by the two devices equal in terms of objective parameters affecting human ear perception, such as sound pressure, frequency response, and reverberation, based on relevant parameters, so that different devices can achieve consistent listening experience at the human ear.

[0310] Before matching the sound field reproduction capability, parameter calculation is required. The specific parameter calculation can be as follows:

[0311] Parameter calculation is mainly based on two methods:

[0312] Method 1: Calculate based on the audio signal collected on the headset and the original audio signal.

[0313] Method 2: Acquire spatial information based on the collected information (such as depth information), so as to model the current space and obtain the acoustic characteristics of the current space and its impact on sound wave transmission.

[0314] Based on the hardware performance parameters of the headset and / or speakers, as well as relevant audio settings, the speakers and headset are modeled. These settings may include, but are not limited to, volume and equalizer settings. During modeling, combined with spatial modeling, the characteristics of sound perceived by the human ear are calculated to determine relevant parameters. These parameters may include the aforementioned output parameters, etc.

[0315] The two methods can each independently meet the needs of parameter calculation, or they can be combined with each other.

[0316] In summary, the present disclosure provides an immersive audio distributed playback method:

[0317] The immersive audio is divided into two parts, which are played back using different playback methods. The two parts of the playback audio are merged into a high-quality immersive audio sound field in the listener's ear canal.

[0318] Of the two parts, the first part can be played back based on a speaker device, and the second part can be played back based on an earphone device.

[0319] For channels other than the front speakers, precise sound image positioning isn't necessary. This is because the human ear has a lower threshold for sound detection in the front and a higher threshold for sound detection in the back. Therefore, in sound design, the main sound and instrumental sounds are primarily placed in the front, while the sides and top are primarily for ambient sound effects.

[0320] Open-ear headphones: They have no effect on the human ear's perception of external sounds, and generally have poor sound quality. However, as a binaural device, they are very suitable for immersive audio playback (binaural audio is suitable for downmixing of different audio formats).

[0321] Speaker system: It can achieve very high sound quality and an audio experience that exceeds that of binaural devices, but it is difficult to expand and it is almost impossible to adapt to more complex audio formats due to physical principles.

[0322] The disclosed embodiment combines the better sound quality of the speaker system and the interoperability of binaural devices for multiple audio formats, using open-ear headphones as an expansion device for the speaker system to supplement the sound channels that the actual speaker system does not support, thereby enabling the existing speaker system to support multiple audio formats while ensuring high sound quality.

[0323] The embodiments of the present disclosure also provide an apparatus for implementing any of the above methods. For example, an apparatus is provided, and the apparatus includes units or modules for implementing each step executed by the terminal in any of the above methods.

[0324] It should be understood that the division of the various units or modules in the above devices is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above devices, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above units or modules are realized by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by software called by the processor, and the rest by hardware circuits.

[0325] In the embodiments of the present disclosure, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP); in another implementation, the processor can implement certain functions through the logical relationship of the hardware circuit, and the logical relationship of the above-mentioned hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by a processor as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0326] Figure 7 is a schematic diagram of the structure of an audio device provided by an embodiment of the present disclosure. The audio device may be the aforementioned first-category audio device and / or the second-category audio device.

[0327] As shown in FIG7 , the audio device provided by an embodiment of the present disclosure includes: a processing module 601 configured to obtain first-category channel data and second-category channel data of a first audio; wherein the first-category channel data is used for output by a first-category audio device; and the second-category channel data is used for output by a second-category audio device.

[0328] The audio device may further include an audio module, which may be used for outputting and / or collecting audio.

[0329] In some embodiments, the audio module may include: an audio output module and / or an audio acquisition module.

[0330] In some embodiments, the audio output module can be used for audio output.

[0331] In other embodiments, the audio acquisition module can be used for acquisition.

[0332] For example, if the audio device is a first-category audio device, the audio output module may include: a speaker.

[0333] For another example, if the audio device is a second-category audio device, the audio output module may be headphones, etc.

[0334] In some embodiments, the audio device further includes a transceiver module.

[0335] The transceiver module can support wired communication and / or wireless communication.

[0336] Wireless communication may include, but is not limited to, Bluetooth communication and / or WiFi communication.

[0337] Optionally, the transceiver module can be used by the audio device to execute the steps related to sending in the aforementioned audio processing method, which will not be described in detail here.

[0338] Figure 8A is a schematic diagram of the structure of an audio device 8100 provided in an embodiment of the present disclosure. Audio device 8100 can be a terminal or network device (e.g., an access network device or a core network device), or a terminal (e.g., a user equipment (UE)). It can also be a chip, chip system, or processor that supports a network device to implement any of the above methods. It can also be a chip, chip system, or processor that supports a terminal to implement any of the above audio processing methods. Audio device 8100 can be used to implement the audio processing methods described in the above method embodiments. For details, please refer to the description of the above method embodiments.

[0339] As shown in Figure 8A, the audio device 8100 includes one or more processors 8101. The processor 8101 can be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control communication devices (such as base stations, baseband chips, terminal devices, terminal device chips, DUs or CUs, etc.), execute programs, and process program data. The processor 8101 is used to call instructions to enable the audio device 8100 to perform any of the above audio processing methods.

[0340] In some embodiments, the audio device 8100 further includes one or more memories 8102 for storing instructions. Optionally, all or part of the memories 8102 may be located outside the audio device 8100.

[0341] In some embodiments, the audio device 8100 further includes one or more transceivers 8103. When the audio device 8100 includes one or more transceivers 8103, the communication steps such as sending and receiving in the above method are performed by the transceiver 8103, and the other steps are performed by the processor 8101.

[0342] In some embodiments, a transceiver may include a receiver and a transmitter, which may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0343] Optionally, the audio device 8100 further includes one or more interface circuits 8104, which are connected to the memory 8102. The interface circuits 8104 can be used to receive signals from the memory 8102 or other devices, and can be used to send signals to the memory 8102 or other devices. For example, the interface circuits 8104 can read instructions stored in the memory 8102 and send the instructions to the processor 8101.

[0344] The audio device 8100 described in the above embodiment may be a network device or a terminal, but the scope of the audio device 8100 described in the present disclosure is not limited thereto, and the structure of the audio device 8100 may not be limited to FIG8A. The audio device may be an independent device or may be part of a larger device. For example, the audio device may be: (1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data and programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, a vehicle-mounted device, a network device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0345] 8B is a schematic diagram of the structure of a chip 8200 provided in an embodiment of the present disclosure. If the audio device 8100 can be a chip or a chip system, please refer to the schematic diagram of the structure of the chip 8200 shown in FIG8B , but the present disclosure is not limited thereto.

[0346] The chip 8200 includes one or more processors 8201 , and the processor 8201 is used to call instructions so that the chip 8200 executes any of the above audio processing methods.

[0347] In some embodiments, chip 8200 further includes one or more interface circuits 8202, which are connected to memory 8203. Interface circuit 8202 can be used to receive signals from memory 8203 or other devices, and can be used to send signals to memory 8203 or other devices. For example, interface circuit 8202 can read instructions stored in memory 8203 and send the instructions to processor 8201. Optionally, the terms interface circuit, interface, transceiver pin, and transceiver are interchangeable.

[0348] In some embodiments, the chip 8200 further includes one or more memories 8203 for storing instructions. Alternatively, all or part of the memories 8203 may be outside the chip 8200.

[0349] The present disclosure also provides a storage medium having instructions stored thereon. When the instructions are executed on the audio device 8100, the audio device 8100 is caused to perform any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but may also be a transient storage medium.

[0350] The present disclosure further provides a program product, which, when executed by the audio device 8100, enables the audio device 8100 to perform any of the above audio processing methods. Optionally, the program product is a computer program product.

[0351] The present disclosure also provides a computer program, which, when executed on a computer, enables the computer to execute any one of the above audio processing methods.

[0352] Other embodiments of the presently disclosed embodiments will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the presently disclosed embodiments that follow the general principles of the presently disclosed embodiments and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered as exemplary only, with the true scope and spirit of the presently disclosed embodiments being indicated by the following claims.

[0353] It should be understood that the embodiments of the present disclosure are not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the embodiments of the present disclosure is limited only by the appended claims.

Claims

1. An audio processing method, wherein: The method comprises: The first channel data and the second channel data of the first audio are obtained; wherein the first channel data is used for output of the first audio device; and the second channel data is used for output of the second audio device.

2. The method according to claim 1, wherein: The channel data of the first audio further includes third-category channel data; wherein the center frequency of the third-category channel data is lower than or equal to the center frequency of the first-category channel data and / or the center frequency of the second-category channel data; The third-category sound channel data is used for output by the device having a better output effect on the third-category sound channel data between the first-category audio device and the second-category audio device.

3. The method according to claim 1 or 2, wherein: The first type of audio device is a speaker device; the second type of audio device is a head-mounted audio device.

4. The method according to any one of claims 1 to 3, wherein: The first type of channel data includes at least one of the following: Front channel data; Surround channel data; Height channel data.

5. The method according to any one of claims 1 to 4, wherein: The second type of channel data includes: Binaural audio data.

6. The method according to any one of claims 1 to 5, wherein: The method further comprises: The sound field reproduction capabilities of the first-category audio device and the second-category audio device are matched.

7. The method according to claim 6, wherein: The method further comprises: Determine a first transfer parameter of the first type of audio device; wherein the first transfer parameter is used to determine a change in sound waves when the audio output of the first type of audio device reaches a target object; Determining a second transfer parameter of the second type of audio device; the second transfer parameter is used to determine the change in sound waves when the audio output by the second type of audio device reaches the target object; The matching of the sound field reproduction capabilities of the first-category audio device and the second-category audio device includes: The sound field reproduction capabilities of the first type of audio device and the second type of audio device are matched according to the first transfer parameter and the second transfer parameter.

8. The method according to claim 7, wherein: The determining of a first transfer parameter of the first type of audio device comprises: Collecting the first-category sound channel data output by the first-category audio device to obtain a second audio; The first transfer parameter of the first type of audio device is obtained according to the first audio and the second audio.

9. The method according to claim 7, wherein: The determining of a first transfer parameter of the first type of audio device comprises: Get the spatial parameters of the target space where the target object is located; Performing space modeling according to the space parameters and the location of the first type of audio device; Based on the spatial modeling, the first transfer parameter is determined.

10. The method according to any one of claims 7 to 9, wherein: The matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device according to the first transfer parameter and the second transfer parameter includes: Determine target parameters for matching the sound field reproduction capabilities of the first-category audio device and the second-category audio device according to the first transfer parameter, the second transfer parameter, device parameters, and output parameters; the device parameters include: device parameters of the first-category audio device and / or the second-category audio device; the output parameters include: output parameters of the audio of the first-category audio device and / or the second-category audio device; The output parameters of the first-category audio device and / or the second-category audio device are adjusted according to the target parameters to achieve matching of the sound field reproduction capabilities of the first-category audio device and the second-category audio device.

11. The method according to claim 6, wherein: The matching of the sound field reproduction capabilities of the first-category audio device and the second-category audio device includes: Collecting the first-category sound channel data output by the first-category audio device to obtain a second audio; According to the first audio, the second audio, device parameters and output parameters, target parameters are determined when the sound field reproduction capabilities of the first type of audio device and the second type of audio device are matched; the device parameters include: device parameters of the first type of audio device and / or the second type of audio device; the output parameters include: Output parameters of device audio; The output parameters of the first type of audio device and / or the second type of audio device are adjusted according to the target parameters.

12. The method according to claim 6, wherein: The matching of the sound field reproduction capabilities of the first-category audio device and the second-category audio device includes: Get the spatial parameters of the target space where the target object is located; Performing space modeling according to the space parameters and the location of the first type of audio device; Determine target parameters for sound field matching between the first-category audio device and the second-category audio device according to the space modeling, device parameters, and output parameters; the device parameters include: device parameters of the first-category audio device and / or the second-category audio device; the output parameters include: output parameters of audio of the first-category audio device and / or the second-category audio device; The output parameters of the first type of audio device and / or the second type of audio device are adjusted according to the target parameters to achieve matching of the sound field reproduction capabilities of the first type of audio device and the second type of audio device.

13. An audio device, wherein: include: A processing module is configured to obtain first-category channel data and second-category channel data of a first audio; wherein the first-category channel data is used for output of a first-category audio device; The second-category sound channel data is used for output by a second-category audio device.

14. An audio processing system, wherein: include: A first-category audio device configured to output first-category sound channel data of a first audio; The second type of audio device is configured to output the second type of channel data of the first audio.

15. The system of claim 14, wherein: include: The channel data of the first audio further includes third-category channel data; wherein the center frequency of the third-category channel data is lower than or equal to the center frequency of the first-category channel data and / or the center frequency of the second-category channel data; The audio device with better output effect of the third category audio data among the first category audio device and the second category audio device is also configured to output the third category audio data.

16. An audio device, wherein: The audio device comprises: one or more processors; The processor is used to call instructions so that the audio device executes the audio processing method according to any one of claims 1 to 12.

17. A storage medium, wherein: The storage medium stores instructions, and when the instructions are executed on an audio device, the audio device executes the audio processing method according to any one of claims 1 to 12.

Images