Hybrid-mode coding method and apparatus for multi-channel audio, and device and medium

Abstract

Provided in the present application are a hybrid-mode coding method and apparatus for multi-channel audio, and a device and a medium. The method comprises: according to corresponding audio types of multi-channel audio to be coded, determining coding units of said multi-channel audio; acquiring importance evaluation indexes of the coding units of said multi-channel audio; according to the importance evaluation indexes, determining coding modes of the corresponding coding units; and on the basis of different coding modes, respectively performing coding on the corresponding coding units in said multi-channel audio. By means of implementing the solution of the present application, corresponding coding units are determined according to audio types of audio to be coded, importance evaluation indexes of the coding units are acquired, corresponding coding modes are determined according to the importance evaluation indexes, and finally, on the basis of different coding modes, coding is performed on the corresponding coding units, such that the corresponding coding modes are used in different scenes, so as to meet the requirements for resources and computing power, and the fluency and continuity of the audio are ensured.

Description

A mixed mode encoding method, device, equipment and medium for multi-channel audio Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a mixed-mode encoding method, apparatus, device, and medium for multi-channel audio. Background Art

[0002] Traditional encoding has the characteristics of minimal sound quality loss and fast decoding speed at medium to high bit rates. However, medium to high bit rates result in a small compression ratio, requiring a larger space to store the encoded audio files, and a larger bandwidth to transmit audio in real-time streaming. In multi-channel situations, the audio of each channel needs to be stored or transmitted separately, resulting in resource constraints.

[0003] With the development of AI technology, encoding and decoding technology based on AI solutions has developed rapidly. It can now support the use of AI technology for encoding and decoding on edge devices, and achieve sound quality effects far exceeding traditional encoding at low bit rates. However, AI solutions require more computing power during decoding. Using AI solutions for all channels in multi-channel situations will lead to computing power constraints. Technical issues

[0004] The main purpose of this application is to provide a mixed-mode encoding method, device, equipment and medium for multi-channel audio, which can at least solve the problem of large resource and computing power consumption caused by the use of a single encoding method in related technologies. Technical Solutions

[0005] To achieve the above objectives, the present application provides, in a first aspect, a mixed-mode encoding method for multi-channel audio, comprising:

[0006] Determining a coding unit of the multi-channel audio to be encoded according to an audio type corresponding to the multi-channel audio to be encoded; wherein the audio type includes a channel-based type, a scene-based type, and an object-based type;

[0007] Obtaining an importance evaluation index of each coding unit of the multi-channel audio to be encoded;

[0008] Determining the encoding mode of the corresponding encoding unit according to each of the importance evaluation indicators;

[0009] The corresponding coding units in the multi-channel audio to be encoded are encoded respectively based on the different coding modes.

[0010] A second aspect of the present application provides a mixed-mode encoding device for multi-channel audio, comprising:

[0011] A first determining module is configured to determine a coding unit of the multi-channel audio to be encoded according to an audio type corresponding to the multi-channel audio to be encoded; wherein the audio type includes a channel-based type, a scene-based type, and an object-based type;

[0012] An acquisition module, configured to acquire an importance evaluation index of each coding unit of the multi-channel audio to be encoded;

[0013] A second determining module is used to determine the encoding mode of the corresponding encoding unit according to each of the importance evaluation indicators;

[0014] The encoding module is configured to encode the corresponding encoding units in the multi-channel audio to be encoded based on different encoding modes.

[0015] The third aspect of the present application provides an electronic device comprising: a memory and a processor, wherein the processor is used to execute a computer program stored in the memory. When the processor executes the computer program, it implements the steps of the mixed mode encoding method for multi-channel audio provided in the first aspect of the present application.

[0016] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the mixed mode encoding method for multi-channel audio provided in the first aspect of the present application are implemented. Beneficial effects

[0017] As can be seen from the above, according to the mixed mode encoding method, device, equipment and medium of multi-channel audio provided by the scheme of the present application, the encoding unit of the multi-channel audio to be encoded is determined according to the corresponding audio type of the multi-channel audio to be encoded; wherein the audio type includes channel-based, scene-based and object-based; the importance evaluation index of each encoding unit of the multi-channel audio to be encoded is obtained; the encoding mode of the corresponding encoding unit is determined according to each importance evaluation index; and the corresponding encoding units in the multi-channel audio to be encoded are encoded based on different encoding modes. Through the implementation of the scheme of the present application, the corresponding encoding unit is determined according to the audio type of the audio to be encoded, and the importance evaluation index of each encoding unit is obtained, and the corresponding encoding mode is determined according to the importance evaluation index. Finally, the corresponding encoding unit is encoded based on different encoding modes, so that the corresponding encoding method is used in different scenarios to meet the resource and computing power requirements and ensure the fluency and continuity of the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0019] FIG1 is a schematic diagram of the basic flow of a mixed-mode multi-channel audio encoding method provided by a first embodiment of the present application;

[0020] FIG2 is a schematic diagram of a sound channel of a third-order HOA signal provided in the first embodiment of the present application;

[0021] FIG3 is a schematic diagram of a detailed flow chart of a hybrid mode encoding method for multi-channel audio provided in a second embodiment of the present application;

[0022] FIG4 is a block diagram of a hybrid mode encoding device for multi-channel audio provided by a third embodiment of the present application;

[0023] FIG5 is a schematic structural diagram of an electronic device provided in a fourth embodiment of the present application. Modes for Carrying Out the Invention

[0024] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0025] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, the meaning of "plurality" is two or more, unless otherwise clearly specified.

[0026] In order to solve the problem of large resource and computing power consumption caused by adopting a single encoding mode in the related art, the first embodiment of the present application provides a mixed mode encoding method for multi-channel audio. FIG1 is a schematic diagram of the basic flow of the mixed mode encoding method for multi-channel audio provided by this embodiment. The mixed mode encoding method for multi-channel audio includes the following steps:

[0027] Step 101: Determine a coding unit of multi-channel audio to be encoded according to an audio type corresponding to the multi-channel audio to be encoded.

[0028] Specifically, in this embodiment, different multi-channel audio types correspond to different coding units, wherein immersive audio types include channel-based, scene-based, and object-based.

[0029] Step 102: Obtain an importance evaluation index of each coding unit of the multi-channel audio to be encoded.

[0030] Specifically, in this embodiment, in order to determine the coding modes of different channels in the multi-channel audio to be encoded, the importance evaluation index of each coding unit of the multi-channel audio to be encoded is obtained respectively.

[0031] In some implementations of this embodiment, the coding unit is a channel; the step of obtaining an importance evaluation index of each coding unit of the multi-channel audio to be encoded includes: obtaining a corresponding importance index table according to the audio type of the multi-channel audio to be encoded; wherein the audio type is channel-based or scene-based; and obtaining an importance evaluation index of each channel of the multi-channel audio to be encoded based on the importance index table.

[0032] Specifically, in this embodiment, when the audio type is channel-based and scene-based, the encoding unit of the multi-channel audio to be encoded is determined to be the channel, and the corresponding importance index table is obtained according to the corresponding audio type, thereby obtaining the importance evaluation index of each channel in the corresponding audio to be encoded.

[0033] Furthermore, in some implementations of this embodiment, the step of obtaining a corresponding importance index table according to the audio type of the multi-channel audio to be encoded includes: obtaining a multi-channel form of the multi-channel audio to be encoded based on the channel type; and obtaining a corresponding importance index table according to the multi-channel form.

[0034] Specifically, in this embodiment, when the audio type of the audio to be encoded is a channel-based type, the corresponding importance index table can be determined according to the multi-channel form of the audio to be encoded. In this embodiment, the channel importance indexes (ranging from 0 to 10) of different channels are ranked according to the multi-channel form. The larger the value, the higher the importance. The multi-channel form can be 5.1 channels, 5.1.2 channels, 7.1 channels, and 7.1.4 channels. Taking 5.1 channels as an example, the 5.1 channels are divided into the front left channel (Left), the front right channel (Right), the center channel (Center), the left surround channel (Left Surround), the right surround channel (Right Surround), and the subwoofer channel (Subwoofer). The audio information is mainly stored in the front left channel L, the front right channel R, and the center channel C. Therefore, the L, R, and C channels are the most important, and the other channels are less important. The corresponding relationship between each channel in the 5.1 channel and the channel importance (Channel Importance, CHI) is shown in Table 1:

[0035] Table 1

[0036] For 5.1.2 channels, two sky channels, Left Top Middle and Right Top Middle, are added on the basis of 5.1 channels. The corresponding relationship between the two sky channels and the channel importance (CHI) in 5.1.2 channels is shown in Table 2:

[0037] Table 2

[0038] Compared with 5.1 channels, 7.1 channels also include left rear surround channels and right rear surround channels. The corresponding relationship between each channel in 7.1 channels and channel importance (CHI) is shown in Table 3:

[0039] Table 3

[0040] For 7.1.4 channels, four sky channels are added on the basis of 7.1 channels, namely Left Top Front, Right Top Front, Left Top Rear, and Right Top Rear. The correspondence between the four sky channels in 7.1.4 channels and channel importance (CHI) is shown in Table 4:

[0041] Table 4

[0042] Furthermore, in some other implementations of this embodiment, the step of obtaining the importance evaluation index of each channel of the multi-channel audio to be encoded based on the importance index table includes: obtaining the HOA order corresponding to the scene-based multi-channel audio to be encoded; determining the number of channels of the multi-channel audio to be encoded based on the HOA order; and querying the importance evaluation index of each channel of the multi-channel audio to be encoded in the importance index table according to the number of channels.

[0043] Specifically, in this embodiment, when the audio type of the audio to be encoded is scene-based, the number of channels can be determined according to the HOA order of the audio to be encoded, and then the importance evaluation index of each channel can be further determined according to the number of channels. It should be noted that when reconstructing the sound field, it is necessary to encode higher-order reverberation signals (Higher Order Ambisonics, HOA). The higher the order of the Ambisonics signal, the higher the spatial resolution and spatial immersion can be achieved, but at the same time, more channels are required. The channel importance can be sorted. Since the audio information is mainly in the low-order signal, the channel importance corresponding to the low-order Ambisonics is set to be higher, and the channel importance corresponding to the high-order Ambisonics is set to be lower. The corresponding relationship between the HOA order and the number of channels can be expressed as (N+1)^2, where N is the HOA order, as shown in Table 5:

[0044] Table 5

[0045] Taking the third-order HOA signal as an example, its channel diagram is shown in Figure 2. The number of channels is 16. Since lower-order channels contain more information, the scene importance (SCI) corresponding to each channel from top to bottom in Figure 2 is 10, 8, 6, and 4 respectively.

[0046] In some other implementations of this embodiment, the coding unit is an object; the step of obtaining an importance evaluation index of each coding unit of the multi-channel audio to be encoded includes: obtaining an object-based importance evaluation index of each object of the multi-channel audio to be encoded based on a preset expert database.

[0047] Specifically, in this embodiment, when the audio type of the audio to be encoded is object-based, the encoding unit is determined to be an object, and the importance evaluation index of the object-based audio to be encoded can be set by the corresponding designer. It should be noted that when encoding and decoding object-based immersive audio, the unit is the object. For example, when there are N objects, their object importance (OBI) are object_1, object_2, ..., object_N respectively. Then, when encoding, each of the N objects needs to be encoded and decoded separately.

[0048] Step 103: Determine the coding mode of the corresponding coding unit according to each importance evaluation index.

[0049] Specifically, in this embodiment, after obtaining the importance evaluation index of the corresponding coding unit, such as the channel or object, the coding mode can be determined accordingly. The coding mode can be an AI coding scheme such as Lyra V2 (very low bit rate speech codec) or a traditional coding scheme such as the Opus encoder. The supported coding types are shown in Table 6:

[0050] Table 6

[0051] In some implementations of this embodiment, the steps of determining the coding mode of the corresponding coding unit according to each importance evaluation index include: calling the corresponding first coding mode index table according to the network bandwidth; querying the first coding mode index table according to each importance evaluation index to obtain the coding mode of the corresponding coding unit.

[0052] Specifically, in this embodiment, the corresponding coding mode index table can be determined according to the network bandwidth, and the coding mode index table includes the corresponding relationship between the importance evaluation index of each coding unit and the network bandwidth. The coding mode index table in this embodiment is generated by sorting the importance evaluation index (CHI) of all channels of the corresponding type (for example, all channels of the audio to be encoded based on the channel type) from low to high, where the CHI of the idxth channel is CHIIdx, the corresponding coding mode bit rate is BitRateIdx, the total bandwidth is BandWidthSum kbps, the minimum coding bit rate is BitRateMin (corresponding to 3kbps), and the maximum coding bit rate is BitRateMax (corresponding to 320kbps). Then, starting from the channel with the lowest CHI, the target bit rate is obtained and the corresponding coding mode is determined. The pseudo code of the calculation method is as follows:

[0053] BitRateIdx=BandWidthSum*CHIIdx / CHISum

[0054] BitRateIdx=min(BitRateIdx,BitRateMin)

[0055] BitRateIdx=floor(BitRateIdx)

[0056] Among them, BitRateIdx=floor(BitRateIdx) means obtaining the highest bit rate in the bit rate table that is not greater than BitRateIdx, and CHISum represents the overall importance evaluation index. Taking 5.1 channels as an example, the initial encoding method of each channel is shown in Table 7:

[0057] Table 7

[0058] When the total bandwidth is 1Mkbps, the encoding method of each channel is shown in Table 8:

[0059] Table 8

[0060] Furthermore, when the network bandwidth decreases during the encoding and decoding process, the encoding method of the channel with the lowest CHI can be preferentially reduced.

[0061] In other implementations of this embodiment, the steps of determining the encoding mode of the corresponding encoding unit according to each importance evaluation index include: obtaining the reserved storage space size corresponding to the encoded audio; determining the average storage space size per unit time according to the reserved storage space size; calling the corresponding second encoding mode index table according to the average storage space size per unit time; querying the second encoding mode index table according to each importance evaluation index to obtain the encoding mode of the corresponding encoding unit.

[0062] Specifically, in the case of non-network transmission, space needs to be reserved for storing the encoded audio signal. In the case of multiple channels, if all channels are encoded using a high bit rate, it will result in a huge space occupation. Therefore, different encoding and decoding methods can be selected according to the importance of each channel (or object). The lower the importance, the less information it contains, and a lower bit rate encoding method can be used. Therefore, this embodiment also provides a coding mode index table that includes a correspondence between the average storage space size per unit time and the importance evaluation index. The generation method of this coding mode index table can refer to the coding mode index table corresponding to the network bandwidth. For example, the average storage space per second of audio is 100kbits, which is analogous to a network bandwidth of 100kbps.

[0063] Step 104: Encode corresponding coding units in the multi-channel audio to be encoded based on different coding modes.

[0064] Specifically, by obtaining the encoding mode of the corresponding encoding unit respectively, the multi-channel audio can be encoded and decoded in a mixed mode; the mixed mode encoding and decoding of the multi-channel audio can play its practical value in audio storage and real-time streaming media. For example, the current AI codec will bring about a large computing power consumption during the decoding process, and in multi-channel audio encoding and decoding, especially on storage media, traditional encoding is still the main method, supplemented by AI encoding. Therefore, using AI encoding for some unimportant information can reduce the resource consumption required for storage; in real-time streaming media, such as live broadcasts and video conferencing, the real-time performance of encoding, CPU occupancy, bandwidth, etc. all directly affect the final experience, especially the advantage of low bit rate in weak network conditions is irreplaceable. Through mixed mode encoding and decoding, the audio bit rate can be autonomously reduced when the network is limited, thereby ensuring the smoothness and continuity of the user's listening experience.

[0065] Based on the technical solution of the embodiment of the present application, the coding unit of the multi-channel audio to be encoded is determined according to the corresponding audio type of the multi-channel audio to be encoded; wherein the audio type includes channel-based, scene-based and object-based types; the importance evaluation index of each coding unit of the multi-channel audio to be encoded is obtained; the coding mode of the corresponding coding unit is determined according to each importance evaluation index; and the corresponding coding units in the multi-channel audio to be encoded are encoded based on different coding modes. Through the implementation of the solution of the present application, the corresponding coding unit is determined according to the audio type of the audio to be encoded, and the importance evaluation index of each coding unit is obtained, and the corresponding coding mode is determined according to the importance evaluation index. Finally, the corresponding coding unit is encoded based on different coding modes, so that the corresponding coding method is used in different scenarios to meet the resource and computing power requirements and ensure the fluency and continuity of the audio.

[0066] The method in FIG3 is a refined mixed-mode encoding method for multi-channel audio provided in the second embodiment of the present application. The mixed-mode encoding method for multi-channel audio includes:

[0067] Step 301: Determine a coding unit of the multi-channel audio to be encoded according to an audio type corresponding to the multi-channel audio to be encoded;

[0068] Step 302: When the coding unit is a channel, obtain a corresponding importance index table according to the audio type of the multi-channel audio to be encoded;

[0069] Step 303: Obtain an importance evaluation index for each channel of the multi-channel audio to be encoded based on the importance index table;

[0070] Step 304: When the coding unit is an object, obtain an object-based importance evaluation index for each object of the multi-channel audio to be coded based on a preset expert database;

[0071] Step 305: Determine the coding mode of the corresponding coding unit according to each importance evaluation index;

[0072] Step 306: Encode corresponding coding units in the multi-channel audio to be encoded based on different coding modes.

[0073] Specifically, in this embodiment, it should be understood that the size of the serial number of each step in this embodiment does not mean the order of execution of the steps. The execution order of each step should be determined by its function and internal logic, and should not constitute a sole limitation on the implementation process of the embodiment of this application.

[0074] Based on the above technical solutions of the embodiments of the present application, when the coding unit is a channel, a corresponding importance index table is obtained according to the audio type of the multi-channel audio to be encoded (channel-based, scene-based); an importance evaluation index of each channel of the multi-channel audio to be encoded is obtained based on the importance index table; when the coding unit is an object, an importance evaluation index of each object of the multi-channel audio to be encoded based on the object type is obtained according to a preset expert database; the coding mode of the corresponding coding unit is determined according to each importance evaluation index; and the corresponding coding units in the multi-channel audio to be encoded are encoded based on different coding modes. Through the implementation of the solution of the present application, the corresponding coding unit is determined according to the audio type of the audio to be encoded, and the importance evaluation index of each coding unit is obtained, and the corresponding coding mode is determined according to the importance evaluation index. Finally, the corresponding coding unit is encoded based on different coding modes, so that the corresponding coding method is used in different scenarios to meet the resource and computing power requirements and ensure the fluency and continuity of the audio.

[0075] FIG4 is a diagram of a mixed-mode encoding device for multi-channel audio provided in a third embodiment of the present application. The mixed-mode encoding device for multi-channel audio can be applied to the mixed-mode encoding method for multi-channel audio described above. As shown in FIG4 , the mixed-mode encoding device for multi-channel audio mainly includes:

[0076] A first determining module 401 is configured to determine a coding unit of the multi-channel audio to be encoded according to an audio type corresponding to the multi-channel audio to be encoded; wherein the audio type includes a channel-based type, a scene-based type, and an object-based type;

[0077] An acquisition module 402 is configured to acquire an importance evaluation index of each coding unit of the multi-channel audio to be encoded;

[0078] The second determining module 403 is used to determine the coding mode of the corresponding coding unit according to each importance evaluation index;

[0079] The encoding module 404 is configured to encode corresponding encoding units in the multi-channel audio to be encoded based on different encoding modes.

[0080] In some implementations of this embodiment, the encoding unit is a channel, and the acquisition module is specifically used to: obtain a corresponding importance index table according to the audio type of the multi-channel audio to be encoded; wherein the audio type is channel-based or scene-based; and obtain an importance evaluation index for each channel of the multi-channel audio to be encoded based on the importance index table.

[0081] Furthermore, in some implementations of this embodiment, the acquisition module is further configured to: acquire a multi-channel form of the multi-channel audio to be encoded based on the channel type; and acquire a corresponding importance index table according to the multi-channel form.

[0082] Furthermore, in some other implementations of this embodiment, the mixed mode encoding device for multi-channel audio also includes: a query module for obtaining the HOA order corresponding to the scene-based multi-channel audio to be encoded; determining the number of channels of the multi-channel audio to be encoded based on the HOA order; and querying the importance evaluation index of each channel of the multi-channel audio to be encoded in the importance index table according to the number of channels.

[0083] In some implementations of this embodiment, the encoding unit is an object; the acquisition module is specifically configured to: obtain, based on a preset expert database, an importance evaluation index of each object of the to-be-encoded multi-channel audio based on the object type.

[0084] In some implementations of this embodiment, the second determination module is specifically used to: call the corresponding first coding mode index table according to the network bandwidth; query the first coding mode index table according to each importance evaluation index to obtain the coding mode of the corresponding coding unit.

[0085] In some other implementations of this embodiment, the second determination module is also used to: obtain the reserved storage space size corresponding to the encoded audio; determine the average storage space size per unit time based on the reserved storage space size; call the corresponding second coding mode index table based on the average storage space size per unit time; query the second coding mode index table according to each importance evaluation index to obtain the coding mode of the corresponding coding unit.

[0086] It should be noted that the mixed mode encoding methods for multi-channel audio in the aforementioned embodiments can all be implemented based on the mixed mode encoding device for multi-channel audio provided in this embodiment. Ordinary technicians in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the mixed mode encoding device for multi-channel audio described in this embodiment can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0087] Based on the technical solutions of the above embodiments of the present application, the coding units of the multi-channel audio to be encoded are determined according to the corresponding audio types of the multi-channel audio to be encoded; wherein the audio types include channel-based, scene-based and object-based; the importance evaluation index of each coding unit of the multi-channel audio to be encoded is obtained; the coding mode of the corresponding coding unit is determined according to each importance evaluation index; and the corresponding coding units in the multi-channel audio to be encoded are encoded respectively based on different coding modes. Through the implementation of the solution of the present application, the corresponding coding units are determined according to the audio type of the audio to be encoded, and the importance evaluation index of each coding unit is obtained, and the corresponding coding mode is determined according to the importance evaluation index. Finally, the corresponding coding units are encoded based on different coding modes, so that the corresponding coding methods are used in different scenarios to meet the resource and computing power requirements and ensure the fluency and continuity of the audio.

[0088] FIG5 is a schematic diagram of the structure of an electronic device provided in a fourth embodiment of the present application. The electronic device can be used to implement the hybrid mode encoding method for multi-channel audio in the aforementioned embodiments, mainly comprising:

[0089] Memory 501, processor 502, and computer program 503 stored in memory 501 and executable on processor 502. Memory 501 and processor 502 are connected via communication. When processor 502 executes computer program 503, the method of the aforementioned embodiment 1 or 2 is implemented. The number of processors may be one or more.

[0090] The memory 501 can be a high-speed random access memory (RAM) memory or a non-volatile memory such as a disk memory. The memory 501 is used to store executable program codes. The processor 502 is coupled to the memory 501 .

[0091] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which may be provided in the above-mentioned electronic device. The computer-readable storage medium may be the memory in the embodiment shown in FIG. 5 .

[0092] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the mixed-mode encoding method for multi-channel audio in the aforementioned embodiments. Furthermore, the computer-readable storage medium may be any medium capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0094] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of these modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0095] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0096] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a readable storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned readable storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0097] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0098] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0099] The above is a description of the mixed-mode encoding method, apparatus, device, and medium for multi-channel audio provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A mixed mode encoding method for multi-channel audio, characterized in that: include: Determining a coding unit of the multi-channel audio to be encoded according to an audio type corresponding to the multi-channel audio to be encoded; wherein the audio type includes a channel-based type, a scene-based type, and an object-based type; Obtaining an importance evaluation index of each coding unit of the multi-channel audio to be encoded; Determining the encoding mode of the corresponding encoding unit according to each of the importance evaluation indicators; The corresponding coding units in the multi-channel audio to be encoded are encoded respectively based on the different coding modes.

2. The mixed mode encoding method according to claim 1, characterized in that: The coding unit is a channel; the step of obtaining the importance evaluation index of each coding unit of the multi-channel audio to be encoded includes: Acquire a corresponding importance index table according to the audio type of the multi-channel audio to be encoded; wherein the audio type is channel-based or scene-based; Obtain an importance evaluation index of each channel of the multi-channel audio to be encoded based on the importance index table.

3. The mixed mode encoding method according to claim 2, characterized in that: The step of obtaining a corresponding importance index table according to the audio type of the multi-channel audio to be encoded comprises: Acquire a multi-channel form of the multi-channel audio to be encoded based on a channel type; A corresponding importance index table is obtained according to the multi-channel form.

4. The mixed mode encoding method according to claim 2, characterized in that: The step of obtaining the importance evaluation index of each channel of the multi-channel audio to be encoded based on the importance index table includes: Obtaining a HOA order corresponding to the multi-channel audio to be encoded based on the scene type; Determine the number of channels of the multi-channel audio to be encoded based on the HOA order; According to the number of channels, the importance evaluation index of each channel of the multi-channel audio to be encoded is queried in the importance index table.

5. The mixed mode encoding method according to claim 1, characterized in that: The coding unit is an object; the step of obtaining the importance evaluation index of each coding unit of the multi-channel audio to be encoded includes: According to a preset expert database, an importance evaluation index of each object of the to-be-encoded multi-channel audio is obtained based on the object type.

6. The mixed mode encoding method according to any one of claims 1 to 5, characterized in that: The step of determining the encoding mode of the corresponding encoding unit according to each of the importance evaluation indicators comprises: Calling a corresponding first coding mode index table according to the network bandwidth; The first coding mode index table is queried according to each of the importance evaluation indicators to obtain the coding mode of the corresponding coding unit.

7. The mixed mode encoding method according to any one of claims 1 to 5, characterized in that: The step of determining the encoding mode of the corresponding encoding unit according to each of the importance evaluation indicators comprises: Get the size of the reserved storage space corresponding to the encoded audio; Determining an average storage space size per unit time according to the reserved storage space size; Calling the corresponding second coding mode index table according to the average storage space size per unit time; The second coding mode index table is queried according to each of the importance evaluation indicators to obtain the coding mode of the corresponding coding unit.

8. A mixed mode encoding device for multi-channel audio, characterized in that: include: A first determination module is used to determine the coding unit of the multi-channel audio to be encoded according to the audio type corresponding to the multi-channel audio to be encoded; wherein the audio type includes a channel-based type, a scene-based type and an object-based type; An acquisition module, used to acquire an importance evaluation index of each encoding unit of the multi-channel audio to be encoded; A second determination module is used to determine the encoding mode of the corresponding encoding unit according to each importance evaluation index; The encoding module is used to encode the corresponding encoding units in the multi-channel audio to be encoded based on different encoding modes.

9. An electronic device, characterized in that: The device comprises a memory and a processor, wherein: The processor is used to execute the computer program stored in the memory; When the processor executes the computer program, the steps in the mixed mode encoding method for multi-channel audio described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the mixed mode encoding method for multi-channel audio described in any one of claims 1 to 7 are implemented.

Images