An audio coding method and apparatus based on perceptual importance-driven and Polar codes
By using a Polar code encoding method driven by perceptual importance, the priority of audio data blocks is determined based on the hearing impairment cost model and differentiated protection is implemented. This solves the problem of the separation between audio channel coding and perceptual coding in the existing technology and improves the sound quality of wireless audio transmission.
Patent Information
- Application Number
- CN202611236613.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-17
AI Technical Summary
Existing audio channel coding methods fail to effectively incorporate auditory perception characteristics to differentiate audio data, resulting in insufficient sound quality in wireless transmission.
The Polar code encoding method based on perceived importance determines the priority level of data blocks through a perceived damage cost model and maps them to protection areas of different reliability, generating payload sequences and control heads for encoding processing.
Without significantly increasing bitrate and computational complexity, it improves audio clarity and listening stability, making it suitable for wireless headphones, smart speakers, and streaming audio transmission.
Smart Images

Figure CN123119918A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of wireless communication and audio technology, specifically relating to an audio coding method and apparatus based on perceptual importance-driven and Polar codes. Background Technology
[0002] With the rapid development of consumer audio devices (such as wireless headphones, smart speakers, and home theater systems), users' requirements for audio quality have gradually shifted from the traditional "no obvious distortion" to "a high-fidelity, clear, and natural listening experience." Existing audio systems typically employ Perceptual Audio Coding technology, which compresses audio signals by utilizing the masking effect of human hearing, thereby preserving the most important auditory information within limited bandwidth. However, during wireless or network transmission, channel coding technology is still necessary to improve data reliability.
[0003] Existing channel coding methods (including Polar codes) typically aim to reduce the bit error rate or block error rate, assuming that each bit has equal importance. However, in audio signals, different frequency bands, time slices, and sound source components have significantly different effects on human hearing. For example, the dominant frequency band of speech and transient signals have a greater impact on clarity, while background noise or weak texture signals have a smaller impact on hearing. Therefore, even if the overall bit error rate is low, errors in key perceptual information can still lead to a significant degradation in hearing.
[0004] Existing technologies lack a mechanism that can combine auditory perception characteristics to differentiate audio data at the channel coding level, resulting in a disconnect between channel coding and perception coding, making it difficult to further improve the subjective sound quality performance of consumer-grade audio systems. Summary of the Invention
[0005] To address the problems raised in the background art, a first aspect of the present invention provides an audio coding method based on perceptual importance-driven and Polar codes, comprising: Obtain the raw audio data to be transmitted, and divide the raw audio data into multiple data blocks with logical dependencies; The priority level of each data block is determined based on a pre-constructed perceptual impairment cost model; the perceptual impairment cost model is established based on the degree of impairment to audio reconstruction quality caused by data block errors. Obtain the target coding rate corresponding to the current communication link, and provide a set of polar code information bits that match the target coding rate. The set of polar code information bits includes multiple protection regions distributed according to channel reliability. According to the priority level, each data block is mapped to its corresponding matching protection region to obtain a payload sequence; wherein, under different target coding rates, the mapping position of the protection region corresponding to the highest priority data block remains constant; A control header is generated to indicate the decoding rules. An enhanced error correction coding independent of the payload sequence is applied to the control header. The control header protected by the enhanced error correction coding and the payload sequence are used together as inputs for polar code encoding.
[0006] In some embodiments of the present invention, determining the priority level of each data block according to a pre-constructed perceptual impairment cost model includes: obtaining the basic impairment weight of each data block by querying a preset impairment priority mapping table, wherein the basic impairment weight is obtained by injecting a preset error mode into the audio test sample in an offline state and evaluating the hearing impairment index; and dynamically correcting the basic impairment weight according to the auditory scene mode and masking effect parameters perceived by the current system to determine the priority level of the current frame.
[0007] In some embodiments of the present invention, during the processing of multiple consecutive audio data frames, if it is determined that the priority level of a certain data block has changed across levels, the switching of the protection region mapped to the data block is only performed when the duration of the cross-level change or the number of consecutive frames reaches a preset hysteresis confirmation threshold.
[0008] In some embodiments of the present invention, mapping each data block to its corresponding matching protection area includes: maintaining a set containing multiple mapping pattern tables based on a dual-loop control strategy, wherein the set includes at least one fixed parent table; the dual-loop control strategy includes: based on a high-frequency execution cycle, strictly executing the mapping configuration of data blocks to protection areas according to the currently selected mapping pattern table in the current frame; based on a low-frequency decision cycle, calculating the benefit score of each mapping pattern table in the current low-frequency decision cycle according to the link quality indicators and perceived scene characteristics fed back by the receiver, and deciding whether to switch to a new mapping pattern table accordingly.
[0009] In some embodiments of the present invention, the plurality of protection regions in the polar code information bit set include at least a first protection region corresponding to high reliability, a second protection region corresponding to medium reliability, and a third protection region corresponding to low reliability; when the link quality index is determined to be lower than a preset security threshold and a code rate reduction operation is triggered by the low-frequency decision cycle, the third protection region is stripped from the updated polar code information bit set, and the arrangement structure of the corresponding data in the first protection region is maintained unchanged.
[0010] In some embodiments of the present invention, after applying enhanced error correction coding independent of the load sequence to the control head, the method further includes: assigning cyclic redundancy check sequences with different check lengths to the data blocks mapped to different protection regions, wherein the data blocks mapped to higher reliability protection regions are assigned cyclic redundancy check sequences with a length not less than that of lower reliability protection regions.
[0011] In a second aspect, the present invention provides an audio encoding apparatus based on perceptual importance-driven and Polar codes, comprising: an acquisition module for acquiring raw audio data to be transmitted and dividing the raw audio data into multiple logically dependent data blocks; a determination module for determining the priority level of each data block according to a pre-constructed perceptual impairment cost model; the perceptual impairment cost model being established based on the degree of impairment to audio reconstruction quality caused by data block errors; a matching module for acquiring the target coding rate corresponding to the current communication link and providing a set of polar code information bits matching the target coding rate, the set of polar code information bits including multiple protection regions distributed according to channel reliability; a mapping module for mapping each data block to its corresponding matching protection region according to the priority level, thereby obtaining a payload sequence; wherein, under different target coding rates, the mapping position of the protection region corresponding to the highest priority data block remains constant; and a generation module for generating a control head for indicating decoding rules, applying enhanced error correction coding independent of the payload sequence to the control head, and using the enhanced error correction coding-protected control head and the payload sequence together as input for polar code encoding.
[0012] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the audio coding method based on perceptual importance-driven and Polar codes provided in the first aspect of the present invention.
[0013] In a fourth aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the audio coding method based on perceptual importance-driven and Polar codes provided in the first aspect of the present invention.
[0014] The beneficial effects of this invention are: This invention extracts perceptual importance weights for different frequency bands, time slices, or acoustic components based on an auditory model; according to the perceptual importance, the audio data is divided into data subsets of different priorities; during Polar coding, high-priority data is mapped to sub-channels with higher reliability, and low-priority data is mapped to sub-channels with lower reliability, thereby achieving non-uniform error protection; at the receiving end, the decoding results are combined for reconstruction to obtain a better auditory perception effect.
[0015] This method can effectively improve audio clarity and listening stability without significantly increasing bit rate and computational complexity, and is particularly suitable for scenarios such as wireless headphones, smart speakers and streaming audio transmission. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the basic process of an audio coding method based on perceptual importance-driven and Polar codes in some embodiments of the present invention; Figure 2 This is one of the schematic diagrams illustrating the specific process of an audio coding method based on perceptual importance-driven and Polar codes in some embodiments of the present invention; Figure 3 This is the second schematic diagram of the specific process of the audio coding method based on perceptual importance-driven and Polar codes in some embodiments of the present invention; Figure 4 This is a schematic diagram illustrating the encoding principle based on perceptual importance-driven and Polar codes in some embodiments of the present invention; Figure 5 This is a schematic diagram illustrating the decoding principle based on perceptual importance-driven and Polar codes in some embodiments of the present invention; Figure 6 This is a schematic diagram of the frame encoding structure of Polar code in some embodiments of the present invention; Figure 7 This is a schematic diagram of the structure of an audio encoding device based on perceptual importance driving and Polar codes in some embodiments of the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. Detailed Implementation
[0017] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.
[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0019] In this document, the term "comprising" indicates the presence of a described feature, integral, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," with exclusions being otherwise specifically emphasized. Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying one or more of the feature. In the description of embodiments of this application, unless otherwise stated, "a plurality of" means two or more.
[0020] Example 1 refer to Figure 1 to Figure 3 In a first aspect of the present invention, an audio coding method based on perceptual importance-driven and Polar codes is provided, comprising: S100. Obtain the raw audio data to be transmitted, and divide the raw audio data into multiple data blocks with logical dependencies; S200. Determine the priority level of each data block according to the pre-constructed perceptual impairment cost model; the perceptual impairment cost model is established based on the degree of impairment of audio reconstruction quality caused by data block errors; S300. Obtain the target coding rate corresponding to the current communication link, and provide a set of polar code information bits that matches the target coding rate, wherein the set of polar code information bits includes multiple protection regions distributed according to channel reliability. S400. According to the priority level, each data block is mapped to its corresponding matching protection region to obtain a payload sequence; wherein, under different target coding rates, the mapping position of the protection region corresponding to the highest priority data block remains constant; S500. Generate a control head for indicating decoding rules, apply enhanced error correction coding independent of the payload sequence to the control head, and use the control head protected by enhanced error correction coding and the payload sequence together as input for polar code encoding.
[0021] In step S100 of some embodiments of the present invention, the original audio data to be transmitted is obtained, and the original audio data is divided into multiple data blocks with logical dependencies. Specifically, audio input is defined as: x [ n ] ,n= 0,1,…, Lf- 1, in, x [ n [] represents the discrete sample value of the current audio frame; n It is the intra-frame sampling index; Lf This is the number of sampling points per frame. If the sampling rate is... Fs The frame length is: , in, Tf Frame duration Fs It's the sampling rate. For 48kHz audio, if... Lf =240, then Tf=5 ms; if Lf= 480, then Tf= 10ms. Low-latency audio equipment should generally prioritize frame lengths of 2.5ms, 5ms, or 10ms. Generally, low-latency systems prefer to use IIR / FIR filter banks or short-window STFTs rather than long-window high-resolution STFTs, because long windows sacrifice transient response, making users more likely to perceive sucking, delay, transient blurring, and tone drift.
[0022] Subband analysis can be represented by a filter bank as follows: , in, xb [ n [Is the] number b Each one carries a signal; hb [ n [Is the] number b The impulse response of a sub-band filter; * indicates convolution; B It refers to the number of sub-tapes. (In audio equipment) B It should not be too large; usually, 4 to 8 subbands are more appropriate. The more subbands there are, the finer the control granularity becomes, but the complexity of computation, caching, parameter synchronization, and mapping will all increase.
[0023] Short-window STFT can also be used: , in, X ( t,k ) is the first t Frame number k Complex spectrum at each frequency point; L H is the length of the FFT (Fast Fourier Transform); H is the hop size. w [ n ] is a window function;k This is a frequency index. The sub-band energy is: , in, Eb ( t ) is the first t Frame number b Each element carries energy; Ωb is the first... b Each subband contains a set of frequency points. After completing the subband analysis, the source-coded output needs to be broken down into data atoms: DJ = d ( t,b,q,s,l ), in, DJ It is the first j One data unit to be protected; t It is a time frame index; b It is a frequency band index; q It is a bit-plane index, such as MSB, second-highest bit, LSB; s It refers to the structure type, such as frame header, scaling factor, codebook index, and residual; l This refers to the location within the packet or the data block number. The value of this definition lies in the fact that subsequent... DJ Importance can be calculated, and continuous audio, full-frequency signals, and indiscriminate bit streams can be decomposed into multiple local objects, each of which receives different protections based on its auditory and structural functions.
[0024] This can be understood as transforming a continuous audio stream into protectable, sortable, and mappable data units. The audio source encoder or packer outputs "atomic data units," such as frame headers, synchronization words, mode fields, scaling factors, codebook indexes, subband energy, spectral envelopes, MSB, LSB, and residuals. The auditory impairment caused by bit errors is determined by the role of that bit in the audio reconstruction process. Frame header errors can cause the entire frame to fail, scaling factor errors can cause severe energy shifts in a certain frequency band, MSB errors can cause significant quantization inversions, while LSB errors in strongly masked frequency bands may be negligible.
[0025] In step S200 of some embodiments of the present invention, determining the priority level of each data block according to a pre-built perceived impairment cost model includes: S201. By querying a preset damage priority mapping table, the basic damage weight of each data block is obtained. The basic damage weight is obtained by injecting a preset error mode into the audio test sample and evaluating the hearing damage index in an offline state. The perceptual impairment of a single frame or a short time window is defined as: , in, Jperc ( t ) is the first t Integrated perceptual impairment of frames; J E ( t ) is the subband energy distortion term; J L ( t ) is the loudness mutation term; J T ( t ) is the transient damage term; J S ( t ) is an item indicating impairment in speech intelligibility or subject clarity; J p ( t ) are penalties for serious incidents such as popping sounds, frame errors, and spikes; l E , l L , l T , l S , l P These are the weighting coefficients for each corresponding damage term. For communication equipment, l S and l P It should be relatively high; for music speakers, l E , l L , l T The requirement should be relatively high; for vehicle-mounted or security scenarios, l P The weight of safety sound retention should be the highest.
[0026] Subband energy distortion can be written as: , in, B It is the number of sub-bands; b It is a sub-band index; E b ( t ) is the first error-free reference reconstruction or the sender's estimate. b Size carries energy; It is the first time after the receiving end recovers. b Size carries energy; ∊ It is a small constant that prevents division by zero; W audio ( t, b ) is the first t Frame number bThe auditory importance weighting of each frequency band can be derived from fundamental frequency band importance, speech intelligibility weighting, equal loudness sensitivity, scene correction, and masking correction. Different frequency bands contribute differently to the auditory experience; 2–5kHz is particularly critical for clarity, harshness, and consonants, while low frequencies affect thickness and boominess, and high frequencies affect airiness and sibilance; therefore, distortion cannot be treated as an average across the entire frequency range.
[0027] The loudness mutation term can be defined as: , in, L out ( t ) is the first t Frame output loudness estimation; Δ L maxt This is the maximum permissible inter-frame loudness variation. Variations exceeding this threshold are considered potentially causing a sucking, abrupt, or popping sound.
[0028] The transient damage term can be written as: , in, O out ( t ) is the output transient intensity. O ref ( t This is the reference transient intensity. If an error causes the drumbeats, consonants, or alarm onset to be weakened or abnormally amplified, this value will increase.
[0029] The final objective function to be optimized is expressed as: , in, It is average sensory impairment; CVaR α ( Jperc ) is the distribution of damage in α Conditional risk values above the quantile level are used to constrain rare but severe pops, frame errors, and spikes. tail It is the tail risk weight; Dtotal It is an end-to-end delay; Cops It's about computational complexity; Jswitch It is a penalty for switching bitrate, mode, or mapping table; ŋD , ŋc , ŋQ These are the delay, calculation, and switching penalty weights, respectively.
[0030] The constraints are: , in, Dmax This is the maximum allowable delay for the product; Cmax It is the chip's computing budget; PH It represents the failure probability of the high-priority layer. This is the maximum number of failures allowed in the high-priority layer; Pop It is the probability of serious auditory incidents such as popping sounds, spikes, and frame errors; This is the maximum allowable limit for the product.
[0031] S202. Based on the auditory scene pattern and masking effect parameters perceived by the current system, the basic damage weight is dynamically adjusted to determine the priority level of the current frame.
[0032] Specifically, the audio source is encoded or packaged to obtain data units: , in, d j It is the first j One data unit; t It is a time frame index; s It refers to the structure type, such as frame header, synchronization field, bitrate mode, scaling factor, codebook index, residual, and enhancement layer; b It is a frequency band index; q It is a bit-plane index, such as MSB, second-highest bit, LSB; g It is a data group index used to describe the smallest recoverable unit composed of multiple bits or parameters.
[0033] Define offline counterfactual impairments for each type of data unit: , in, The structure type is s Frequency band is b The position plane is q Scene mode is m Average perceived impairment following data errors; It is an offline training or validation audio set; It is an error-free audio reconstruction; It only reconstructs the audio after an error is injected into this type of data; It is the perceived damage function, which can be derived from the previous step. Jperc or its sub-items constitute; m These are scene modes, such as voice, music, meeting, alarm, game, call, etc.
[0034] Offline error injection should not only flip a single bit, but should cover all possible error types that may occur in the actual circuit. It can be defined as follows: , in, It is a set of error modes, including single bit flip, burst bit error, whole group loss, error codebook index, scaling factor offset, packet header misinterpretation, etc. In channel quality c Error mode e The probability of; It is an injection error mode e The reconstructed audio afterwards.
[0035] Online runtime no longer involves complex calculations, but rather table lookups: , in, oh j ( t ) is the first j The importance of each data unit online; s j , b j , q j This refers to the structure, frequency band, and bit plane label of the data unit; m ( t This is the scenario mode currently being determined by the slow control loop; G mask ( t,b j This is a masking correction; its importance may be reduced if the frequency band is strongly masked. G link ( t This is a link risk correction mechanism; when the link deteriorates, the weight gap between the core layer and the enhancement layer can be widened. G age ( t,j This is a time-related correction used to prevent certain low-priority data from being left unprotected for a long time, leading to a continuous deterioration in sound quality.
[0036] To avoid excessive fluctuations in dynamic weights, amplitude limiting and smoothing are necessary. , in, oh min It is the lowest protection weight to prevent low-priority enhancement layers from being completely unprotected for a long period of time. oh max It has the highest weight to prevent a single type of data from monopolizing the most reliable sub-channel for a long period of time. The importance of smoothing; α ωIt is a smoothing coefficient; the closer it is to 1, the slower the change. Dynamic weights control risk through amplitude limiting and smoothing. This step embodies "measurement," "feedback," and "local quality." Counterfactual experiments are used to measure the actual damage of each type of error, ensuring that data with high damage receives highly reliable resources. Offline damage tables are built, and online table lookups are performed, enabling real-time realization of "cheap alternatives."
[0037] refer to Figure 4 In step S200 of some embodiments of the present invention, if it is determined that the priority level of a certain data block has changed across levels during the processing of multiple consecutive audio data frames, the switching of the protection area mapped to the data block is only performed when the duration of the cross-level change or the number of consecutive frames reaches a preset hysteresis confirmation threshold.
[0038] Specifically, define the data group: , in, G g It is the first g One data set; d g,i It is the first in the group i One data unit; n g, This refers to the number of data units contained in the group. A group can be a frame header group, a mode control group, a scaling factor group for a specific frequency band, a MSB group for a specific frequency band, a residual group for a specific frequency band, or an enhancement layer group. The importance of a group is defined as follows: , in, It is the first g Group 1 in t The importance of frames; ag,i These are the weighting coefficients of the data units within the group; It is the first in the group i The importance of smoothing each data unit; Rg It is a structural dependency risk, such as whether the failure of this group results in the unavailability of the entire frame, the entire frequency band, or multiple subsequent parameters; ŋ dep depends on risk weights; Cg There is a risk of catastrophic errors, such as whether an error in this group could cause pops, spikes, or incorrect codebook selections; ŋ cat represents the catastrophic risk weight.
[0039] according to Divide the data into layers: , in, These are sets of high, medium, and low priority groups, respectively. It is a high-priority threshold; This is a medium priority threshold. For more granular control, a core layer can be added. and enhancement layer However, it is not recommended to have too many layers in practice, because the more layers there are, the more complex the CRC, packet header, mapping table, and error hiding state become.
[0040] To ensure inter-frame stability, the layering results also need to be hysteretic. The previous frame layer is defined as... LG ( t-1 The current candidate level is Only when importance crosses the threshold and remains so... ML Switching only at frame intervals: , in, LG ( t ) is the first g Group final level; Hold ( g () is the number of frames that the candidate level is maintained continuously; ML This refers to the number of frames confirmed for level switching. This prevents frequent jumps between high and low guard layers in a certain frequency band due to short-term energy changes. The audio stream is segmented into analyzable objects, and strongly dependent bits are merged into engineering-controllable data groups. It avoids the side effects of overly fine-grained optimization, and the final output is a stable grouping list and level labels, thus ensuring audio stability.
[0041] It's understandable that traditional UEP (Unequal Error Protection) would place high-priority data in more reliable sub-channels. However, if each bit rate and each mode uses a completely different set of information, receiver synchronization, hardware implementation, and fallback due to anomalies would become more complex. The optimized scheme employs a nested reliability set: the core layer at low bit rates uses the most reliable subset; as the bit rate increases, only low-to-medium priority enhancement layers are added, without changing the core layer's position. This way, when the bit rate decreases in a weak channel, only the enhancement layers are discarded; when the bit rate increases in a strong channel, detail layers are added, but the core audio structure remains stable.
[0042] Therefore, refer to Figure 4 In step S300 of some embodiments of the present invention, the target coding rate corresponding to the current communication link is obtained, and a set of polar code information bits matching the target coding rate is provided. The set of polar code information bits includes multiple protection regions distributed according to channel reliability. Specifically, the Polar coding model is as follows: , in, x It is the encoded output codeword; u It is the Polar input vector; GNIt is a generating matrix; BN It is a bit-reversed permutation matrix; It is the fundamental polarization matrix; yes F of n Kronecker product; N It is the code length. Reliability is defined for each synthesized sub-channel: , in, ri It is the first i The reliability score of each Polar synthetic subchannel; pi It is the estimated error probability under a specified channel model, code length, code rate, and decoding algorithm; It is a small constant that prevents division by zero when taking the logarithm. Ordered in descending order of reliability: , in, a 1 is the most reliable index. aN It is the least reliable index.
[0043] Define nested information collections: , in, K 0< K 1< K 2< K 3; It is the lowest bitrate core layer information set; It is a basic audio collection; It is a set of normal quality; It is a high-quality augmented collection. Specifically defined as: , in, q= 0, 1, 2, 3. The frozen set is: , in, Bitrate level q The set of frozen bits below. Thus, when the system from... K 3 dropped to K At 1 o'clock, the most reliable K The position of one information bit remains completely unchanged; only the subsequent enhanced information bits are discarded, which will not affect the core layer mapping.
[0044] In each Further delineation of protected areas within: , in, It is a high-reliability protection zone, containing control headers, frame structures, and high-priority groups; It is a medium reliability zone, containing the main frequency band, general parameters, and medium priority residuals; This is a low-reliability information zone, so the enhancement layer is given lower priority. The capacity of each zone is: , in, Bitrate level q The capacity of the lower three types of protection zones should be increased. This should be increased for weak links. The ratio can be increased during strong links. To enhance sound quality details.
[0045] It should be noted that the nesting is reflected in the core set embedding all higher code rate sets; the pre-processing is reflected in the fact that the reliability sequence, information set, frozen set, and protected area boundary are all generated offline and solidified into a ROM table. This is more stable than reconstructing Polar codes at runtime and also avoids changes in the location of core data due to code rate switching.
[0046] The step of mapping each data block to its corresponding matching protection area includes: maintaining a set containing multiple mapping mode tables based on a dual-loop control strategy, wherein the set includes at least one fixed master table; the dual-loop control strategy includes: based on a high-frequency execution cycle, strictly executing the mapping configuration of data blocks to protection areas according to the currently selected mapping mode table in the current frame; based on a low-frequency decision cycle, calculating the benefit score of each mapping mode table in the current low-frequency decision cycle according to the link quality indicators and perceived scene characteristics fed back by the receiver, and deciding whether to switch to a new mapping mode table accordingly.
[0047] Specifically, define the parent table: , in, It is the default mapping table; Gg It is the first g One data set; It is the information set at the current bitrate. The parent table is calculated using the long-term average counterfactual impairment. Sorting generation: , , in, G It represents the number of data sets; It is the first g Long-term average importance of each group; ri This refers to the reliability of the Polar information bits. The mapping rule is: , in, It is the first gm The number of bits occupied by the group; sm This is the starting position of the group in the information bit sequence. The bits within the group maintain a fixed order and are no longer globally scattered bit by bit. This sacrifices very little theoretical optimality but significantly reduces implementation complexity and the difficulty of error localization.
[0048] Based on the parent table, only a small number of schema tables are allowed: , in, It is more inclined to protect 800–5000Hz, speech scaling factor, and consonant-related information; It prioritizes maintaining spectral balance and transient details, avoiding excessive sacrifice of high-frequency enhancement layers; Expand core layer protection in weak links; Ensure that alarm, danger, or safety alert frequencies are not incorrectly hidden. The material mentions that lightweight classifiers should only output a few control labels and should not aim for complete understanding; this corresponds to a small number of pattern tables rather than an infinite dynamic table.
[0049] Mode table switching only occurs at superframe boundaries: Tsf=MfTf , in, Tsf It is the duration of superframes; Mf It is the number of audio frames contained in a superframe; Tf This refers to the duration of a single frame. If... Tf =5ms Mf= 8, then Tsf= 40ms; if Mf= 16, then Tsf= 80ms. The switching rule is: And continue H One superframe, of which k It is a superframe index; m ( k ) is the current table number; m* ( k ) is the candidate optimal table; Sm ( k ) is the first m Zhang's score; Δ m It is a switching threshold; H It involves continuously confirming the number of frames exceeded. This rule ensures that mode switching is low-frequency, stable, and synchronized.
[0050] The score can be defined as: , in, U m ( c k ) is a table m For the current scenarioc k Adaptability; V m ( c k ) is a table m Regarding the current link quality c k Adaptability; E m ( Q k (This refers to the current link risk) Q k The following are the risks of using this table incorrectly; I [ m≠m(k-1) ] is the function to switch the penalty indicator; i c , i γ , i e , i s These are the weights of each item. This step is a balance between "dynamics" and "prevention of harmful effects": the system can be dynamic, but dynamic actions are limited to a small number of tables, superframe boundaries, and hysteresis rules.
[0051] Specifically, in scenarios that trigger bit rate reduction operations, the multiple protection regions in the polar code information bit set include at least a first protection region corresponding to high reliability, a second protection region corresponding to medium reliability, and a third protection region corresponding to low reliability. When the link quality index is determined to be lower than a preset security threshold and bit rate reduction operations are triggered through the low-frequency decision cycle, the third protection region is stripped from the updated polar code information bit set, while maintaining the arrangement structure of the corresponding data in the first protection region unchanged.
[0052] Audio equipment is not suitable for deploying complex, large-scale audio understanding models as real-time core controllers. The classifier should be lightweight, outputting only a few control labels, and its state should be smooth, with backoff required when confidence is insufficient. Therefore, the fast loop runs every frame, performing only deterministic operations; the slow loop runs every one or more superframes, performing only mode, bitrate, and table selection.
[0053] Therefore, for dual-loop control, the fast-loop procedure is as follows: frame → grouping → tablelookup → header / CRC → Polar encode → transmit → decode → CRC / LLR → PLC Complex optimizations, temporary generation of new mapping tables, and alterations to the semantics of data groups are not permitted in the fast loop. The time budget for the fast loop is: , in, D fast It is the fast loop processing latency per frame; D pack It's a delay in grouping and packaging; D enc It is a delay in Polar encoding; D dec It's a Polar decoding delay; D depack It's the delay in unpacking and demapping; D plc It's an error hiding a delay; It is the maximum allowable delay for the fast loop.
[0054] The slow loop input characteristics are: , in, k It is a superframe index; It is the first in superframe b Average energy per unit; It is the centroid of the average spectrum; It is the average spectral flatness; It is the average transient intensity; It is the average link quality; It is the smoothing error rate of high, medium and low layers; This is the retransmission rate. The lightweight classifier is: , in, p c ( k ) is a scene c The probability of; A It is a weight matrix; r It is a bias; z ( k () is the input feature. The confidence level is: , when r ( k )< r th During this time, slow loops must not be switched to dedicated tables; only the parent table or robust table can be maintained. , in, r th It is the confidence threshold; m * ( k ) is the candidate table; m base It is the base table.
[0055] Link risk is defined as: , in, Q ( k This is a risk associated with superframe links; β H , β M , β L , β R , β γ These are the weights of each item, generally β H > β M > β L Because failures at higher priority levels are more severe. The bitrate is updated to: , in, q ( k () represents the current bitrate level; Qhigh It is the bitrate reduction threshold; Qlow It is the bitrate upscaling threshold; MQ This confirms the restoration of the superframe rate. Because... Qlow < Qhigh The system exhibits hysteresis and does not frequently switch between up and down loops. This step involves "feedback," "dynamics," and "prevention of harmful effects": the system can adapt based on link feedback, but all adaptation occurs in the slow loop and is subject to gating and hysteresis.
[0056] refer to Figure 4 and Figure 6 In step S400 of some embodiments of the present invention, each data block is mapped to the corresponding matching protection region according to the priority level to obtain a payload sequence; wherein, under different target coding rates, the mapping position of the protection region corresponding to the highest priority data block remains constant; Specifically, define the security control head. H(t) : , in, seq ( t ) is the package number; frame _ id( t ) is the frame number; map _ id( t ) is the mapping table number; rate _ id( t ) is the bitrate or information set level number; rc _ modet ) represents the current CRC configuration; sf_flag (t) This is a superframe boundary or mode switching flag. The header CRC is: cH ( t ) =CRCrH ( H ( t )), in, cH ( t ) is the header CRC; rH It is the number of bits in the header CRC.
[0057] Strong protection head is: , in, H safe t ) indicates the control head after strong protection; || indicates series connection; Rep Indicates repetition r Alternatively, an equivalent short code protection may be adopted; r It is a repetition factor. Alternatively, instead of simple repetition, short BCH, short Polar, or parity combinations can be used, but the principle is that the error probability of the control head must be lower than the error probability of the core audio layer.
[0058] Layered load is defined as: , in, DH ( t This is high-priority audio data, including core parameters, key frequency band MSB, and necessary intra-frame structures; DM ( t This is medium-priority data, including common frequency band parameters and residuals; DL ( t ) is a low-priority enhancement layer, including LSB, strong masking residual and high-frequency details.
[0059] The CRC values for each layer are: ,in, Cl ( t ) is the first l Layer CRC; rl It is the number of CRC bits for this layer.
[0060] Generally, the following should be met: rH ≥ rM ≥ rL In this case, the number of bits for the higher-level CRC should not be less than that for the middle and lower-level CRC. Lower-priority layers may also omit the CRC setting and instead use LLR confidence and error concealment, since low-level errors should not trigger retransmissions or complex path selection.
[0061] The final Polar information payload is: , in, It is a high-priority audio data layer CRC, distinct from the control header CRC. cH ( t If the total length exceeds the current... Kq Cutting should be given priority. DL Secondly, compression DM No cutting allowed Hsafe and DH .
[0062] This can be written as: ; If not satisfied, then execute: , Among them, truncate( . ) is the enhancement layer clipping function. This rule ensures that the system reduces audio quality details first, rather than sacrificing parsing security, under weak links or low bit rates.
[0063] Polar input padding is: , in, It is the information payload number j One bit; Π m ( k) , q This is the current superframe mode. m ( k) and bitrate level q The corresponding mapping table; This is the frozen set at the current bitrate. Encoding complete: x ( t )= u ( t ) GN , The key point of this step is: always place the protection head in the most reliable position. The front layer; the high-priority audio layer follows immediately; the middle and lower layers can only use the remaining resources. The control header is not ordinary overhead, but a security anchor point for the entire UEP system.
[0064] In step S500 of some embodiments of the present invention, after applying enhanced error correction coding independent of the load sequence to the control head, the method further includes: assigning cyclic redundancy check sequences with different check lengths to the data blocks mapped to different protection regions, wherein the data blocks mapped to higher reliability protection regions are assigned cyclic redundancy check sequences with a length not less than that of lower reliability protection regions.
[0065] Specifically, the receiver first obtains channel soft information: , in, Li It is the first i Log-likelihood ratio of each codeword bit; yes It is a received symbol; xi It is the transmitted codeword bit. Li >0 indicates that it is more likely to be 0. Li >0 indicates that it is more likely to be 1, | Li The larger the value, the higher the confidence level. Polar decoding is then used to demap the data using the current mapping table: , in, It is the recovered data bit or data unit; These are the decoded Polar input bits; Π m,q It is a mapping table for the current mode and bitrate.
[0066] If CA-SCL decoding is used, path selection should consider not only the total path metric but also prioritize the control header CRC and high-priority layer CRC. Define the candidate path set: , in, It is the SCL candidate path index; It is the set of paths through which the control header CRC passes; This is the set of paths that both the control header and the high-priority data layer CRC pass through. The final path selection is: , in It is the first Path metrics for each path; This is the final selected path. This rule ensures that the control head takes precedence over ordinary audio data, and higher priority layers take precedence over mid-to-lower priority layers.
[0067] Group-level credibility is defined as: , in, Rg ( t ) is the group in the . g Frame reliability; | Gg | represents the number of bits within the group; LLRj It is the soft information mapped to the data unit, which can be obtained from the decoder's internal soft output or by approximating LLR; It is a confidence mapping function, for example: , in, It is the confidence growth factor. x The larger, The closer it is to 1, the more credible it is.
[0068] Receiver outputs availability label: in, Ag ( t ) is the first g Is the group available? LG This is the level to which this group belongs; CRC LG It checks whether the corresponding level's CRC passes; This is the confidence threshold for that layer. Higher priority layers should have higher thresholds, while lower priority layers can have more lenient thresholds or even be discarded entirely. The output to the audio restoration module is not simply... Instead: , in, It is the set of recovery states at the receiving end; eH , eM , eL These are the error states at each level; mt It is the mapping table number; q ( t () represents the bitrate level. This allows the subsequent PLC to process data more precisely based on its reliability.
[0069] It's understandable that the goal of error handling in audio equipment isn't to output all decoding results, but rather to output a sound that "sounds safe." If a high-priority layer is untrusted, historical parameters, interpolation, or a mute transition should be used; if a low-priority layer is untrusted, the enhancement layer should be discarded directly; if the control head is untrusted, conservative decoding or PLC should be used, rather than forcibly interpreting the current frame. The material emphasizes that safety protection, limiting, smoothing, confidence backoff, and bypass mechanisms are the core of productization, not just additional patches; on the communication recovery side, these principles are manifested in hierarchical PLCs and output safety valves.
[0070] refer to Figure 5 This illustrates the audio recovery and error concealment (decoding) process. The goal of error handling in audio equipment is not to output all decoded results, but rather to output a "safe-sounding" sound. If a high-priority layer is untrusted, historical parameters, interpolation, or a mute transition should be used; if a low-priority layer is untrusted, the enhancement layer should be discarded directly; if the control head is untrusted, conservative decoding or PLC should be initiated, rather than forcibly interpreting the current frame. The recovery process is the reverse of the encoding process and will not be repeated here.
[0071] Example 2 refer to Figure 7A second aspect of the present invention provides an audio encoding apparatus 1 based on perceptual importance-driven and Polar codes, comprising: The acquisition module 11 is used to acquire the original audio data to be transmitted and divide the original audio data into multiple data blocks with logical dependencies. The determination module 12 is used to determine the priority level of each data block according to a pre-built perceptual impairment cost model; the perceptual impairment cost model is established based on the degree of impairment of audio reconstruction quality caused by data block errors. Matching module 13 is used to obtain the target coding rate corresponding to the current communication link and provide a set of polar code information bits that match the target coding rate. The set of polar code information bits includes multiple protection regions distributed according to channel reliability. The mapping module 14 is used to map each of the data blocks to the corresponding matching protection region according to the priority level, so as to obtain a payload sequence; wherein, under different target coding rates, the mapping position of the protection region corresponding to the highest priority data block remains constant; The generation module 15 is used to generate a control head for indicating decoding rules, apply enhanced error correction coding independent of the payload sequence to the control head, and use the control head protected by the enhanced error correction coding and the payload sequence together as input for polar code encoding.
[0072] Furthermore, the determining module 12 includes: a query unit, used to obtain the basic damage weight of each data block by querying a preset damage priority mapping table, wherein the basic damage weight is obtained by injecting a preset error mode into the audio test sample in an offline state and evaluating the hearing damage index; and a determining unit, used to dynamically correct the basic damage weight according to the hearing scene mode and masking effect parameters perceived by the current system, so as to determine the priority level of the current frame.
[0073] Example 3 refer to Figure 8 A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the audio coding method based on perceptual importance-driven and Polar codes of the first aspect of the present invention.
[0074] Electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.
[0075] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, hard disks; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 8 Each box shown can represent a device or multiple devices as needed.
[0076] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0077] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to: Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and Python—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0078] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An audio coding method based on perceptual importance-driven and Polar codes, characterized in that, include: Obtain the raw audio data to be transmitted, and divide the raw audio data into multiple data blocks with logical dependencies; The priority level of each data block is determined based on a pre-constructed perceptual impairment cost model; the perceptual impairment cost model is established based on the degree of impairment to audio reconstruction quality caused by data block errors. Obtain the target coding rate corresponding to the current communication link, and provide a set of polar code information bits that match the target coding rate. The set of polar code information bits includes multiple protection regions distributed according to channel reliability. According to the priority level, each data block is mapped to its corresponding matching protection region to obtain a payload sequence; wherein, under different target coding rates, the mapping position of the protection region corresponding to the highest priority data block remains constant; A control header is generated to indicate the decoding rules. An enhanced error correction coding independent of the payload sequence is applied to the control header. The control header protected by the enhanced error correction coding and the payload sequence are used together as inputs for polar code encoding.
2. The audio coding method based on perceptual importance-driven and Polar codes according to claim 1, characterized in that, The step of determining the priority level of each data block based on the pre-built perceived damage cost model includes: The basic damage weight of each data block is obtained by querying a preset damage priority mapping table. The basic damage weight is obtained by injecting a preset error mode into the audio test sample and evaluating the hearing damage index in an offline state. The basic damage weights are dynamically adjusted based on the auditory scene patterns and masking effect parameters perceived by the current system to determine the priority level of the current frame.
3. The audio coding method based on perceptual importance-driven and Polar codes according to claim 1, characterized in that, During the processing of multiple consecutive audio data frames, if it is determined that the priority level of a certain data block has changed across levels, the switching of the protection region mapped to that data block will only be performed when the duration of the cross-level change or the number of consecutive frames reaches a preset hysteresis acknowledgment threshold.
4. The audio coding method based on perceptual importance-driven and Polar codes according to claim 1, characterized in that, The step of mapping each of the data blocks to its corresponding matching protection region includes: Based on a dual-loop control strategy, a set containing multiple mapping pattern tables is maintained, wherein the set includes at least one fixed parent table; the dual-loop control strategy includes: Based on the high-frequency execution cycle, the mapping configuration from data blocks to protected areas is executed strictly according to the currently selected mapping mode table in the current frame; Based on the low-frequency decision cycle, the benefit score of each mapping mode table under the current low-frequency decision cycle is calculated according to the link quality indicators and perception scene characteristics fed back by the receiver, and a decision is made on whether to switch to a new mapping mode table accordingly.
5. The audio coding method based on perceptual importance-driven and Polar codes according to claim 1, characterized in that, The multiple protection regions in the polar code information bit set include at least a first protection region corresponding to high reliability, a second protection region corresponding to medium reliability, and a third protection region corresponding to low reliability. When the link quality index is determined to be lower than the preset security threshold through the low-frequency decision cycle and a code rate reduction operation is triggered, the third protection region is stripped from the updated polar code information bit set, and the arrangement structure of the corresponding data in the first protection region remains unchanged.
6. The audio coding method based on perceptual importance-driven and Polar codes according to claim 1, characterized in that, After applying enhanced error correction coding independent of the payload sequence to the control head, the method further includes: The data blocks mapped to different protection regions are assigned cyclic redundancy check sequences with different check lengths, wherein the data blocks mapped to higher reliability protection regions are assigned cyclic redundancy check sequences with a length not less than that of lower reliability protection regions.
7. An audio encoding device based on perceptual importance-driven and Polar codes, characterized in that, The acquisition module is used to acquire the raw audio data to be transmitted and divide the raw audio data into multiple data blocks with logical dependencies. The determination module is used to determine the priority level of each data block based on a pre-built perceived damage cost model; The perceived impairment cost model is based on the degree of impairment to audio reconstruction quality caused by data block errors; The matching module is used to obtain the target coding rate corresponding to the current communication link and provide a set of polar code information bits that match the target coding rate. The set of polar code information bits includes multiple protection regions distributed according to the channel reliability. The mapping module is used to map each of the data blocks to the corresponding matching protection region according to the priority level, so as to obtain the payload sequence; wherein, under different target coding rates, the mapping position of the protection region corresponding to the highest priority data block remains constant; A generation module is used to generate a control head for indicating decoding rules, apply enhanced error correction coding independent of the payload sequence to the control head, and use the control head protected by enhanced error correction coding and the payload sequence together as input for polar code encoding.
8. The audio encoding device based on perceptual importance driving and Polar codes according to claim 7, characterized in that, The determining module includes: The query unit is used to obtain the basic damage weight of each data block by querying a preset damage priority mapping table. The basic damage weight is obtained by injecting a preset error mode into the audio test sample and evaluating the hearing damage index in an offline state. The determining unit is used to dynamically adjust the basic damage weights based on the auditory scene pattern and masking effect parameters perceived by the current system, so as to determine the priority level of the current frame.
9. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the audio coding method based on perceptual importance-driven and Polar codes as described in any one of claims 1 to 6.
10. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the audio encoding method based on perceptual importance-driven and Polar codes as described in any one of claims 1 to 6.
