Frame level parallel decoding method and system for speech codec

Abstract

The application discloses a frame-level parallel decoding method and system for a speech codec, which comprises the following steps: obtaining speech compression frames of a same speech session and analyzing frame parameters, dividing decoding sub-tasks into sub-tasks without inter-frame dependency, sub-tasks with speculatable inter-frame dependency and sequentially executed sub-tasks according to inter-frame dependency; dividing continuous speech compression frames into stage frame groups, executing the sub-tasks without inter-frame dependency in parallel based on stage input states and calculating speculatable intermediate quantities; checking according to real previous frame states according to frame serial numbers, re-calculating inconsistent intermediate quantities locally, sequentially completing corresponding decoding processing, obtaining decoding output and updated session states; and writing the last frame session state into a boundary state buffer for use by a next stage frame group. The application improves the parallel processing efficiency of speech frame decoding while maintaining the continuity of session states and the accuracy of decoding results.

Description

Technical Field

[0001] This application relates to the field of speech decoding technology, specifically to a frame-level parallel decoding method and system for speech codecs. Background Technology

[0002] With the development of services such as real-time audio and video communication, intelligent customer service, voice archiving, and communication quality analysis, servers need to centrally decode a large number of voice sessions. Existing narrowband voice codecs typically employ code-excited linear prediction technology. The decoding process includes bitstream parameter parsing, line spectrum parameter recovery, adaptive codebook and fixed codebook excitation generation, gain decoding, and linear prediction synthesis filtering. Some processing relies only on the coding parameters of the current voice frame, but line spectrum parameter interpolation, adaptive codebook excitation generation, gain prediction, and synthesis filtering also require the use of line spectrum parameters formed by previous frames, historical excitations, filter memory, and other states. Therefore, the same voice session is usually decoded sequentially according to frame number.

[0003] Currently, speech decoding primarily employs session-by-session processing by the central processing unit (CPU) or multi-threaded parallel processing of independent speech sessions. When the number of concurrent sessions is large, this approach is easily limited by the number of processor cores and memory access bandwidth. While graphics processing units (GPUs) possess strong parallel computing capabilities, directly decoding multiple speech frames from the same session simultaneously may result in erroneous results because the preceding states have not yet been generated. Conversely, executing each frame completely sequentially to ensure consistent results fails to leverage the parallel advantages of GPUs. Furthermore, frequently transferring session states between the host and GPU, or re-executing the entire decoding process when results are inconsistent, increases data transmission, thread synchronization, and redundant computation overhead. Therefore, improving frame-level parallelism for the same speech session while maintaining the continuity of speech frame states and the accuracy of decoding results remains a key technical challenge for batch speech decoding. Summary of the Invention

[0004] In view of this, the present disclosure provides a frame-level parallel decoding method and system for speech codecs, which can alleviate, reduce or even eliminate at least the above-mentioned problems.

[0005] According to one aspect of this application, a frame-level parallel decoding method for a speech codec is provided, comprising: Obtain the sequentially arranged voice compression frames in the same voice conversation, parse the frame parameters of the voice compression frames, and divide the decoding subtasks into subtasks without inter-frame dependencies, subtasks with predictable inter-frame dependencies, and sequential execution subtasks based on the inter-frame dependencies of the decoding subtasks. Multiple consecutive voice compression frames are divided into stage frame groups. The session initialization state is used as the stage input state of the first stage frame group, and the boundary true state output by the previous stage frame group is used as the stage input state of the next stage frame group. Parallel execution of each speech compression frame within a stage frame group without inter-frame dependency subtasks; intermediate quantities can be inferred by parallel calculation based on the stage input state and the frame parameters. According to the frame sequence number, each inferred intermediate quantity is checked against the actual preceding frame state. For inconsistent inferred intermediate quantities, local recalculation is performed. Based on the intermediate quantities that are consistent or locally recalculated, the corresponding speech compression frame sequential execution subtasks are executed to obtain the decoding output and the updated session state. Write the updated session state corresponding to the last valid voice compression frame into the boundary state cache to form the boundary true state of the next stage frame group, and output the decoded output according to the frame sequence number.

[0006] Optionally, the non-inter-frame-dependent subtasks include bit field parsing, frame type determination, fixed codebook parameter decoding, and gain index reading; the predictable inter-frame-dependent subtasks include line spectrum pair parameter decoding and interpolation, and adaptive codebook excitation generation; the sequentially executed subtasks include linear prediction synthesis filtering and gain predictor state update.

[0007] Optionally, each stage frame group contains a preset number of consecutive voice compressed frames in the same voice session; when there are fewer than the preset number of voice compressed frames at the end of the session, the number of valid frames in the corresponding stage frame group is recorded, and logical frame threads exceeding the number of valid frames are prohibited from reading voice compressed frames, updating session status, or writing to the decoding output.

[0008] Optionally, the inferred intermediate quantity includes the inferred line spectrum pair vector; when calculating the inferred line spectrum pair vector, the last frame line spectrum pair vector in the stage input state is read as the inferred reference vector, and inverse quantization and subframe interpolation are performed in combination with the line spectrum pair quantization index of the current speech compression frame; during verification, the real line spectrum pair vector is recalculated according to the real preceding frame state, and whether to perform local recalculation is determined based on the quantization step size normalization difference and the fixed-point coefficient comparison results between the inferred line spectrum pair vector and the real line spectrum pair vector.

[0009] Optionally, the predictable intermediate quantity further includes the predicted total excitation; when calculating the predicted total excitation, historical excitation samples in the stage input state are read, and a predicted pre-sequence excitation window is formed by combining the fixed codebook excitation and quantization gain candidate value in the previous speech compression frame within the current stage frame group; an adaptive codebook excitation is generated based on the predicted pre-sequence excitation window to form the predicted total excitation; during verification, the true total excitation is recalculated based on the true historical excitation, and whether to perform local recalculation is determined based on the normalized energy difference between the predicted total excitation and the true total excitation and the item-by-item comparison results of fixed-point samples.

[0010] Optionally, the sequential execution of the corresponding speech compression frame subtasks includes: converting the line spectrum pair vectors after verification or local recalculation into linear prediction coefficients according to the frame number; performing linear prediction synthesis filtering based on the linear prediction coefficients and the total excitation; updating the gain predictor state according to the gain index of the current speech compression frame; and combining the obtained line spectrum pair vectors, synthesis filter memory, historical excitation, pitch period, and gain predictor state to form the updated session state.

[0011] Optionally, the boundary state cache adopts a two-level state cache, including a first-level state cache located in the shared memory of the graphics processor and a second-level state cache located in the global memory of the graphics processor; when the stage frame group starts processing, the stage input state is loaded from the second-level state cache into the first-level state cache, each voice compression frame in the stage frame group accesses the first-level state cache, and after the stage frame group processing is completed, the updated session state is written to the second-level state cache.

[0012] Optionally, each voice session is configured with two boundary state slots for alternating read and write in the secondary state buffer; after the previous stage frame group completes the writing of the updated session state, a completion event is recorded; when the next stage frame group is in a different computing stream than the previous stage frame group, the completion event is waited for before the next stage frame group is started; when they are in the same computing stream, the next stage frame group is started according to the command order within the stream.

[0013] Optionally, host-to-graphics processor (GPU) transmission, GPU decoding, and GPU-to-host backhaul are performed through multiple computing streams. Each computing stream is configured with multiple circular buffer slots. The central processing unit (CPU) writes the pre-processor ready stage frame group into page-locked memory and asynchronously transmits it to the GPU. The CPU starts the decoding kernel in the corresponding computing stream and then asynchronously transmits the decoded output back to the host. The decoded output is written into the session output queue according to the voice session identifier and frame sequence number.

[0014] According to another aspect, a frame-level parallel decoding system for a speech codec is provided, comprising: The decoding task partitioning module is used to obtain the voice compression frames arranged sequentially in the same voice session, parse the frame parameters of the voice compression frames, and divide the decoding subtasks into subtasks without inter-frame dependencies, subtasks with predictable inter-frame dependencies, and sequential execution subtasks based on the inter-frame dependencies of the decoding subtasks. The stage frame group construction module is used to divide multiple consecutive voice compression frames into stage frame groups, take the session initialization state as the stage input state of the first stage frame group, and take the boundary real state output by the previous stage frame group as the stage input state of the next stage frame group. The parallel computing module is used to execute non-inter-frame dependent subtasks of each speech compression frame within the stage frame group in parallel, and can infer intermediate quantities based on the stage input state and the frame parameters in parallel computing. The verification decoding module is used to verify each inferred intermediate quantity according to the frame sequence number and the state of the real preceding frame. For inconsistent inferred intermediate quantities, it performs local recalculation and executes the corresponding speech compression frame sequential subtasks based on the intermediate quantities that are consistent or have been recalculated, so as to obtain the decoding output and the updated session state. The state buffer and output module is used to write the updated session state corresponding to the last valid voice compression frame into the boundary state buffer to form the boundary real state of the next stage frame group, and output the decoded output according to the frame sequence number.

[0015] According to the method of this application, the speech decoding subtasks are classified based on their inter-frame dependencies. Parameter parsing without inter-frame dependencies and the calculation of predictable intermediate quantities are arranged to be executed in parallel within the stage frame group. At the same time, sequential processing that depends on the real preceding state, such as synthesis filtering and gain predictor state updates, is retained. This improves the level of frame-level parallelism without disrupting the continuity of the same speech session state. The predicted line spectrum pair vector and the predicted total excitation are checked frame by frame according to the real preceding frame state. Only when the check is inconsistent, the corresponding intermediate quantities and their direct dependencies are recalculated, avoiding repeated execution of correctly completed parameter parsing and fixed codebook decoding, and reducing the computational overhead caused by prediction mismatch. Stage frame groups are connected by boundary real states, and a two-level state cache consisting of shared memory and global memory is used to reduce the repeated loading of session states and global memory access. Multiple computational flows further interleave data transmission, decoding calculation, and result return, thereby improving the decoding throughput of batch speech sessions and ensuring that the frame order, fixed-point operation rules, and state update results of the decoded output are consistent with the sequential decoding process. Attached Figure Description

[0016] This application will be more fully understood by referring to the following detailed description of specific embodiments in conjunction with the accompanying drawings, wherein the same reference numerals in the drawings refer to the same elements, wherein: Figure 1A flowchart illustrating a frame-level parallel decoding method for a speech codec according to some embodiments of the present disclosure is shown schematically.

[0017] Figure 2 A schematic block diagram of a frame-level parallel decoding system for a speech codec according to some embodiments of the present disclosure is shown.

[0018] Figure 3 A schematic block diagram of a computing device capable of implementing some embodiments of the present disclosure is shown. Detailed Implementation

[0019] Several embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to understand and implement the disclosure. However, the present disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided herein to make the disclosure comprehensive and complete, and to fully convey the scope of the disclosure to those skilled in the art. These embodiments are illustrative and not limiting of the disclosure.

[0020] It will be understood that although the terms first, second, third, etc., may be used herein to describe various elements, steps, and / or parts, these elements, steps, and / or parts should not be limited by these terms. These terms are used only to distinguish one element, step, or part from another. Therefore, the first element, step, or part discussed below may be referred to as the second element, step, or part without departing from the teachings of this disclosure.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprising” and / or “including” as used in this specification specify the presence of a feature, integral, step, operation, element, and / or component, but do not exclude the presence of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0022] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the relevant field and / or the context of this specification, and will not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0023] It should be noted that, provided there is no conflict, features in different embodiments can be combined and used arbitrarily.

[0024] This implementation uses a narrowband adaptive multi-rate speech codec (AMR-NB) as an example, processing multiple independent speech sessions. Each speech session contains speech compression frames arranged in chronological order, with each compressed frame corresponding to 20 milliseconds of speech. After the speech compression frames are input to the central processing unit (CPU), the CPU completes bitstream reading, frame parameter organization, stage frame group organization, and computational flow scheduling. The graphics processing unit (GPU) completes frame parameter decoding, inferable intermediate quantity calculation, local recalculation, linear predictive synthesis filtering, and gain predictor state updates. One example uses a GPU with shared memory, global memory, and multiple computational flows, with page-locked memory and a circular buffer slot configured on the host side. The decoding process uses the same fixed-point data format, shift rules, rounding rules, and saturation rules as the reference decoder.

[0025] The session state of a speech session includes at least the line spectrum vector, synthesis filter memory, historical excitation, pitch period, and gain predictor state. The session initialization state refers to the initial values ​​written into the state area before the speech session begins decoding. The line spectrum vector uses the initial stable values ​​specified by the reference decoder, the synthesis filter memory and historical excitation are set to zero, and the pitch period and gain predictor state are initialized according to the corresponding coding mode. A stage frame group refers to a processing unit formed by multiple consecutively obtained speech compression frames from the same speech session according to their frame numbers. The stage input state refers to the session state read when the current stage frame group begins processing. The boundary true state refers to the updated session state written after the previous stage frame group has completed the verification, local recalculation, and sequential execution of all valid speech compression frames. Inferential intermediate quantities refer to intermediate results pre-calculated based on the stage input state and frame parameters before the true preceding frame state within the stage frame group is obtained frame by frame.

[0026] The implementation process of the method described in this application will be described in detail below with reference to specific embodiments. It should be noted that this embodiment is only used to explain this application and is not intended to limit the scope of protection of this application. Conventional adjustments or substitutions of each step by those skilled in the art without departing from the concept of this application should be included in the scope of protection of this application.

[0027] Please see Figure 1 , Figure 1 A flowchart illustrating a frame-level parallel decoding method for a speech codec according to some embodiments of the present disclosure is shown schematically.

[0028] In step S1, the voice compression frames arranged sequentially in the same voice session are obtained, the frame parameters of the voice compression frames are parsed, and the decoding subtasks are divided into subtasks without inter-frame dependencies, subtasks with predictable inter-frame dependencies, and sequential execution subtasks based on the inter-frame dependencies of the decoding subtasks.

[0029] According to embodiments of this disclosure, the central processing unit (CPU) receives voice compression frames according to the voice session identifier and generates a frame sequence number based on the temporal order of each voice compression frame within the voice session. For an adaptive multi-rate narrowband bitstream, the CPU reads the encoding mode, frame quality flag, and frame type from the frame header, and then determines the frame payload length and the bit position of each frame parameter within the frame payload. The parsed frame parameters include a fixed codebook index, a fixed codebook pulse symbol, a gain index, a pitch delay index, a line spectrum pair quantization index, and a frame type flag. The CPU writes the above information into a frame description record, which also stores the voice session identifier, frame sequence number, bitstream start offset, bitstream length, output sampling offset, and valid frame flag.

[0030] The output sampling offset is determined by the frame number and the number of output sampling points per frame. Taking adaptive multi-rate narrowband speech with a sampling rate of 8 kHz as an example, each frame outputs 160 pulse code modulation sampling points. The output sampling offset corresponding to the speech compression frame with frame number 80 is 12,800 sampling points, and the subsequent decoding output is written to the session output buffer accordingly.

[0031] The central processing unit divides the decoding subtasks according to the input sources required for each frame parameter. Specifically, bit field parsing directly reads the bitstream of the current speech compression frame; frame type determination directly reads the frame header; fixed codebook parameter decoding reads the fixed codebook index and fixed codebook pulse symbol of the current speech compression frame; and gain index reading directly obtains the gain index in the current speech compression frame.

[0032] In one embodiment, line spectrum pair parameter decoding and interpolation require reference to the line spectrum pair vectors in the real preceding frame state, while adaptive codebook excitation generation requires reference to historical excitations in the real preceding frame state; both are categorized into the predictable inter-frame dependency subtask. Linear predictive synthesis filtering requires inheriting the synthesis filter memory according to the sampling order, and gain predictor state updates require inheriting the gain predictor state according to the frame sequence number; both are categorized into sequential execution subtasks. The CPU writes the corresponding dependency category into the frame description record, and the GPU reads the dependency category and enters a fixed processing branch, without reconstructing the general graph structure during runtime.

[0033] In one example, the frame type flag of a certain speech compression frame indicates the speech frame, and the parsing yields a fixed codebook index of 125, a gain index of 7, a pitch delay index of 42, and a 10-dimensional line spectrum pair quantization index. The graphics processor first recovers the fixed codebook excitation based on the fixed codebook index and fixed codebook pulse symbols, and then reads the quantization gain candidate values ​​based on the gain index. The line spectrum pair quantization index and pitch delay index are then used for line spectrum pair parameter decoding and interpolation, as well as adaptive codebook excitation generation, respectively. The intermediate results are then checked and fed into the linear predictive synthesis filter.

[0034] For different frame types, the central processing unit also calculates complexity weights. The complexity weight represents the ratio of the average decoding time for the corresponding frame type to the average decoding time for a speech frame, and is determined by the following formula:

[0035] in, For the first Complexity weights for speech-like compressed frames; This represents the average decoding time of this type of voice compression frame on the target graphics processor, in microseconds. This represents the average decoding time of speech frames under the same graphics processor and kernel configuration, expressed in microseconds. During calibration, at least 10,000 speech frames and silence description frames are selected respectively. After buffer warm-up, the kernel execution time is recorded and averaged. For example, if the average decoding time of a speech frame is 10 microseconds and the average decoding time of a silence description frame is 2 microseconds, then the complexity weight of the speech frame is 1.0, and the complexity weight of the silence description frame is 0.2. The complexity weight is written to the frame description record and is subsequently used to calculate the cumulative complexity weight of the frame group and allocate computational flow during the calculation phase; it does not participate in the numerical calculation of the decoded output.

[0036] In step S2, multiple consecutive voice compression frames are divided into stage frame groups. The session initialization state is used as the stage input state of the first stage frame group, and the boundary true state output by the previous stage frame group is used as the stage input state of the next stage frame group.

[0037] According to embodiments of this disclosure, the central processing unit (CPU) divides multiple consecutive voice compression frames into stage frame groups based on frame sequence numbers, using voice sessions as units. The number of voice compression frames in a stage frame group is determined based on the shared memory capacity of the graphics processor, the number of available threads in a single thread block, and local recalculation overhead; in this embodiment, each stage frame group contains 8 consecutive voice compression frames. If there are fewer than 8 voice compression frames at the end of a session, the CPU still generates a stage frame group and writes the number of valid frames into the stage frame group description record. If the frame sequence number corresponding to a logical frame thread exceeds the number of valid frames, no voice compression frames are read, the session state is not updated, and no decoding output is written.

[0038] Each stage frame group description record includes a voice session identifier, first frame number, number of valid frames, frame description record start position, input stream pointer, output buffer pointer, stage input state pointer, boundary true state write pointer, and completion event identifier. The stage input state pointer of the first stage frame group points to the session initialization state; the stage input state pointer of the subsequent stage frame group points to the boundary true state written by the previous stage frame group. The central processing unit (CPU) only submits the stage frame group to the graphics processing unit (GPU) after the corresponding stage input state has been written.

[0039] For example, a voice conversation may contain 1500 voice compression frames. Dividing this into 188 stage frame groups (each with 8 frames), the first 187 stage frame groups each contain 8 valid voice compression frames, and the 188th stage frame group contains 4 valid voice compression frames. The 11th stage frame group corresponds to frame numbers 80 to 87, and its stage input state comes from the boundary real states written after processing frames 72 to 79 in the 10th stage frame group. The 12th stage frame group only begins decoding after the boundary real states of the 11th stage frame group have been written.

[0040] The stage frame group size is calibrated when the graphics processor model or kernel version changes. Specifically, the CPU uses 8, 12, and 16 frames as candidate values, recording the shared memory usage, average execution time of the stage frame group, number of local recalculations, and preceding state wait time for each candidate value. The candidate value with the shortest average execution time and shared memory usage of a single thread block does not exceed the hardware limit is selected. In one example, the average execution time for an 8-frame stage frame group is 6.2 microseconds, for 12 frames it is 7.1 microseconds, and for 16 frames it is 8.5 microseconds. The shared memory usage for 8 frames is 42 kilobytes, which does not exceed the available shared memory capacity of a single thread block, so the stage frame group size is fixed at 8 frames. This value remains unchanged during a batch of voice session processing.

[0041] The central processing unit (CPU) sums the complexity weights of each voice compression frame within each stage frame group to obtain the cumulative complexity weight of the stage frame group. It then assigns the stage frame group to be submitted to the computation stream with the lowest current cumulative complexity weight. For example, if the cumulative complexity weights of the three stage frame groups to be submitted are 8.0, 5.6, and 7.2, and the current cumulative complexity weights of the four computation streams are 40.0, 36.4, 38.1, and 41.5, then the stage frame group with a cumulative complexity weight of 8.0 is first assigned to the second computation stream, updating its cumulative complexity weight to 44.4. The next stage frame group is then assigned based on these updated four values. If adjacent stage frame groups of the same voice session are located in the same computation stream, the intra-stream command order is used; if they are located in different computation streams, the later computation stream waits for the completion event of the previous stage frame group before starting the next stage frame group.

[0042] In step S3, the non-inter-frame dependent subtasks of each speech compression frame within the stage frame group are executed in parallel, and intermediate quantities can be inferred in parallel based on the stage input state and the frame parameters.

[0043] According to embodiments of this disclosure, the graphics processor uses stage frame groups as the scheduling unit, with one thread block processing one stage frame group. One implementation configures the thread block with 32 threads, where the first 8 threads correspond to 8 speech compression frames respectively, and the remaining threads participate in fixed codebook excitation recovery, linear prediction coefficient transformation, and synthesis filter multiplication and addition. After the thread block starts, the threads collaboratively read the stage input state and load the line spectrum pair vector, synthesis filter memory, gain predictor state, and the currently accessed 147 historical excitation samples into shared memory. This state region in shared memory constitutes a first-level state cache, and the session state region in the graphics processor's global memory constitutes a second-level state cache.

[0044] The first eight logical frame threads read the corresponding frame description records and execute bit field parsing, frame type determination, fixed codebook parameter decoding, and gain index reading in parallel. Bit field parsing extracts the fixed codebook index, fixed codebook pulse symbol, gain index, pitch delay index, and line spectrum pair quantization index from the input bitstream according to the bit allocation table corresponding to the encoding mode. Fixed codebook parameter decoding determines the pulse position based on the fixed codebook index and the pulse sign based on the fixed codebook pulse symbol, forming a fixed codebook excitation consisting of 160 fixed-point samples. After reading the gain index, the gain quantization table stored in the read-only area of ​​the graphics processor's global memory is queried to obtain quantization gain candidate values.

[0045] For example, in a certain encoding mode, the fixed codebook index corresponds to four non-zero pulses at positions 12, 48, 91, and 133. The fixed codebook pulse signs are positive, negative, positive, and positive, respectively. The fixed codebook excitation is written with the corresponding fixed-point amplitude at these four positions, and 0 is written at the remaining 156 positions. Gain index 7 corresponds to the adaptive codebook gain candidate value of 0.72 and the fixed codebook gain candidate value of 0.35 in the gain quantization table. These two quantized gain candidate values, along with the fixed codebook excitation and the subsequently generated adaptive codebook excitation, participate in the calculation of the predicted total excitation.

[0046] In one embodiment, the predictable intermediate quantities in the parallel phase include the predicted line spectrum pair vector and the predicted total excitation. When calculating the predicted line spectrum pair vector, each logical frame thread reads the last frame's line spectrum pair vector from the phase input state as the prediction reference vector, and performs dequantization and subframe interpolation in conjunction with the line spectrum pair quantization index of the current speech compression frame. Adaptive multi-rate narrowband decoding employs 10th-order linear prediction, thus each subframe forms a 10-dimensional predicted line spectrum pair vector. This predicted line spectrum pair vector is stored in shared memory and subsequently checked against the actual preceding frame state.

[0047] Taking a 10-dimensional inferred line spectrum pair vector as an example, the coefficients of the first three line spectrum pairs in the stage input state are 0.12, 0.27, and 0.41. The inverse quantization increments of the current speech compression frame are 0.01, -0.02, and 0.00, so the corresponding inferred values ​​are 0.13, 0.25, and 0.41. The remaining seven coefficients are processed in the same way. During subframe interpolation, the line spectrum pair vectors of each subframe are calculated according to the interpolation coefficients specified by the reference decoder. The line spectrum pair coefficients are stored in fixed-point format. The decimals in the example are only used to illustrate the numerical relationships and are converted to the fixed-point format used by the reference decoder before being written to shared memory.

[0048] In calculating the total speculative excitation, thread 0 first establishes a speculative pre-excitation window for each speech compression frame based on historical excitation samples from the stage input state. For the first speech compression frame, the speculative pre-excitation window is entirely derived from historical excitations in the stage input state; for subsequent speech compression frames, the portion of the window falling into the current stage frame group uses the fixed codebook excitation and quantization gain candidate values ​​from the previous speech compression frames to form a temporary excitation. Each logical frame thread reads the corresponding speculative pre-excitation window, obtains the delay position based on the pitch delay index, and performs adaptive codebook interpolation to generate an adaptive codebook excitation; subsequently, it scales the adaptive codebook excitation and the fixed codebook excitation according to the quantization gain candidate values, and adds them sample by sample to form a total speculative excitation consisting of 160 fixed-point samples.

[0049] For example, if the pitch delay of the current speech compression frame is 40 samples, and the 0th to 3rd referenced samples in the inferred pre-excitation window are 100, 80, -20, and 60 respectively, the interpolation coefficients correspond to adaptive codebook excitation samples of 90, 35, 10, and 55. The candidate value for adaptive codebook gain is 0.72, and the candidate value for fixed codebook gain is 0.35. The samples for the fixed codebook excitation at the corresponding four positions are 512, -512, 512, and 512. After scaling and summing, the inferred total excitation at the corresponding positions is obtained. The actual computation uses fixed-point multiplication, specified shifting, and saturation processing, and does not directly use floating-point numbers. The inferred total excitation is written to shared memory as a subsequent verification object.

[0050] The parallel processing described above pre-processes most of the parameter parsing, fixed codebook processing, and calculation of inferred intermediate quantities for each speech compression frame in the current stage frame group; the actual preceding frame state is still formed frame by frame according to the frame number, so the inferred line spectrum pair vector and the inferred total excitation only enter the sequential execution of subtasks after they are verified to be consistent or after local recalculation is completed.

[0051] In step S4, each inferred intermediate quantity is checked sequentially according to the frame number based on the actual preceding frame state. Inconsistent inferred intermediate quantities are locally recalculated. Based on the intermediate quantities that are checked and found to be consistent or locally recalculated, the corresponding speech compression frame sequential execution subtasks are executed to obtain the decoding output and the updated session state.

[0052] According to an embodiment of this disclosure, after the thread block completes the parallel phase, it performs intra-block synchronization. Thread 0 processes the first valid audio compression frame of the stage frame group according to the frame sequence number. The real preceding frame state corresponding to the current audio compression frame comes from the updated session state formed after the previous audio compression frame completed the sequential execution of subtasks; for the first valid audio compression frame of the stage frame group, the real preceding frame state is equal to the stage input state. This processing order ensures that each audio compression frame enters the verification process after the real state of the previous audio compression frame has been formed, without using states that have not yet been generated.

[0053] In the process of verifying the inferred line spectrum pair vector, thread 0 re-executes the line spectrum pair parameter decoding and interpolation based on the line spectrum pair vector in the real preceding frame state and the line spectrum pair quantization index of the current speech compression frame to obtain the real line spectrum pair vector; then, it calculates the normalized difference in quantization step size between the inferred line spectrum pair vector and the real line spectrum pair vector:

[0054] in, Indicates the first Normalized difference in quantization step size of each voice compression frame; This represents the first line in the vector of the inferred line spectrum pair. Each line spectrum pair coefficient; Represents the first line in the vector of true line pairs. Each line spectrum pair coefficient; Indicates the number of quantization intervals within the current quantization interval. The quantization step size for each line spectrum pair coefficient. Both the line spectrum pair coefficients and the quantization step size use the same fixed-point numerical scale.

[0055] In one example, the predicted values ​​of three line spectrum pair coefficients are 0.130, 0.250, and 0.410, while the actual values ​​are 0.132, 0.246, and 0.410. The corresponding quantization step size is 0.010. Therefore, the three normalized differences are 0.2, 0.4, and 0, respectively, and the quantization step size normalized difference is 0.4. When the quantization step size normalized difference is no greater than 0.5, thread 0 processes the predicted line spectrum pair vector according to the reference decoder's fixed-point rounding rules and continues to perform a step-by-step comparison of fixed-point coefficients. If all 10 fixed-point coefficients are the same, the predicted line spectrum pair vector is retained. If any coefficient is different, the decoding and interpolation of the line spectrum pair parameters for the current speech compression frame are locally recalculated. When the quantization step size normalized difference is greater than 0.5, the aforementioned local recalculation is performed directly. 0.5 corresponds to half a quantization step size; the final decision on whether to retain the predicted line spectrum pair vector is still based on the result of the step-by-step comparison of fixed-point coefficients.

[0056] In the process of verifying the inferred total excitation, thread 0 regenerates the adaptive codebook excitation and forms the true total excitation based on the historical excitation in the true preceding frame state, the pitch delay index of the current speech compression frame, the fixed codebook excitation, and the quantization gain candidate value; then, the normalized energy difference between the inferred total excitation and the true total excitation is calculated:

[0057] in, Indicates the first Normalized energy difference of each voice compression frame; Indicates the first in the projected total incentive One fixed-point sample; Represents the first in the true total incentive One fixed-point sample; This represents the lower limit of energy, expressed in squares of the fixed-point excitation amplitude, and is used to handle the case where the actual total excitation energy is 0.

[0058] For example, simplifying the verification process by taking 4 samples, the predicted total excitation is 100, 60, -20, and 40, while the actual total excitation is 98, 62, -20, and 39. The sum of squares of the differences is 9, and the sum of squares of the actual total excitation is 15089. Ignoring the lower limit of energy values ​​much smaller than the sum of squares of the actual total excitation, the normalized energy difference is approximately 0.024. The complete processing calculates for 160 samples. When the normalized energy difference is no greater than 0.05, thread 0 continues to perform item-by-item comparison of fixed-point samples. If all 160 fixed-point samples are the same, the predicted total excitation is retained. If there are different samples, the adaptive codebook excitation generation and total excitation combination of the current speech compression frame are locally recalculated. When the normalized energy difference is greater than 0.05, the aforementioned local recalculation is performed directly.

[0059] In one embodiment, the comparison threshold for normalized energy differences is obtained through offline calibration on the target graphics processor. Calibration data covers various encoding modes, speech frames, silence description frames, and different input amplitudes, testing values ​​of 0.01, 0.03, 0.05, 0.08, and 0.10. The kernel execution time, the number of speech compression frames directly entering local recalculation, and the number of fixed-point sample comparisons corresponding to each candidate value are statistically analyzed, and the candidate value with the shortest total kernel execution time is selected. For example, if the average kernel execution times corresponding to 0.03, 0.05, and 0.08 are 7.0 microseconds, 6.4 microseconds, and 6.8 microseconds respectively, then 0.05 is adopted. Recalibration is performed when the graphics processor model, kernel version, or fixed-point format changes, and remains unchanged during a batch of speech session processing.

[0060] In one embodiment, local recalculation only covers inconsistent speculative intermediate values ​​and their direct dependencies. When the speculative line spectrum pair vectors are inconsistent, the line spectrum pair parameter decoding and interpolation of the current speech compression frame are re-executed; when the speculative total excitation is inconsistent, the adaptive codebook excitation generation and the combination of adaptive and fixed codebook excitations for the current speech compression frame are re-executed. The results of bit field parsing, frame type determination, fixed codebook parameter decoding, and gain index reading are reused without being processed repeatedly. The locally recalculated line spectrum pair vectors and total excitations cover the corresponding speculative intermediate values ​​in shared memory, becoming the input for the current speech compression frame to enter the sequential execution subtasks.

[0061] After the line spectrum pair vector and the total excitation are determined, the thread block converts the line spectrum pair vector into linear prediction coefficients and performs linear prediction synthesis filtering according to the subframe order. Each speech compression frame of adaptive multi-rate narrowband speech contains 4 subframes, each with 40 sampling points. The synthesis filtering starts from the synthesis filter memory in the state of the real preceding frame, calculates 160 pulse code modulation sampling points in the sampling point order, and updates the synthesis filter memory after processing each sampling point. Thread 0 controls the sampling order, and the other threads are responsible for the multiplication and addition terms of the previous sampling points. The reduction result is written to the decoding output according to the shift, rounding, and saturation rules of the reference decoder.

[0062] The gain index of the current speech compression frame and the gain predictor state in the true preceding frame state jointly participate in gain decoding. Thread 0 reads the quantization error term based on the gain index, combines it with the gain predictor state to obtain the current frame gain, and writes the prediction history corresponding to the current frame gain back to the gain predictor state. After the current speech compression frame is processed, the resulting line spectrum pair vector, synthesis filter memory, historical excitation, pitch period, and gain predictor state constitute the updated session state, which is immediately used as the true preceding frame state for the next speech compression frame.

[0063] Taking frames 80 and 81 in the stage frame group as examples, frame 80 uses the stage input state as the real preceding frame state. After completing verification, local recalculation, and sequential execution of subtasks, it forms the updated session state. Frame 81 then reads this updated session state and verifies its own inferred line spectrum pair vector and inferred total excitation. If local recalculation occurs in frame 80, the historical excitation and line spectrum pair vector generated by the recalculation will enter the real preceding frame state of frame 81, and therefore will not continue to use the inferred reference values ​​used by frame 81 in the parallel stage. After processing sequentially until the last valid speech compression frame, the stage frame group obtains continuous decoding output and the final updated session state.

[0064] In step S5, the updated session state corresponding to the last valid voice compression frame is written into the boundary state cache to form the boundary true state of the next stage frame group, and the decoded output is output according to the frame sequence number.

[0065] According to an embodiment of this disclosure, after the last valid speech compression frame of a stage frame group completes the sequential execution of subtasks, thread 0 writes the corresponding updated session state into the boundary state cache, forming the boundary true state of the next stage frame group. The boundary state cache adopts a two-level state cache. The first-level state cache is located in the shared memory of the graphics processor and stores the line spectrum pair vectors, synthesis filter memory, historical excitation, pitch period, gain predictor state, and predictable intermediate quantities currently accessed by the stage frame group. The second-level state cache is located in the global memory of the graphics processor and stores the session initialization state, boundary true state, and complete decoding output of each speech session.

[0066] Each voice session has two boundary state slots in the secondary state buffer that are alternately read and written. When processing the current stage frame group, one boundary state slot provides the stage input state, and the other boundary state slot receives the boundary real state formed by the current stage frame group; after writing is completed, the read and write relationship of the two boundary state slots is swapped. The boundary real state record also stores the voice session identifier and the last frame sequence number. Before starting the next stage frame group, it checks whether the sequence number of its first frame is the next frame after the sequence number of the last frame. If the sequence numbers are not consecutive, the submission of the next stage frame group of the voice session is stopped, and the task is reorganized from the most recent valid boundary real state.

[0067] When a frame group in the current stage resides within a specific computational stream, the CPU records a completion event after the decoding kernel of that computational stream. When a frame group in the next stage resides within the same computational stream, the boundary state is read sequentially based on the intra-stream commands. When they reside in different computational streams, the CPU inserts a wait for the completion event in the next computational stream; the completion event indicates that the decoding kernel and boundary state writing for the previous frame group have both been completed. Different voice sessions do not have session state transfer relationships and can be processed simultaneously in different computational streams.

[0068] The host-to-GPU transmission, GPU decoding, and GPU-to-host backhaul are organized in parallel using multiple computational streams. Each computational stream is configured with multiple circular buffer slots, each containing an input bitstream area, a stage frame group description area, a decoding output area, and a completion event. The CPU retrieves stage frame groups with ready input states from the submission queue, writes the corresponding input bitstream and stage frame group description records into page-locked memory, and sends them to the GPU asynchronously. The same computational stream then starts the decoding kernel, records the completion event after the decoding kernel finishes, and asynchronously sends the decoded output corresponding to the valid voice compression frame back to the host.

[0069] In one example, eight computational streams are used, each with four circular buffer slots, holding a maximum of 32 in-transit stage frame groups simultaneously. When a circular buffer slot receives a stage frame group containing eight valid voice compression frames, it sends back 1280 16-bit pulse code modulation (PCM) samples; if it receives a stage frame group at the end of the session containing only four valid voice compression frames, it sends back 640 samples. The central processing unit reads the voice session identifier and the first frame sequence number, determines the write position based on 160 samples per frame, writes the decoded output to the session output queue, then releases the circular buffer slot and submits the next stage frame group.

[0070] The session output queue manages the write intervals according to frame sequence numbers, not the order in which different computational flows are completed. Different speech session stage frame groups can be returned in different orders. Within the same speech session, a later stage frame group is only initiated after the boundary state of the previous stage frame group has been written. The central processing unit (CPU) then writes the decoded output to the corresponding sampling interval based on the first frame sequence number of each stage frame group. After all stage frame groups are completed, the CPU reads the session output queue according to the output sampling offset order, forming continuous pulse-code modulated speech data, which is then written to an audio file or sent to subsequent speech processing units.

[0071] In actual deployment, the graphics processor (GPU) pre-allocates frame description records, stage frame group description records, two boundary state slots, a frame parameter area, and a decoding output buffer. The decoding kernel does not request dynamic video memory during operation. The shared memory required for a stage frame group includes the stage input state, frame parameters for eight speech compression frames, fixed codebook excitations, speculative line spectrum pair vectors, speculative total excitations, and a comparison area. After compilation, the shared memory usage of a single thread block is read. If this exceeds the single-block capacity of the target GPU, the fixed codebook excitations or comparison area used only for temporary storage are moved to pre-allocated global memory, without changing the processing relationships between stage frame groups, stage input states, speculative intermediate quantities, and boundary true states.

[0072] Using the above method, the sequential state transfer within the voice session is preserved during frame-by-frame verification, local recalculation, linear predictive synthesis filtering, and gain predictor state updates. Inter-frame dependent subtasks and predictable intermediate quantities are computed in parallel within stage frame groups. Boundary true states are only transferred between adjacent stage frame groups. A two-level state buffer reduces repeated accesses to global memory within stage frame groups, and multiple computational streams allow input transmission, GPU decoding, and output feedback to be executed interleaved. The final decoded output is written to the session output queue according to the frame number, and the updated session state is continuously transferred according to the order of the voice session and stage frame groups.

[0073] Figure 2 A schematic block diagram of a frame-level parallel decoding system for a speech codec according to some embodiments of the present disclosure is shown. Figure 2As shown, the system includes: The decoding task partitioning module 2001 is used to obtain the voice compression frames arranged sequentially in the same voice session, parse the frame parameters of the voice compression frames, and divide the decoding subtasks into subtasks without inter-frame dependencies, subtasks with predictable inter-frame dependencies, and sequential execution subtasks based on the inter-frame dependencies of the decoding subtasks. The stage frame group construction module 2002 is used to divide multiple consecutive voice compression frames into stage frame groups, take the session initialization state as the stage input state of the first stage frame group, and take the boundary real state output by the previous stage frame group as the stage input state of the next stage frame group. Parallel computing module 2003 is used to execute non-inter-frame dependent subtasks of each speech compression frame within a stage frame group in parallel, and to infer intermediate quantities based on the stage input state and the frame parameters in parallel computing. The verification decoding module 2004 is used to verify each inferred intermediate quantity according to the frame sequence number and the state of the real preceding frame. For inconsistent inferred intermediate quantities, it performs local recalculation and executes the corresponding speech compression frame sequential subtasks based on the intermediate quantities that are verified to be consistent or partially recalculated, so as to obtain the decoding output and the updated session state. The state buffer and output module 2005 is used to write the updated session state corresponding to the last valid voice compression frame into the boundary state buffer to form the boundary real state of the next stage frame group, and output the decoded output according to the frame sequence number.

[0074] The specific implementation method and technical effects of this system can be referred to the description of the method above, and will not be repeated here.

[0075] Figure 3 A schematic block diagram of a computing device capable of implementing some embodiments of the present disclosure is shown. In some embodiments, the computing device may include various different types of devices, such as computing devices, computers, client devices, system-on-a-chip, and / or any other suitable computing devices.

[0076] The computing device may include at least one processor 1302, memory 1304, multiple communication interfaces 1306, display device 1308, other input / output (I / O) devices 1310, and one or more mass storage devices 1313 that are capable of communicating with each other, such as via system bus 1311 or other suitable means.

[0077] Processor 1302 may be a single processing unit or multiple processing units, and all processing units may include a single or multiple computing units or multiple cores. Processor 1302 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, processor 1302 may be configured to acquire and execute computer-readable instructions stored in memory 1304, mass storage device 1313, or other computer-readable media, such as program code of operating system 1316, program code of application program 1318, program code of other program 1320, etc., to implement the methods provided in the embodiments of this disclosure.

[0078] Memory 1304 and mass storage device 1313 are examples of computer storage media for storing instructions that are executed by processor 1302 to perform the various functions described above. For example, memory 1304 may generally include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, mass storage device 1313 may generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Both memory 1304 and mass storage device 1313 may be collectively referred to herein as memory or computer storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by processor 1302 as a specific machine configured to perform the operations and functions described in the examples herein.

[0079] Multiple program modules can be stored on mass storage device 1313. These programs include operating system 1316, one or more application programs 1318, other programs 1320, and program data 1322, and they can be loaded into memory 1304 for execution. Examples of such application programs or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the methods provided herein. Moreover, these program modules can be distributed across different physical locations to achieve their respective functions.

[0080] Although Figure 3The modules 1313, 1318, 1320, and 1322, or portions thereof, are illustrated as being stored in memory 1304 of a computing device; however, modules 1313, 1318, 1320, and 1322 may be implemented using any form of computer-readable medium accessible by a computing device. As used herein, “computer-readable medium” includes at least two types of computer-readable media: computer storage media and communication media.

[0081] Computer storage media includes volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD, or other optical storage devices, magnetic cartridges, magnetic tapes, disk storage devices or other magnetic storage devices, or any other non-transfer medium that can be used to store information for access by computing devices.

[0082] In contrast, communication media can embody computer-readable instructions, data structures, program modules, or other data within modulated data signals such as carrier waves or other transmission mechanisms. Computer storage media as defined herein do not include communication media.

[0083] The computing device may also include one or more communication interfaces 1306 for exchanging data with other devices, such as via a network, direct connection, etc. Communication interface 1306 can facilitate communication across various network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. Communication interface 1306 can also provide communication with external storage devices (not shown), such as storage arrays, network-attached storage, storage area networks, etc.

[0084] In some examples, a display device 1308, such as a monitor, may be included for displaying information and images. Other I / O devices 1310 may be devices that receive various inputs from a user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and so on.

[0085] This application provides a computer-readable storage medium storing computer-readable instructions thereon, which, when executed, implement the above-described method.

[0086] This application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computing device to perform the methods provided in the various optional implementations described above.

[0087] In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example that is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0088] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed (including substantially simultaneously or in reverse order depending on the functions involved), as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0089] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0090] In the above embodiments, the descriptions of each embodiment have different focuses. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. The above descriptions are merely preferred embodiments of this application and explanations of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by specific combinations of the above technical features, but should also cover other technical solutions formed by arbitrary combinations of the above technical features or their equivalent features without departing from the inventive concept.

Claims

1. A frame-level parallel decoding method for a speech codec, characterized in that, include: The system acquires sequentially arranged audio compressed frames from the same audio conversation, parses the frame parameters of the audio compressed frames, and divides the decoding subtasks into subtasks without inter-frame dependencies, subtasks with predictable inter-frame dependencies, and sequential execution subtasks based on the inter-frame dependencies of the decoding subtasks. The subtasks with predictable inter-frame dependencies include line spectrum pair parameter decoding and interpolation, as well as adaptive codebook excitation generation. Multiple consecutive voice compression frames are divided into stage frame groups. The session initialization state is used as the stage input state of the first stage frame group, and the boundary true state output by the previous stage frame group is used as the stage input state of the next stage frame group. Parallel execution of each speech compression frame within a stage frame group without inter-frame dependency subtasks; intermediate quantities can be inferred by parallel calculation based on the stage input state and the frame parameters. According to the frame number, each inferred intermediate quantity is checked against the state of the real preceding frame. Inconsistent inferred intermediate quantities are locally recalculated. Based on the consistent or locally recalculated intermediate quantities, the corresponding speech compression frames are executed sequentially to obtain the decoding output and the updated session state. The inferred intermediate quantities include inferred line spectrum pair vectors. When calculating the inferred line spectrum pair vector, the last frame line spectrum pair vector in the stage input state is read as the inferred reference vector, and inverse quantization and subframe interpolation are performed in conjunction with the line spectrum pair quantization index of the current speech compression frame. During verification, the real line spectrum pair vector is recalculated based on the state of the real preceding frame, and the inferred line spectrum pair vector is compared with the real preceding frame state. The difference in quantization step size normalization of the solid line spectrum vector and the item-by-item comparison of fixed-point coefficients determine whether to perform local recalculation; the inferred intermediate quantity also includes the inferred total excitation; when calculating the inferred total excitation, historical excitation samples in the stage input state are read, and a pre-inferred excitation window is formed by combining the fixed codebook excitation and quantization gain candidate value in the previous speech compression frame within the current stage frame group; an adaptive codebook excitation is generated based on the pre-inferred excitation window to form the inferred total excitation; during verification, the real total excitation is recalculated based on the real historical excitation, and the difference in normalized energy between the inferred total excitation and the real total excitation and the item-by-item comparison of fixed-point samples determine whether to perform local recalculation; Write the updated session state corresponding to the last valid voice compression frame into the boundary state cache to form the boundary true state of the next stage frame group, and output the decoded output according to the frame sequence number.

2. The method according to claim 1, characterized in that, The non-inter-frame-dependent subtasks include bit field parsing, frame type identification, fixed codebook parameter decoding, and gain index reading; the sequential execution subtasks include linear prediction synthesis filtering and gain predictor state update.

3. The method according to claim 2, characterized in that, Each stage frame group contains a preset number of consecutive voice compressed frames in the same voice session; when there are fewer than the preset number of voice compressed frames at the end of the session, the number of valid frames in the corresponding stage frame group is recorded, and logical frame threads that exceed the number of valid frames are prohibited from reading voice compressed frames, updating session status, or writing to the decoding output; wherein the logical frame thread is a thread that establishes a correspondence with the voice compressed frames in the stage frame group.

4. The method according to claim 3, characterized in that, The sequential execution of the corresponding speech compression frame subtasks includes: converting the line spectrum pair vectors after verification or local recalculation into linear prediction coefficients according to the frame number; performing linear prediction synthesis filtering based on the linear prediction coefficients and the total excitation; updating the gain predictor state according to the gain index of the current speech compression frame; and combining the obtained line spectrum pair vectors, synthesis filter memory, historical excitation, pitch period, and gain predictor state to form the updated session state.

5. The method according to claim 4, characterized in that, The boundary state cache adopts a two-level state cache, including a first-level state cache located in the shared memory of the graphics processor and a second-level state cache located in the global memory of the graphics processor. When a stage frame group begins processing, the stage input state is loaded from the secondary state cache to the primary state cache. Each voice compression frame in the stage frame group accesses the primary state cache. After the stage frame group processing is completed, the updated session state is written to the secondary state cache.

6. The method according to claim 5, characterized in that, Each voice session is configured with two boundary state slots for alternating read and write in the secondary state buffer; after the previous stage frame group completes the writing of the updated session state, a completion event is recorded. When the next stage frame group is in a different computing stream than the previous stage frame group, it waits for the completion event before starting the next stage frame group. When they are in the same computing stream, the next stage frame group is started according to the command order within the stream.

7. The method according to claim 6, characterized in that, The process involves multiple compute streams to perform host-to-graphics processor (GPU) transmission, GPU decoding, and GPU-to-host backhaul. Each compute stream is configured with multiple circular buffer slots. The central processing unit (CPU) writes the pre-processor-ready stage frame group into page-locked memory and asynchronously transmits it to the GPU. In the corresponding compute stream, the decoding kernel is started, and the decoded output is then asynchronously transmitted back to the host and written into the session output queue according to the voice session identifier and frame sequence number.

8. A frame-level parallel decoding system for a speech codec, characterized in that, include: The decoding task partitioning module is used to acquire sequentially arranged voice compression frames in the same voice session, parse the frame parameters of the voice compression frames, and divide the decoding subtasks into subtasks without inter-frame dependencies, subtasks with predictable inter-frame dependencies, and sequential execution subtasks based on the inter-frame dependencies of the decoding subtasks; wherein, the subtasks with predictable inter-frame dependencies include line spectrum pair parameter decoding and interpolation and adaptive codebook excitation generation. The stage frame group construction module is used to divide multiple consecutive voice compression frames into stage frame groups, take the session initialization state as the stage input state of the first stage frame group, and take the boundary real state output by the previous stage frame group as the stage input state of the next stage frame group. The parallel computing module is used to execute non-inter-frame dependent subtasks of each speech compression frame within the stage frame group in parallel, and can infer intermediate quantities based on the stage input state and the frame parameters in parallel computing. The decoding verification module is used to sequentially verify each inferred intermediate quantity according to the frame number and the state of the real preceding frame. Inconsistent inferred intermediate quantities are locally recalculated. Based on the verified or recalculated intermediate quantities, the corresponding speech compression frames are executed sequentially to obtain the decoding output and the updated session state. The inferred intermediate quantities include inferred line spectrum pair vectors. When calculating the inferred line spectrum pair vector, the last frame line spectrum pair vector in the stage input state is read as the inferred reference vector, and inverse quantization and subframe interpolation are performed in conjunction with the line spectrum pair quantization index of the current speech compression frame. During verification, the real line spectrum pair vector is recalculated based on the state of the real preceding frame, and the inferred line spectrum pair vector is used as the reference vector. The normalized difference in quantization step size between the quantity and the true line spectrum vector, and the item-by-item comparison of fixed-point coefficients, determine whether to perform local recalculation; the inferred intermediate quantity also includes the inferred total excitation; when calculating the inferred total excitation, historical excitation samples in the stage input state are read, and a pre-inferred excitation window is formed by combining the fixed codebook excitation and quantization gain candidate value in the previous speech compression frame within the current stage frame group; an adaptive codebook excitation is generated based on the pre-inferred excitation window to form the inferred total excitation; during verification, the true total excitation is recalculated based on the true historical excitation, and the item-by-item comparison of the normalized energy difference between the inferred total excitation and the true total excitation, and the item-by-item comparison of fixed-point samples, determine whether to perform local recalculation; The state buffer and output module is used to write the updated session state corresponding to the last valid voice compression frame into the boundary state buffer to form the boundary real state of the next stage frame group, and output the decoded output according to the frame sequence number.

Images