Encoding method, encoding apparatus, decoding method, decoding apparatus, and transmission system

Abstract

The present disclosure relates to the technical field of signal processing, and in particular relates to an encoding method, an encoding apparatus, a decoding method, a decoding apparatus, and a transmission system. The encoding method comprises: on the basis of a signal to be encoded, obtaining a plurality of multi-description signals; performing quantization processing on the plurality of multi-description signals, so as to determine a plurality of first candidate quantized signals corresponding to each of the plurality of multi-description signals; on the basis of the signal to be encoded and a fused quantized signal, determining from the plurality of first candidate quantized signals a final quantized signal corresponding to each multi-description signal, wherein the fused quantized signal is determined on the basis of the first candidate quantized signals corresponding to different multi-description signals; and on the basis of the final quantized signal corresponding to each multi-description signal, generating a bitstream. The technical solution of the present disclosure can improve packet loss resistance, thereby improving the quality of audio.

Description

Coding method, coding device, decoding method, decoding device and transmission system

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority to an application filed in China with application number 202410155422.X and filing date February 2, 2024. The disclosed content of the Chinese application is hereby introduced as a whole into this application. Technical Field

[0003] The present disclosure relates to the field of signal processing technology, and in particular to an encoding method, an encoding device, a decoding method, a decoding device, a transmission system, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0004] Interactive entertainment applications encompass a wide range of music-related scenarios beyond social interaction. These include live streaming, chorus singing, and Karaoke TV (Karaoke TV). In these music scenarios, users generally have high quality requirements and expect a smooth music experience, even on weak network connections.

[0005] In the related art, in an RTC (Real-Time Communication) music scenario, a PLC (Packet Loss Concealment) algorithm in the Opus music coding mode is used as an anti-packet loss technology. Summary of the Invention

[0006] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: weak anti-packet loss capability, resulting in poor audio quality.

[0007] In view of this, the present disclosure proposes a coding technology solution that can improve the anti-packet loss capability and thus improve the audio quality.

[0008] According to some embodiments of the present disclosure, a coding method is provided, comprising: obtaining multiple multi-description signals based on a signal to be coded; performing quantization processing on the multiple multi-description signals to determine multiple first candidate quantization signals corresponding to each of the multiple description signals; determining, based on the signal to be coded and a fused quantization signal, a final quantization signal corresponding to each of the multiple description signals from the multiple first candidate quantization signals, wherein the fused quantization signal is determined based on the first candidate quantization signals corresponding to different multiple description signals; and generating a code stream based on the final quantization signal corresponding to each of the multiple description signals.

[0009] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0010] In some embodiments, the final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

[0011] In some embodiments, determining multiple first candidate quantization signals corresponding to each of multiple multi-description signals includes: inserting a current pulse at each of multiple frequency point positions of each multi-description signal to obtain multiple second candidate quantization signals corresponding to the current pulse; and using a first objective function to determine the multiple second candidate quantization signals having the greatest first correlation with the frequency spectrum of the signal to be encoded as the multiple first candidate quantization signals corresponding to the current pulse.

[0012] In some embodiments, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the frequency spectra of the second candidate quantized signal and the signal to be encoded.

[0013] In some embodiments, the value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, and the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

[0014] In some embodiments, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0015] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.

[0016] According to other embodiments of the present disclosure, a decoding method is provided, comprising: receiving at least one multiple-description codestream, generating the at least one multiple-description codestream based on a final quantization signal corresponding to each of multiple description signals of a signal, determining the final quantization signal from multiple first candidate quantization signals corresponding to each of the multiple description signals based on the signal and a fused quantization signal, determining the fused quantization signal based on first candidate quantization signals corresponding to different multiple description signals, and determining the multiple first candidate quantization signals by performing quantization processing on the multiple multiple description signals; and decoding the at least one multiple-description codestream to obtain a signal.

[0017] In some embodiments, the at least one multiple description codestream includes multiple multiple description codestreams, and decoding the at least one multiple description codestream to obtain the signal includes: decoding each of the multiple description codestreams to obtain multiple sub-signals; and obtaining the signal based on the multiple sub-signals.

[0018] In some embodiments, obtaining the signal according to the multiple sub-signals includes normalizing a weighted average of the multiple sub-signals to obtain the signal.

[0019] In some embodiments, the at least one multiple description codestream includes a multiple description codestream, and decoding the at least one multiple description codestream to obtain a signal includes decoding the multiple description codestream to obtain a signal.

[0020] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0021] In some embodiments, the final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

[0022] In some embodiments, multiple first candidate quantization signals are determined as follows: a current pulse is inserted at each of the multiple frequency positions of each multi-description signal to obtain multiple second candidate quantization signals corresponding to the current pulse; and a first objective function is used to determine the multiple second candidate quantization signals having the greatest first correlation with the frequency spectrum of the signal to be encoded as the multiple first candidate quantization signals corresponding to the current pulse.

[0023] In some embodiments, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the frequency spectra of the second candidate quantized signal and the signal to be encoded.

[0024] In some embodiments, the value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, and the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

[0025] In some embodiments, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0026] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.

[0027] According to further embodiments of the present disclosure, a coding apparatus is provided, comprising: an acquisition unit configured to obtain a plurality of multi-description signals based on a signal to be coded; a quantization unit configured to perform quantization processing on the plurality of multi-description signals to determine a plurality of first candidate quantization signals corresponding to each of the plurality of multi-description signals; a determination unit configured to determine, from the plurality of first candidate quantization signals, a final quantization signal corresponding to each of the plurality of multi-description signals based on the signal to be coded and a fused quantization signal, wherein the fused quantization signal is determined based on the first candidate quantization signals corresponding to different multi-description signals; and a generation unit configured to generate a code stream based on the final quantization signal corresponding to each of the plurality of multi-description signals.

[0028] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0029] In some embodiments, the final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

[0030] In some embodiments, the quantization unit inserts the current pulse at each of the multiple frequency positions of each multi-description signal to obtain multiple second candidate quantization signals corresponding to the current pulse, and uses the first objective function to determine the multiple second candidate quantization signals with the largest first correlation with the frequency spectrum of the signal to be encoded as the multiple first candidate quantization signals corresponding to the current pulse.

[0031] In some embodiments, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the frequency spectra of the second candidate quantized signal and the signal to be encoded.

[0032] In some embodiments, the value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, and the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

[0033] In some embodiments, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0034] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.

[0035] According to further embodiments of the present disclosure, a decoding apparatus is provided, comprising: a receiving unit, configured to receive at least one multiple-description codestream, wherein the at least one multiple-description codestream is generated based on a final quantization signal corresponding to each of multiple description signals of a signal, wherein the final quantization signal is determined from multiple first candidate quantization signals corresponding to each of the multiple description signals based on the signal and a fused quantization signal, wherein the fused quantization signal is determined based on first candidate quantization signals corresponding to different multiple description signals, wherein the multiple first candidate quantization signals are determined by performing quantization processing on the multiple multiple description signals; and a decoding unit, configured to decode the at least one multiple-description codestream to obtain a signal.

[0036] In some embodiments, the decoding unit decodes each of the multiple description code streams to obtain a plurality of sub-signals; and obtains the signal according to the plurality of sub-signals.

[0037] In some embodiments, the decoding unit normalizes a weighted average of the plurality of sub-signals to obtain the signal.

[0038] In some embodiments, the at least one multiple description code stream includes a multiple description code stream, and the decoding unit decodes the multiple description code stream to obtain a signal.

[0039] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0040] In some embodiments, the final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

[0041] In some embodiments, multiple first candidate quantization signals are determined as follows: a current pulse is inserted at each of the multiple frequency positions of each multi-description signal to obtain multiple second candidate quantization signals corresponding to the current pulse; and a first objective function is used to determine the multiple second candidate quantization signals having the greatest first correlation with the frequency spectrum of the signal to be encoded as the multiple first candidate quantization signals corresponding to the current pulse.

[0042] In some embodiments, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the frequency spectra of the second candidate quantized signal and the signal to be encoded.

[0043] In some embodiments, the value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, and the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

[0044] In some embodiments, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0045] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.

[0046] According to some further embodiments of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the encoding method or decoding method in any one of the above embodiments based on instructions stored in the memory device.

[0047] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the encoding method or decoding method in any of the above embodiments is implemented.

[0048] According to some further embodiments of the present disclosure, a transmission system is provided, comprising: an encoding device for executing the encoding method in any one of the above embodiments; and a decoding device for executing the decoding method in any one of the above embodiments.

[0049] According to some further embodiments of the present disclosure, a computer program product is provided, comprising instructions, which, when executed by a processor, enable the processor to perform the encoding method or decoding method in any one of the above embodiments.

[0050] In the above embodiment, multiple multi-description signals are used to generate multiple final quantized signals for generating a code stream of the signal to be encoded. In this way, the multi-description signals can be used to effectively combat packet loss, thereby improving the anti-packet loss capability and thus improving audio quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0052] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings:

[0053] FIG1 shows a flowchart of some embodiments of the encoding method of the present disclosure;

[0054] FIG2 a shows a schematic diagram of some embodiments of the encoding method of the present disclosure;

[0055] FIG2 b shows a flowchart of some other embodiments of the encoding method disclosed herein;

[0056] FIG3 shows a flowchart of some embodiments of the decoding method of the present disclosure;

[0057] FIG4a shows a block diagram of some embodiments of the encoding device of the present disclosure;

[0058] FIG4 b shows a block diagram of some embodiments of a decoding device of the present disclosure;

[0059] FIG5 shows a block diagram of some embodiments of an electronic device of the present disclosure;

[0060] FIG6 shows a block diagram of another embodiment of the electronic device of the present disclosure;

[0061] FIG7 shows a block diagram of some other embodiments of the transmission system of the present disclosure. DETAILED DESCRIPTION

[0062] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0063] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0064] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0065] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0066] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0067] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0068] As mentioned earlier, the Opus music coding mode, also known as the CELT (Constrained Energy Lapped Transform) coding mode, does not have any other anti-packet loss measures besides the PLC packet loss compensation algorithm. This results in poor audio quality when restored through packet loss compensation.

[0069] To address the audio quality issues over weak networks, out-of-band FEC (Forward Error Correction) or active retransmission can be used to improve packet loss resistance. However, this increases bandwidth overhead and end-to-end latency, resulting in low transmission efficiency and poor audio quality.

[0070] To address the above technical issues, the present disclosure proposes a multi-description quantization technical solution to improve the anti-packet loss capability, thereby improving audio quality. For example, the technical solution of the present disclosure can be implemented through the following embodiments.

[0071] FIG1 shows a flowchart of some embodiments of the encoding method of the present disclosure.

[0072] As shown in FIG1 , in step 110 , a plurality of multi-description signals are obtained according to the signal to be encoded.

[0073] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.For example, a multiple description code stream may be generated by the embodiment in FIG2a.

[0074] FIG2 a shows a schematic diagram of some embodiments of the encoding method of the present disclosure.

[0075] As shown in FIG2a , the processing flow before bit allocation of the input signal to be encoded and the bit rate, etc. can be consistent with that of CELT.

[0076] For example, the number of bits b to be allocated may be divided into two, such as two b / 2. In step 201a, bit allocation is performed once using the number of bits b / 2.

[0077] In step 202a, energy fine quantization is performed according to the number of fine quantization bits for each frequency band.

[0078] In step 203a, two multiple description code streams md1_stream and md2_stream are defined, and the side information generated in the previous step is encoded into md1_stream1.

[0079] In step 204a, the md1_stream1 data is copied to md2_stream.

[0080] For example, the above bit allocation result can be applied to two multiple description code streams respectively to ensure that the number of bits in each frequency band of the two multiple description code streams is consistent.

[0081] In step 205a, multi-description PVQ (Pyramid Vector Quantization) spectrum quantization is performed using the encoding method in any embodiment according to the normalized spectrum X and the number of spectrum quantization bits of each frequency band.

[0082] In this way, since the number of bits of the two multiple description code streams in each frequency band is the same when performing spectrum quantization, the number of pulses K of the two multiple description code streams after conversion also remains consistent.

[0083] In step 206a, energy residual quantization is performed to output processed md1_stream and md2_stream.

[0084] After multiple multiple description code streams are generated in step 204a, quantization, searching, encoding, etc. may be continued through steps 120 to 140 in FIG. 1 .

[0085] In step 120 , a quantization process is performed on the multiple multi-description signals to determine a plurality of first candidate quantized signals corresponding to each of the multiple description signals.

[0086] For example, the difference between the signals before and after quantization can be determined by using indicators such as the correlation coefficient of the signals before and after quantization or MSE (Mean Squared Error).

[0087] For example, the length of the signal's normalized spectrum X to be quantized is N, and the number of pulses is K. Initial quantization is performed based on K pulses to calculate the initial vector y0 and obtain the number of remaining pulses. The method used in Opus can be used to record the sign of each frequency point, take the absolute value of each frequency point, and project the spectrum onto the K pulses. The initial vector can be calculated using the following formula: y0 = truncate_towards_zero((K-1)×X / sum(abs(X)))

[0088] For example, the number of pulses used by the initial vector can be subtracted from the total number of pulses K to obtain the remaining number of pulses K_left; and the multiple multi-description signals can be further quantized based on the K_left remaining pulses.

[0089] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0090] For example, the K_left remaining pulses may be traversed, and for each remaining pulse, the value of the first objective function when the pulse is added to each position may be calculated, and M positions where the value of the first objective function is minimized may be found, where M is an integer greater than 1.

[0091] In some embodiments, the current pulse is inserted into each of the multiple frequency locations of each multi-description signal to obtain multiple second candidate quantized signals corresponding to the current pulse; and using a first objective function, the multiple second candidate quantized signals having the greatest first correlation with the spectrum of the signal to be encoded are determined as the multiple first candidate quantized signals corresponding to the current pulse. For example, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the spectrum of the second candidate quantized signal and the spectrum of the signal to be encoded.

[0092] For example, each pulse may be inserted into a corresponding frequency point position, or multiple pulses may be inserted into corresponding frequency point positions at the same time.

[0093] For example, the first objective function J1 can be calculated by the following formula: J1 = -X×y / ||y||

[0094] The smaller J1 is, the greater the first correlation between the second candidate quantized signal y and the spectrum X of the signal to be encoded is, and the smaller the difference is.

[0095] For example, by using the first objective function, M second candidate quantized signals y_md1 of the multiple description signal md1_stream and M second candidate quantized signals y_md2 of the multiple description signal md2_stream can be determined for M pulse insertion positions.

[0096] In this way, for each multi-description signal, multiple second candidate quantized signals that minimize the difference between the signals before and after quantization can be cached, thereby improving the quantization effect and thus improving the quality of the multi-description code stream.

[0097] In step 130, a final quantized signal corresponding to each multi-description signal is determined from a plurality of first candidate quantized signals according to the signal to be encoded and the fused quantized signal. The fused quantized signal is determined according to the first candidate quantized signals corresponding to different multi-description signals.

[0098] For example, M y_md1 and M y_md2 may be fused one by one to generate M×M fused quantized signals. For example, the fused quantized signal may be (y_md1+y_md2) / 2.

[0099] In some embodiments, the final quantized signal is determined based on a value of a second objective function, where the second objective function represents the difference between the signal to be encoded and the fused quantized signal. For example, the difference between the signal to be encoded and the fused quantized signal can be determined using an indicator such as a correlation coefficient or MSE.

[0100] For example, the value of the second objective function is determined based on the second correlation between the fused quantization signal corresponding to each candidate quantization signal combination in multiple candidate quantization signal combinations and the spectrum of the signal to be encoded, and the multiple candidate quantization signal combinations are generated based on multiple first candidate quantization signals corresponding to different multi-description signals.

[0101] For example, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0102] For example, the value of the second objective function J2 is determined according to the correlation coefficient between the spectrum X of the signal to be encoded and the fused quantized signal (y_md1+y_md2) / 2:

[0103] R in the formula ab Represents the correlation coefficient between a and b. With the minimum value of J2 as the constraint, a set of y_md1 and y_md2 combinations (corresponding to the pulse insertion positions) can be searched out as the final quantized signals of the multi-description signals md1_stream and md2_stream respectively.

[0104] In this way, by jointly searching multiple multi-description signals, multiple final quantized signals that minimize the difference between the signals before and after quantization can be determined again from multiple second candidate quantized signals, thereby improving the quantization effect and the quality of the multi-description code stream.

[0105] In step 140, a code stream is generated based on the final quantized signal corresponding to each multiple description signal. For example, y_md1 and y_md2 can be transmitted as two independent multiple description code streams, or y_md1 and y_md2 can be processed into one multiple description code stream for transmission.

[0106] In this way, multiple multi-description signals are used to generate multiple final quantized signals, which are used to generate the code stream of the signal to be encoded. In this way, the multi-description signals can be used to effectively combat packet loss, thereby improving the anti-packet loss capability and thus improving audio quality.

[0107] FIG2 b shows a flowchart of other embodiments of the encoding method of the present disclosure.

[0108] As shown in Figure 2b, the length of the normalized spectrum X of the signal to be quantized is N, and the number of pulses in a single stream is K. In step 201b, the sign of each frequency point is recorded, and the absolute value of each frequency point is taken.

[0109] In step 202b, initial quantization is performed based on K pulses to calculate the initial vector y0 and obtain the remaining number of pulses. For example, the spectrum can be projected onto K pulses, and the number of pulses used in the initial vector can be subtracted from the total number of pulses K to obtain the remaining number of pulses K_left.

[0110] In steps 203b and 204b, each remaining pulse is traversed for searching. For example, the currently searched pulse k can be initialized to 0, and the initial quantized spectrum y_md1 = y_md2 = y. A determination is then made as to whether k is less than K_left. If not, then all remaining pulses have been traversed, and step 205b is executed. If so, then not all remaining pulses have been traversed, and step 206b is executed.

[0111] In step 205b, the frequency bin symbols are applied to the quantized vectors and the quantized spectrum is output.

[0112] In step 206b, a search is performed with the goal of minimizing the correlation between the quantized spectrum y and the input spectrum X.

[0113] For example, the first objective function J1 in any of the above embodiments can be constructed; for the kth iteration, each position of the quantized spectrum y is traversed, a pulse is added to the corresponding position, and M frequency point positions pos that minimize the value of J1 are found.

[0114] Through the above steps, M candidate positions are determined for y_md1 and y_md2 respectively. Next, with the maximum correlation coefficient between X and (y_md1+y_md2) / 2 as the constraint, y_md1 and y_md2 are jointly searched among M×M pulse insertion positions to obtain a pair of pulse insertion positions.

[0115] In step 207b, i is initialized to 0, the optimal insertion position of y_md1 is p = 0, and the optimal insertion position of y_md2 is q = 0. i is the current position among the M candidate positions of y_md1.

[0116] In step 208b, it is determined whether i is less than M; if not, step 209b is executed to enter the search for the next pulse; if yes, step 210b is executed.

[0117] In step 209b, a pulse y_md1 and y_md2 are inserted at the positions of p and q respectively, and the value of k is updated to k+1; and the process returns to step 204b.

[0118] In step 210b, a pulse is added at the pos[i]th position of y_md1.

[0119] In step 211b, j=0 is initialized, where j is the current position among the M candidate positions of y_md2.

[0120] In step 212b, it is determined whether j is less than M; if not, step 213b is executed to search for the next candidate position of y_md1; if yes, step 214b is executed.

[0121] In step 213b, the value of i is updated to i+1 to search for the next candidate position of y_md1.

[0122] In step 214b, a pulse is added at the pos[j]th position of y_md2.

[0123] In step 215b, the values ​​of the correlation coefficients in the formula of the second objective function J2 are calculated, and the current value of J2 is determined.

[0124] In step 216, it is determined whether the current value of J2 reaches the minimum; if so, step 217b is executed; if not, step 218b is executed.

[0125] In step 217 , the optimal added pulse position p=pos[i] of y_md1 and the optimal added pulse position q=pos[j] of y_md2 are updated.

[0126] In step 218b, the value of j is updated to j+1 to search for the next candidate position of y_md2.

[0127] In the above embodiment, using J1 and J2 as objective functions, a single search for a multi-description signal and a joint search for multiple multi-description signals were performed, determining the optimal quantization signal for each multi-description signal. This allows the use of multi-description signals to effectively combat packet loss, improving packet loss resilience and, consequently, audio quality.

[0128] FIG3 shows a flowchart of some embodiments of the decoding method of the present disclosure.

[0129] As shown in Figure 3, in step 310, at least one multiple-description codestream is received. The at least one multiple-description codestream is generated based on a final quantized signal corresponding to each of multiple description signals of a signal. The final quantized signal is determined from multiple first candidate quantized signals corresponding to each of the multiple description signals based on the signal and the fused quantized signal. The fused quantized signal is determined based on the first candidate quantized signals corresponding to different multiple description signals. The multiple first candidate quantized signals are determined by quantizing the multiple description signals.

[0130] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0131] In some embodiments, the final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

[0132] In some embodiments, multiple first candidate quantization signals are determined as follows: a current pulse is inserted at each of the multiple frequency positions of each multi-description signal to obtain multiple second candidate quantization signals corresponding to the current pulse; and a first objective function is used to determine the multiple second candidate quantization signals having the greatest first correlation with the frequency spectrum of the signal to be encoded as the multiple first candidate quantization signals corresponding to the current pulse.

[0133] In some embodiments, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the frequency spectra of the second candidate quantized signal and the signal to be encoded.

[0134] In some embodiments, the value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, and the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

[0135] In some embodiments, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0136] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.

[0137] In step 320, at least one multiple description code stream is decoded to obtain a signal.

[0138] In some embodiments, the at least one multiple description codestream includes multiple multiple description codestreams. Each of the multiple description codestreams is decoded to obtain multiple sub-signals. The signal is obtained based on the multiple sub-signals. For example, a weighted average of the multiple sub-signals is normalized to obtain the signal.

[0139] For example, for each frequency band, quantization indices are decoded from two multiple description code streams and converted into normalized spectra y_md1 and y_md2; the two normalized spectra are fused to obtain y = (y_md1 + y_md2) / 2; the fused spectrum is renormalized to obtain an output spectrum y = y / ||y||.

[0140] In some embodiments, the at least one multiple description code stream includes a multiple description code stream, and the multiple description code stream is decoded to obtain a signal.

[0141] FIG4 a shows a block diagram of some embodiments of the encoding apparatus of the present disclosure.

[0142] As shown in FIG4a , the encoding device 4a includes: an acquisition unit 41a for obtaining a plurality of multi-description signals based on a signal to be encoded; a quantization unit 42a for performing quantization processing on the plurality of multi-description signals to determine a plurality of first candidate quantization signals corresponding to each of the plurality of multi-description signals; a determination unit 43a for determining a final quantization signal corresponding to each of the plurality of multi-description signals from the plurality of first candidate quantization signals based on the signal to be encoded and the fused quantization signal, wherein the fused quantization signal is determined based on the first candidate quantization signals corresponding to different multi-description signals; and a generation unit 44a for generating a code stream based on the final quantization signal corresponding to each of the plurality of multi-description signals.

[0143] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0144] In some embodiments, the final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

[0145] In some embodiments, the quantization unit 42a inserts the current pulse at each of the multiple frequency positions of each multi-description signal to obtain multiple second candidate quantization signals corresponding to the current pulse, and uses the first objective function to determine the multiple second candidate quantization signals with the largest first correlation with the frequency spectrum of the signal to be encoded as the multiple first candidate quantization signals corresponding to the current pulse.

[0146] In some embodiments, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the frequency spectra of the second candidate quantized signal and the signal to be encoded.

[0147] In some embodiments, the value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, and the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

[0148] In some embodiments, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0149] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.

[0150] FIG4 b shows a block diagram of some embodiments of a decoding device of the present disclosure.

[0151] As shown in FIG4b , the decoding device 4b includes: a receiving unit 41b for receiving at least one multiple-description codestream, where the at least one multiple-description codestream is generated based on a final quantization signal corresponding to each of multiple description signals of a signal, where the final quantization signal is determined from multiple first candidate quantization signals corresponding to each multiple description signal based on the signal and a fused quantization signal, where the fused quantization signal is determined based on first candidate quantization signals corresponding to different multiple description signals, where the multiple first candidate quantization signals are determined by performing quantization processing on the multiple multiple description signals; and a decoding unit 42b for decoding the at least one multiple-description codestream to obtain a signal.

[0152] In some embodiments, the decoding unit 42b decodes each of the multiple description code streams to obtain multiple sub-signals; and obtains a signal based on the multiple sub-signals.

[0153] In some embodiments, the decoding unit 42b normalizes the weighted average of the multiple sub-signals to obtain a signal.

[0154] In some embodiments, the at least one multiple description code stream includes a multiple description code stream, and the decoding unit 42b decodes the multiple description code stream to obtain a signal.

[0155] In some embodiments, the plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after quantization processing.

[0156] In some embodiments, the final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

[0157] In some embodiments, multiple first candidate quantization signals are determined as follows: a current pulse is inserted at each of the multiple frequency positions of each multi-description signal to obtain multiple second candidate quantization signals corresponding to the current pulse; and a first objective function is used to determine the multiple second candidate quantization signals having the greatest first correlation with the frequency spectrum of the signal to be encoded as the multiple first candidate quantization signals corresponding to the current pulse.

[0158] In some embodiments, the value of the first objective function corresponding to the second candidate quantized signal is calculated based on the frequency spectra of the second candidate quantized signal and the signal to be encoded.

[0159] In some embodiments, the value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, and the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

[0160] In some embodiments, the value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

[0161] In some embodiments, different multiple description signals in the plurality of multiple description signals have the same number of bits allocated on the same frequency band.

[0162] FIG5 shows a block diagram of some embodiments of an electronic device of the present disclosure.

[0163] As shown in FIG5 , the electronic device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51 , and the processor 52 is configured to execute the encoding method or decoding method in any one embodiment of the present disclosure based on instructions stored in the memory 51 .

[0164] The memory 51 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, application programs, a boot loader, a database, and other programs.

[0165] FIG6 shows a block diagram of some other embodiments of the electronic device of the present disclosure.

[0166] As shown in FIG6 , the electronic device 6 of this embodiment includes a memory 610 and a processor 620 coupled to the memory 610 . The processor 620 is configured to execute the encoding method or decoding method in any one of the aforementioned embodiments based on instructions stored in the memory 610 .

[0167] The memory 610 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs.

[0168] The electronic device 6 may further include an input / output interface 630, a network interface 640, a storage interface 650, and the like. These interfaces 630, 640, 650, as well as the memory 610 and the processor 620, may be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a display, mouse, keyboard, touch screen, microphone, and speakers. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0169] FIG7 shows a block diagram of some other embodiments of the transmission system of the present disclosure.

[0170] As shown in FIG7 , the transmission system 7 includes: an encoding device 71 for executing the encoding method in any one of the above embodiments; and a decoding device 72 for executing the decoding method in any one of the above embodiments.

[0171] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media, including but not limited to magnetic disk storage, CD-ROMs, optical storage, and the like, containing computer-usable program code.

[0172] The encoding method, encoding device, decoding method, decoding device, transmission system, electronic device, computer-readable storage medium, and computer program product according to the present disclosure have been described in detail. To avoid obscuring the concepts of the present disclosure, some details well known in the art have been omitted. Based on the above description, those skilled in the art will fully understand how to implement the technical solutions disclosed herein.

[0173] The methods and systems of the present disclosure may be implemented in many ways. For example, the methods and systems of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Therefore, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0174] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that modifications may be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A coding method comprising: According to the signal to be encoded, a plurality of multi-description signals are obtained; performing quantization processing on the multiple multi-description signals to determine multiple first candidate quantized signals corresponding to each of the multiple description signals; Determining, according to the signal to be encoded and the fused quantized signal, a final quantized signal corresponding to each of the multiple-description signals from the multiple first candidate quantized signals, wherein the fused quantized signal is determined according to the first candidate quantized signals corresponding to different multiple-description signals; A code stream is generated according to the final quantized signal corresponding to each of the multiple description signals.

2. The encoding method according to claim 1, wherein: The plurality of first candidate quantization signals are determined according to a value of a first objective function, where the first objective function represents a difference between the signals before and after the quantization process.

3. The encoding method according to claim 1, wherein: The final quantized signal is determined according to a value of a second objective function, where the second objective function represents a difference between the signal to be encoded and the fused quantized signal.

4. The encoding method according to claim 2, wherein: The determining of a plurality of first candidate quantized signals corresponding to each of the plurality of multi-description signals comprises: Inserting the current pulse at each of the multiple frequency positions of each multi-description signal to obtain multiple second candidate quantized signals corresponding to the current pulse; The first objective function is used to determine a plurality of second candidate quantization signals having the greatest first correlation with the frequency spectrum of the signal to be encoded as the plurality of first candidate quantization signals corresponding to the current pulse.

5. The encoding method according to claim 4, wherein: The value of the first objective function corresponding to the second candidate quantized signal is calculated according to the frequency spectra of the second candidate quantized signal and the signal to be encoded.

6. The encoding method according to claim 3, wherein: The value of the second objective function is determined based on a second correlation between a fused quantization signal corresponding to each candidate quantization signal combination in a plurality of candidate quantization signal combinations and a frequency spectrum of the signal to be encoded, wherein the plurality of candidate quantization signal combinations are generated based on a plurality of first candidate quantization signals corresponding to different multi-description signals.

7. The encoding method according to claim 6, wherein: The value of the second objective function is calculated based on the mutual correlation coefficient between multiple first candidate quantization signals in the candidate quantization signal set, the autocorrelation coefficient of each first candidate quantization signal in the multiple first candidate quantization signals, and the mutual correlation coefficient between each first candidate quantization signal and the frequency spectrum of the signal to be encoded.

8. The encoding method according to any one of claims 1 to 7, wherein: Different multiple description signals among the multiple description signals have the same number of bits allocated on the same frequency band.

9. A decoding method comprising: receiving at least one multi-description codestream, the at least one multi-description codestream being generated based on a final quantized signal corresponding to each of a plurality of multi-description signals of a signal, the final quantized signal being determined from a plurality of first candidate quantized signals corresponding to each of the multi-description signals based on the signal and a fused quantized signal, the fused quantized signal being determined based on first candidate quantized signals corresponding to different multi-description signals, the plurality of first candidate quantized signals being determined by performing quantization processing on the plurality of multi-description signals; The at least one multiple description code stream is decoded to obtain the signal.

10. The decoding method according to claim 9, wherein: The at least one multiple description code stream includes multiple multiple description code streams, The decoding of the at least one multiple description code stream to obtain the signal includes: Decoding each of the multiple description code streams to obtain a plurality of sub-signals; The signal is obtained according to the multiple sub-signals.

11. The decoding method according to claim 10, wherein: The obtaining the signal according to the multiple sub-signals includes: Normalizing the weighted average of the multiple sub-signals to obtain the signal.

12. The decoding method according to claim 9, wherein: The at least one multiple description code stream comprises a multiple description code stream, The decoding of the at least one multiple description code stream to obtain the signal includes: The multiple description code stream is decoded to obtain the signal.

13. An encoding device comprising: an acquisition unit, configured to obtain a plurality of multi-description signals according to a signal to be encoded; a quantization unit, configured to perform quantization processing on the multiple multi-description signals to determine a plurality of first candidate quantization signals corresponding to each of the multiple description signals; a determining unit, configured to determine, from the plurality of first candidate quantized signals, a final quantized signal corresponding to each of the multiple-description signals based on the signal to be encoded and the fused quantized signal, wherein the fused quantized signal is determined based on the first candidate quantized signals corresponding to different multiple-description signals; The generating unit is configured to generate a code stream according to the final quantized signal corresponding to each of the multiple description signals.

14. A decoding device comprising: a receiving unit, configured to receive at least one multi-description codestream, the at least one multi-description codestream being generated based on a final quantization signal corresponding to each of a plurality of multi-description signals of a signal, the final quantization signal being determined from a plurality of first candidate quantization signals corresponding to each of the multi-description signals based on the signal and a fused quantization signal, the fused quantization signal being determined based on first candidate quantization signals corresponding to different multi-description signals, the plurality of first candidate quantization signals being determined by performing quantization processing on the plurality of multi-description signals; A decoding unit is configured to decode the at least one multiple description code stream to obtain the signal.

15. An electronic device comprising: Memory; and A processor coupled to the memory, the processor being configured to execute the encoding method according to any one of claims 1 to 8 or the decoding method according to any one of claims 9 to 12 based on instructions stored in the memory.

16. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the encoding method according to any one of claims 1 to 8 or the decoding method according to any one of claims 9 to 12 is implemented.

17. A transmission system comprising: An encoding device, configured to execute the encoding method according to any one of claims 1 to 8; A decoding device, configured to execute the decoding method according to any one of claims 9 to 12.

18. A computer program product comprising instructions, which, when executed by a processor, cause the processor to perform the encoding method according to any one of claims 1 to 8, or the decoding method according to any one of claims 9 to 12.

Images