A speech encoding method, a speech decoding method and a speech processing device

Abstract

The present application discloses a speech encoding method, a speech decoding method and a speech processing device. The speech encoding method is applied to a speech processing device, and the speech processing device includes a coupled general processor and a logical processor; the general processor and the logical processor cooperate to encode at least one first speech data to obtain first encoded data, wherein the general processor is used to execute the non-cyclic calculation part of the encoding process, and the logical processor is used to execute the cyclic calculation part of the encoding process. In the above manner, the present application can improve the efficiency of speech encoding and decoding.

Description

Technical Field

[0001] The present application relates to the field of speech technology, and in particular to a speech encoding method, a speech decoding method and a speech processing device. Background Art

[0002] With the rapid development of communication, computer network and other technologies, speech coding and decoding technology has been developed and widely used. Especially in recent decades, speech coding and decoding technology has been widely used in mobile communication, satellite communication, multimedia technology and other fields. Speech is the most important, effective, commonly used and convenient form of information exchange for human beings. It is the main way for people to communicate their thoughts and emotions. In actual voice calls, some channels are difficult to expand and have poor communication quality; some channels are widely used and difficult to update in the short term; for some expensive channels, every bit compressed means saving money. Therefore, speech coding and decoding undoubtedly occupies a pivotal position in voice communication and human information exchange. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a speech encoding method, a speech decoding method and a speech processing device, which can improve the efficiency of speech encoding and decoding.

[0004] To solve the above technical problems, the first aspect of the present application provides a speech encoding method, which is applied to a speech processing device, wherein the speech processing device includes a coupled general-purpose processor and a logical processor; the general-purpose processor and the logical processor cooperate to encode at least one first speech data to obtain first encoded data, wherein the general-purpose processor is used to execute the non-cyclic calculation part of the encoding process, and the logical processor is used to execute the cyclic calculation part of the encoding process.

[0005] To solve the above technical problems, the second aspect of the present application provides a speech decoding method, which is applied to a speech processing device, wherein the speech processing device includes a coupled general-purpose processor and a logical processor; the general-purpose processor and the logical processor cooperate to decode at least one second encoded data to obtain second speech data, wherein the general-purpose processor is used to execute the non-cyclic calculation part of the decoding process, and the logical processor is used to execute the cyclic calculation part of the decoding process.

[0006] To solve the above technical problems, the third aspect of the present application provides a speech processing device, which includes a coupled general-purpose processor, a logical processor and a memory; the general-purpose processor and the logical processor are used to collaboratively encode at least one first speech data to obtain first encoded data, and / or decode at least one second encoded data to obtain second speech data, wherein the general-purpose processor is used to execute the non-cyclic calculation part of the encoding and / or decoding process, and the logical processor is used to execute the cyclic calculation part of the encoding and / or decoding process; the memory is used to store the instructions and data required for the operation of the general-purpose processor and the logical processor.

[0007] The beneficial effect of the present application is as follows: different from the prior art, the present application utilizes a coupled general-purpose processor and a logical processor to collaboratively perform encoding and / or decoding processing, wherein the logical processor can provide powerful computing power, and the general-purpose processor can provide an efficient and convenient development method, thereby, the cyclic calculation part of the speech encoding and decoding algorithm that occupies more kernel resources and takes a longer time can be executed in the logical processor, and the remaining non-cyclic calculation part is executed by the general-purpose processor, thereby improving the efficiency of speech encoding and decoding, and further realizing multi-channel parallel speech encoding and decoding, so that resources are fully utilized, and hardware costs are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in this application, the following briefly introduces the drawings required in the description of the embodiments. Obviously, the drawings described below are only some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a structural schematic block diagram of an embodiment of a speech processing device of the present application;

[0010] Figure 2 is a schematic block diagram of the structure of another embodiment of the speech processing device of the present application;

[0011] Figure 3 yes Figure 2 A structural schematic block diagram of an embodiment of a voice processing circuit;

[0012] Figure 4 It is a structural schematic block diagram of another embodiment of the speech processing device of the present application;

[0013] Figure 5 is a schematic block diagram of the structure of another embodiment of the speech processing device of the present application;

[0014] Figure 6 is a schematic block diagram of the structure of a speech processing device corresponding to the encoder;

[0015] Figure 7 is a schematic block diagram of the structure of a speech processing device corresponding to the decoder;

[0016] Figure 8 It is a flowchart of an embodiment of a speech coding method of the present application;

[0017] Fig. 9 yes Figure 8 A schematic flow chart of another implementation of step S11;

[0018] Fig.10 yes Fig. 9 A schematic flow chart of another implementation of step S21;

[0019] Fig.11 It is a schematic diagram of the format of encoding part of the general processor to transfer data to the logical processor;

[0020] Fig.12 yes Fig.11 A schematic flow chart of another implementation of step S212;

[0021] Fig.13 yes Fig.11 A schematic flow chart of another implementation of step S214;

[0022] Fig.14 It is a schematic diagram of the format of encoding part of the logical processor to pass data to the general processor;

[0023] Fig.15 It is a flowchart of an embodiment of a speech decoding method of the present application;

[0024] Fig.16 yes Fig.15 A schematic flow chart of another implementation of step S31;

[0025] Fig.17 yes Fig.16 A schematic flow chart of another implementation of step S41;

[0026] Fig.18 It is a schematic diagram of the format for decoding part of the general-purpose processor to pass data to the logical processor;

[0027] Fig.19 yes Fig.17 A schematic flow chart of another implementation of step S412;

[0028] Fig. 20 yes Fig.17 A schematic flow chart of another implementation of step S414;

[0029] Fig.21It is a schematic diagram of the format of decoding part of the logical processor to pass data to the general processor;

[0030] Fig. 22 It is a comparison diagram of the first voice data and the second voice data in the time domain and frequency domain. DETAILED DESCRIPTION

[0031] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] The terms "first" and "second" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0033] Speech coding and decoding is to digitize speech signals, and then transmit, store or process them after digitization, which can make full use of various digital signal processing technologies. In order to reduce storage space or reduce transmission bit rate to save bandwidth, it is also necessary to codec the digitized speech signals.

[0034] The main organizations that study and formulate mobile audio standards in the world include the International Telecommunications Union (ITU), the International Organization for Standardization (ISO), the 3rd Generation Partnership Project (3GPP), the European Telecommunications Standards Institute (ETSI), etc. ITU has released a series of speech codec standards, including narrowband speech coding standards such as G711, G721, G723, G726, G727, G728, G729, and G723.1; and wideband speech codec standards such as G722, G722.1, G722.2, and G729.1. These G series mobile speech codec standards can meet the application requirements of different types of mobile voice communications.

[0035] At present, the implementation schemes of voice codec technology are mainly divided into external codec and built-in software codec. The external codec scheme is to implement the voice compression algorithm with a dedicated chip and apply it to the project in the form of components; the built-in software codec scheme is to implement the voice codec algorithm based on a general microcontroller platform (MCU platform).

[0036] Among them, the chip used in the external codec is a dedicated chip that integrates acquisition and compression, and uses a UART interface to transmit compressed voice data. Among them, the chip of the external codec uses analog signal processing, and the external interface is generally a UART serial port. The supported baud rate is low, and the transmission bandwidth cannot be improved, resulting in large distortion, slow rate, and low transmission bandwidth of the external codec; secondly, the current vocoder (codec) chip has a single function, supports few algorithms, and has low algorithm complexity, resulting in unsatisfactory results in actual applications, cannot be flexibly applied, and has high product replacement costs, resulting in high costs, poor flexibility, and poor scalability of external codecs; in addition, the external encoder cannot realize voice encoding and decoding at the same time, and cannot support more than 3 channels of voice encoding or decoding at the same time.

[0037] The built-in software codec is based on a general-purpose MCU platform to implement the corresponding voice compression algorithm. With the development of general-purpose MCUs, their performance, memory, data read and write speed, etc. have been greatly improved, making it possible to implement more complex voice compression algorithms and improving the performance of voice compression. However, general-purpose MCUs process based on instruction sets, making it impossible to implement multi-channel parallel voice compression algorithms; at the same time, due to the limitations of the embedded processor core frequency and computing power, a large number of operations in the voice compression algorithm are very time-consuming and have large delays; the voice compression algorithm occupies a high level of processor core resources during operation, making it impossible to effectively utilize the processor core, which greatly wastes resources. In addition, although the GPU contains a large number of matrix operation units, which can provide super computing power, its hardware cost is very high and cannot be used in embedded systems.

[0038] Goal: In the field of military equipment, the performance of domestically produced chips using external codec solutions cannot meet the requirements. The built-in software codec solution has high complexity in voice coding and decoding algorithms and high performance requirements for domestically produced embedded processors. It is difficult to achieve outstanding performance in actual applications. Therefore, existing products cannot meet the hard index requirements of military products.

[0039] Based on this, the present application provides a speech processing device, including a coupled general-purpose processor and a logical processor, the general-purpose processor and the logical processor cooperate to perform encoding and / or decoding processing, wherein the logical processor can provide powerful computing power, and the general-purpose processor can provide an efficient and convenient development method, thereby, the cyclic calculation part of the speech encoding and decoding algorithm that occupies more kernel resources and takes a long time can be executed in the logical processor, and the remaining non-cyclic calculation part is executed by the general-purpose processor, thereby improving the efficiency of speech encoding and decoding, and further realizing multi-channel (more than 2 channels) parallel speech encoding and decoding, so that resources are fully utilized, and hardware costs are reduced. Here, only some solutions and effects of the present application are briefly described.

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] See also Figure 1 , Figure 1 It is a structural schematic block diagram of an embodiment of a speech processing device of the present application.

[0042] like Figure 1As shown, in this embodiment, the speech processing device 100 may include a general processor 10 , a logic processor 20 , and a memory 30 that are coupled.

[0043] The general processor 10 and the logic processor 20 are used to cooperate to encode at least one first voice data to obtain first encoded data, and / or decode at least one second encoded data to obtain second voice data, wherein the general processor 10 is used to perform the non-cyclic calculation part of the encoding and / or decoding process, and the logic processor 20 is used to perform the cyclic calculation part of the encoding and / or decoding process. The memory 30 is used to store instructions and data required for the general processor 10 and the logic processor 20 to work.

[0044] Optionally, the general processor 10 may be a microprocessor, such as an ARM (Advanced RISC Machines) processor. In this embodiment, the general processor 10 is, for example, an ARM Cortex-A9 core, that is, an ARM processor with a dual-core A9 architecture.

[0045] The memory 30 may be, but is not limited to, a random access memory (RAM), such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), etc. In this embodiment, the memory 30 may be a DDR, i.e., a double data rate synchronous dynamic random access memory, which is one type of memory. DDR includes DDR1, DDR2, and DDR3, i.e., the first generation, second generation, and third generation double data rate synchronous dynamic random access memory, respectively.

[0046] The logic processor 20 may be, but is not limited to, a field programmable gate array (FPGA) and / or a digital signal processor (DSP). Among them, with the development of FPGA, especially the emergence of embedded soft cores or hard cores (such as embedded ARM cortex-A9 hard cores or micoblaze soft cores), its advantages have become more and more prominent, greatly improving the computing power and flexibility. In some embodiments, the FPGA may be XC7Z045 / FMQL45. After testing, the implementation of this solution based on XC7Z045 / FMQL45 can achieve a frame of voice data (10ms) with an encoding running time of 2.8ms and a decoding running time of 0.8ms, and can simultaneously execute 5-way encoding and 10-way decoding. The time consumption of multi-way voice encoding and decoding is almost the same as the operation time consumption of 1-way voice encoding and decoding. The specific number of execution paths is determined by the FPGA resources and can be flexibly configured.

[0047] XC7Z020CLG400 / FMQL20CLG400 are two other types of FPGA. When implementing the cyclic calculation part of the encoding process and the decoding process on XC7Z020CLG400 / FMQL20CLG400, the size of the core board can be controlled within 40mm*35mm, and the overall power consumption can be controlled within 5W, thus achieving the advantages of low cost, low power consumption and small size.

[0048] See also Figures 2 to 3 , Figure 2 is a schematic block diagram of the structure of another embodiment of the speech processing device of the present application, Figure 3 yes Figure 2 A structural schematic block diagram of an embodiment of a voice processing circuit in FIG.

[0049] like Figure 2 As shown, in some embodiments, the logic processor 20 may include at least one parallel voice processing circuit 21. Each voice processing circuit 21 is used to perform a loop calculation part in the encoding process of a first voice data channel, and / or a loop calculation part in the decoding process of a second encoded data channel. The voice processing circuit 21 can be multiplexed by the encoding part and the decoding part. For example, the voice processing circuit 21 can be used to perform a loop calculation part in the encoding process of a first voice data channel, and can also be used to perform a loop calculation part in the decoding process of a second encoded data channel.

[0050] like Figure 3As shown, in some embodiments, the speech processing circuit 21 includes a loop calculation unit 211, a cache unit 212, a lookup table unit 213 and a management configuration unit 214, and the loop calculation unit 211 is connected to the cache unit 212, the lookup table unit 213 and the management configuration unit 214. The loop calculation unit 211 is used to perform loop calculations and can be composed of a configurable logic block (CLB). The cache unit 212 is used to provide a cache space, and the cache space is mainly composed of an on-chip memory (such as a Block RAM) and a FIFO (First Input First Output, a first-in first-out queue), and the FIFO is essentially a dual-port Block RAM. Among them, the Block RAM is a storage block inside the chip. The lookup table (Look-Up-Table, referred to as LUT) unit is used to complete a purely combined logic function and to provide search data for the loop calculation unit 211. The management configuration unit 214 is used to manage and configure the speech processing circuit 21.

[0051] In some implementations, the cache unit 212 may include a temporary variable cache unit 212 and an associated variable cache unit 212, which are used to cache temporary variables and associated variables, respectively.

[0052] In some implementations, the lookup table unit 213 may be an on-chip memory (such as a Block RAM, OCM). After a user describes a logic circuit through a schematic diagram or HDL language, the PLD / FPGA development software will automatically calculate all possible results of the logic circuit and write the results into the Block RAM in advance. In this way, each input of a signal for a logic operation is equivalent to inputting an address for a table lookup, finding the content corresponding to the address, and then outputting it.

[0053] The general processor 10 and the logic processor 20 can be connected to the management configuration unit 214 in each voice processing circuit 21 through a bus, so as to complete the management and configuration of each voice processing circuit 21, wherein the encoding part mainly includes: configuring relevant address signals (such as the first data address for storing the first intermediate data and the second data address for storing the second intermediate data, that is, the address of the first voice data to be encoded and the storage address of the encoded parameters); giving control signals (such as encoding start information, reset signal, etc.); and reading relevant parameters (such as the current encoding status information, version information, etc.). The decoding part mainly includes: configuring relevant address signals (such as the third data address for storing the third intermediate data and the fourth data address for storing the fourth intermediate data, that is, the storage address of the synthesized voice data after decoding and the reading address of the parameters required for encoding); giving control signals (such as decoding start signal, reset signal, etc.); and reading relevant parameters (such as the current decoding status information, version information, etc.).

[0054] In some other embodiments, the speech processing circuit 21 further includes a cross-clock domain processing unit 215 connected to the cyclic calculation unit 211. The cross-clock domain processing unit 215 can be composed of registers and / or FIFOs, and the asynchronous clock domain can be synchronized after being registered to solve the problems caused by cross-clock domains.

[0055] Please continue reading Figure 4 , Figure 4 It is a structural schematic block diagram of another embodiment of the speech processing device of the present application.

[0056] like Figure 2 and 4 As shown, in some embodiments, the speech processing device 100 may include a bus 40, which is respectively connected to the general processor 10, the logical processor 20 and the memory 30, and the general processor 10, the logical processor 20 and the memory 30 transmit data through the bus 40. Correspondingly, the logical processor 20 may also include a bus interface 22, so that the speech processing circuit 21 is connected to the bus through the bus interface 22 to realize data transmission. Figure 6 and Figure 7 As shown, in one example, the bus 40 may be an AIX (Advanced eXtensible Interface) bus, and the bus interface 22 may include an AXI-Hp interface and an AXI-Lite interface, and data transmission is implemented through the read and write logic of the AXI-Hp interface and the AXI-Lite interface. The AXI-Hp interface can provide a maximum bandwidth of 9.6 Gbps, so there is no limitation on data transmission efficiency.

[0057] like Figure 4 As shown, in one example, when the logical processor 20 includes at least one voice processing circuit 21 (0-n), the logical processor 20 may further include a data distribution and aggregation circuit 23, which is respectively connected to the plurality of voice processing circuits 21 and the bus 40, and is used to aggregate the data output by the plurality of voice processing circuits 21 for transmission, and to distribute the entire data to the plurality of voice processing circuits 21, so as to realize parallel execution and improve the processing speed. In particular, an efficient scheduling, arbitration and caching mechanism may be designed in the data distribution and aggregation circuit 23, so that when data is exchanged between the general processor 10, the logical processor 20 and the memory 30, it can be completed efficiently and orderly, and multi-channel encoding and multi-channel decoding can be executed in parallel. In particular, the data distribution and aggregation circuit 23 may be connected to the bus 40 through the bus interface 22 (AXI-Hp interface and AXI-Lite interface) in the logical processor 20.

[0058] See also Figures 5 to 7 , Figure 5is a schematic block diagram of the structure of another embodiment of the speech processing device of the present application. Figure 6 is a schematic block diagram of the structure of a speech processing device corresponding to the encoder, Figure 7 It is a structural schematic block diagram of a speech processing device corresponding to a decoder.

[0059] like Figure 5 As shown, in some embodiments, the encoding part and the decoding part can be designed independently, that is, the logic processor 20 includes an encoder and a decoder (not marked in the figure), wherein the encoder is used to execute the loop calculation part in the encoding process, and the decoder is used to execute the loop calculation part in the decoding process.

[0060] like Figure 6 As shown, in the encoding part, the encoder may include at least one parallel voice processing circuit 21, denoted as the first voice processing circuit 0 to the first voice processing circuit n, and may also be denoted as (amr_cod_calc_0) to (amr_cod_calc_n), which are used to respectively execute the cyclic calculation part in the encoding process of one channel of the first voice data.

[0061] like Figure 7 As shown, in the decoding part, the decoder may include at least one parallel voice processing circuit 21, denoted as the second voice processing circuit 0 to the second voice processing circuit n, or also denoted as (amr_dec_calc_0) to (amr_dec_calc_n), which are used to respectively execute the cyclic calculation part in the decoding process of one channel of second encoded data.

[0062] In one application scenario, this solution can also be implemented based on the PSOC (Programmable System-On-Chip) platform, which integrates PS (Processing System) and PL (programmable logic) on a single chip, where the PS part is a general-purpose processor core (DualCortex-A9), and the PL part is FPGA logic. PS and PL exchange data through the AXI bus. In this way, the part of the speech encoding and decoding algorithm that occupies the most core processor resources and takes the longest processing time can be transplanted to the PL side for implementation, and the rest is still implemented on the PS side, which can greatly improve the efficiency of encoding and decoding, and can simultaneously realize multi-channel speech encoding and decoding, which is impossible for general-purpose processors. Optionally, the PSOC platform can be Xilinx's ZYNQ7000 platform.

[0063] For the military equipment field, the PSOC platform can adopt the Shanghai Fudan Microelectronics FMQL PSOC platform to replace the ZYNQ7000 platform. The pins of the two are 100% compatible and the performance is almost the same, thus meeting the localization requirements.

[0064] It has been verified that on the ZYNQ7000 / FMQL series device platform, for AMR-NB (Adaptive Multi Rate-Narrow Band Speech Codec) mode 0, multi-channel encoding and multi-channel decoding operations can be performed simultaneously. In the 1-channel encoding and decoding comparison test, the encoding operation time of one frame of this scheme is about 2.8ms, which is 4ms shorter than that of a pure general-purpose processor (such as Cortex A9 core) platform; the decoding operation time of one frame is about 0.8ms, which is 0.4ms shorter than that of a pure general-purpose processor platform. The general-purpose processor platform cannot realize the simultaneous operation of multi-channel voice encoding and decoding within the specified time, so there is no comparison test for multi-channel encoding and decoding. It has been verified that on the ZYNQ7045 platform, the completion time of one frame of voice data encoding and decoding is reduced by 55% overall compared with that on a pure ARM (Cortex A9).

[0065] In this embodiment, an implementation method based on the PSOC platform (ARM+FPGA architecture) is proposed, in which the main computing part of the encoding and decoding is implemented on the FPGA, which greatly improves the performance while reducing the hardware cost. It is called in the form of an IP core, which improves the flexibility, versatility and portability of the application.

[0066] See also Figure 8 , Figure 8 1 is a flow chart of an embodiment of the speech coding method of the present application. In this embodiment, the speech coding method is applied to the speech processing device in the above embodiment, and the speech processing device includes a coupled general processor and a logic processor.

[0067] The method may include the following steps:

[0068] Step S11: the general processor and the logical processor cooperate to encode at least one first voice data to obtain first encoded data, wherein the general processor is used to execute the non-cyclic calculation part of the encoding process, and the logical processor is used to execute the cyclic calculation part of the encoding process.

[0069] The first voice data is voice data in frames. The size of each frame of voice data can be set. For example, each frame can input 10ms of voice data, which is equivalent to 160 sampling points (i.e., 16K sampling rate), and the bit width of each point is 16 bits. Due to the different types of voice coding algorithms executed, the first coded data obtained also differs, so the specific content of the first coded data is not limited in this embodiment, for example, it can include coding parameters.

[0070] In the codec algorithm, no matter encoding or decoding, the part that occupies the most core processor resources and takes the longest processing time is the corresponding loop calculation part. Therefore, this embodiment implements the loop calculation part in the logical processor and implements the rest of the encoding and decoding in the general processor, which can achieve a huge performance improvement, greatly improve the encoding and decoding efficiency, and can simultaneously implement multi-channel voice encoding and decoding, reducing processing delays.

[0071] In some implementations, the general processor and the logic processor cooperate to execute the encoding part of the codec algorithm to encode at least one first voice data to obtain the first encoded data. Optionally, this embodiment does not limit the type of codec algorithm, for example, it can support the codec algorithm released by the International Telecommunication Union ITU, such as G.711, G.723 (low bit rate speech coding algorithm), G.726, G.729, ILBC (internet Low Bitrate Codec, Internet Low Bitrate Codec), QCELP (Qualcomm Code Excited Linear Prediction), EVRC (Enhanced Variable Rate Codec, Enhanced Variable Rate Codec), SMV (Selectable Mode Coder) and other algorithms. In one example, it can be an adaptive multi-rate codec algorithm (AMR), such as an adaptive multi-rate narrowband speech coding algorithm (AMR-NB), an adaptive multi-rate wideband speech coding algorithm (AMR-WB, AMR-WB+).

[0072] In this embodiment, a coupled general-purpose processor and a logical processor are used to collaboratively perform encoding and / or decoding processing, wherein the logical processor can provide powerful computing power, and the general-purpose processor can provide an efficient and convenient development method. Thus, the cyclic calculation part of the speech encoding and decoding algorithm that occupies more kernel resources and takes a longer time can be executed in the logical processor, and the remaining non-cyclic calculation part is executed by the general-purpose processor, thereby improving the efficiency of speech encoding and decoding, and further realizing multi-channel parallel speech encoding and decoding, so that resources are fully utilized and hardware costs are reduced.

[0073] See also Fig. 9 , Fig. 9 yes Figure 8FIG. 1 is a flow chart of another implementation of step S11 in FIG. 1 . In this embodiment, the AMR-NB encoding and decoding algorithm is taken as an example for description.

[0074] Step S21: the general processor and the logic processor cooperate to execute the encoding part of the adaptive multi-rate speech encoding algorithm to encode at least one first speech data to obtain first encoded data.

[0075] In this embodiment, when the coding algorithm is an adaptive multi-rate speech coding algorithm (AMR, including AMR-NB, AMR-WB, AMR-WB+), the main implementation function of AMR coding is to set / generate coding parameters, and the corresponding first coded data includes coding parameters, where the coding parameters may include but are not limited to: LSP (Language Server Protocol), an adaptive codebook and a fixed codebook quantization index (including address and gain).

[0076] Among them, AMR-NB is a speech compression coder standard based on the Code Excited Linear Prediction (CELP) algorithm formulated by the 3rd Generation Partnership Project (3GPP) in August 1999, mainly used in the third-generation mobile communication W-CDMA system. AMR-NB voice bandwidth range: 300-3400Hz, 8KHz sampling, support eight rate modes, namely: Mode 0 (4.75kbit / s), Mode 1 (5.15kbit / s), Mode 2 (5.90kbit / s), Mode 3 (6.70kbit / s), Mode 4 (7.40kbit / s), Mode 5 (7.95kbit / s), Mode 6 (10.2kbit / s), Mode 7 (12.2kbit / s), it solves the rate allocation problem of source and channel coding in a more intelligent way, and adaptively selects a coding mode for transmission according to the wireless channel and transmission conditions, making the configuration and utilization of wireless resources more flexible and effective.

[0077] The encoder for implementing AMR-NB uses the ACELP hybrid coding method, which is a process in which the digital voice signal includes both some voice feature parameters and some waveform coding information, and then uses these feature information to resynthesize the voice signal. The number of these parameters extracted is controlled, and the information is selected according to the rate requirements to obtain the following 8 rates, which are mixed to form an adaptive voice encoder. For example, mode AMR_12.20 extracts 244 bits of parameter information, while mode AMR_4.75 only extracts 95 bits of information. According to the amount of information contained in these bits, they can be divided into 3 categories of bits: class 0, 1, and 2. During channel coding, class 0 and 1 will use cyclic redundancy check codes for error detection, and class 2 will be restored based on the previous frame.

[0078] Please participate Figures 10 to 14 , Fig.10 yes Fig. 9 A schematic flow chart of another implementation of step S21 in FIG. Fig.11 It is a schematic diagram of the format of encoding part of the general processor to transfer data to the logical processor. Fig.12 yes Fig.11 A schematic flow chart of another implementation of step S212 in FIG. Fig.13 yes Fig.11 A flow chart of another implementation of step S214 is shown in FIG. Fig.14 It is a schematic diagram of the format in which the encoding part of the logical processor transfers data to the general processor.

[0079] In this embodiment, step S21 may further include steps S211 to S215.

[0080] Step S211: The general processor imports the first speech data, and analyzes and calculates the linear prediction coding parameters to obtain the first intermediate data.

[0081] In some implementations, the first intermediate data includes first speech data and linear predictive coding (LPC) parameters.

[0082] like Fig.11 As shown, the format of data transmitted from the general processor to the logic processor in the encoding part includes the first voice processing circuit (0-n), and the data transmission format corresponding to each first voice circuit is the first voice data and other parameters (A_t and Aq_t) arranged in sequence. The other parameters include encoding parameters (such as LPC).

[0083] In one example, the first speech data (speech[]) includes 160 sampling points; A_t and Aq_t are fixed parameters in the encoding, and the number is 44, including the encoding parameters. The number of A_t and Aq_t can vary according to the change of the encoding and decoding algorithm, and can also be selected according to the actual situation, which is not limited here.

[0084] Step S212: the general processor transmits the first intermediate data to the logical processor, and notifies the logical processor to process the first intermediate data.

[0085] Specifically, the general processor may send a first instruction to the logical processor to notify the logical processor to process the first intermediate data.

[0086] like Fig.12 As shown, in some implementations, in step S212 , the general processor transmitting the first intermediate data to the logical processor may include sub-steps S2121 to S2123 .

[0087] Step S2121: The general processor stores the first intermediate data into the memory according to the first data address.

[0088] Step S2122: the general processor notifies the logical processor to read the first intermediate data.

[0089] The general processor may send a second instruction to the logical processor to notify the logical processor to read the first intermediate data. Here, the second instruction and the first instruction may be the same instruction or different instructions.

[0090] Step S2123: the logical processor reads the first intermediate data from the memory according to the first data address.

[0091] Since the logical processor has configured the first data address based on the management configuration unit, the logical processor can directly obtain the first data address, and then read the first intermediate data from the memory through the bus based on the first data address.

[0092] Step S213: The logic processor cyclically calculates the pitch and innovation parameters of the first intermediate data, and updates the associated data to obtain the second intermediate data.

[0093] Specifically, the first speech data can be divided into a preset number of subframe data (numbered as subframe0-n), and then the logic processor is used to calculate the analysis parameters (including pitch and innovation parameters) of the preset number of subframe data respectively, that is, the calculation is looped for a preset number of times, and after the calculation is completed, the associated data is updated to obtain the second intermediate data.

[0094] In one embodiment, the preset number is 4, and each frame of speech data (including 160 sampling points) can be divided into 4 subframes (numbered as subframe0-3). The function of the logic processor is to calculate the analysis parameters of the 4 subframes respectively, that is, to loop the calculation 4 times. Among them, for AMR-NB, the steps of each calculation may include: calculating the coefficients of the weighted LPC, calculating the LPC residual signal, calculating the target signal of the tone search, calculating the impulse response of the weighted synthesis filter (h1[]), calculating the closed-loop pitch (fundamental tone) parameters, encoding the pitch delay, updating the impulse response h1[] (including the pitch of the fixed gain), calculating the target vector of the codebook search, codebook search, encoding the codebook address, quantizing the pitch and codebook gain, calculating the synthesized speech and updating the state of the weighted filter, etc. After the 4 calculations are completed, the associated data is updated. The encoding calculation steps of other algorithms in AMR are similar.

[0095] Step S214: the logical processor transmits the second intermediate data to the general processor.

[0096] like Fig.13 As shown, in some implementations, step S214 may include sub-steps S2141 - S2143 .

[0097] Step S2141: the logical processor stores the second intermediate data into the memory according to the second data address.

[0098] Specifically, the logic processor stores the second intermediate data into the memory according to the second data address through the bus.

[0099] like Fig.14 As shown, the format of data transmitted from the encoding part logic processor to the general processor includes the first voice processing circuit (0-n), and the data transmission format corresponding to each first voice circuit is the sequentially arranged tone and innovation parameters (recorded as Parm[], a total of 14) and padding data 16'h0000, the number of which is 2. The padding data is used to make the output encoded data meet the fixed length. The sequential arrangement mentioned in this application can be arranged in sequence, or arranged in a little-endian arrangement, for example, the low byte is located at the low address, such as Fig.14 The left data (pitch and innovation parameters) in are located in the low address space. In one example, the 14 parameters in the pitch and innovation parameters are arranged in a little-endian arrangement.

[0100] Step S2142: The logical processor notifies the general processor to read the second intermediate data.

[0101] Specifically, the logical processor may read the second intermediate data by interrupting the general processor.

[0102] Step S2143: The general processor reads the second intermediate data from the memory according to the second data address.

[0103] Similarly, the general processor can configure the storage address of the parameters encoded by the logical processor, ie, the second data address, through the management configuration unit, so that the general processor can read the second intermediate data from the memory according to the second data address.

[0104] Step S215: The general processor packages the second intermediate data to obtain first encoded data.

[0105] The general processor packages the second intermediate data into a bit stream, obtains the first coded data, and outputs it.

[0106] See also Fig.15 , Fig.15 1 is a flow chart of an embodiment of the speech decoding method of the present application. In this embodiment, the speech decoding method is applied to the speech processing device in the above embodiment, and the speech processing device includes a coupled general processor and a logic processor.

[0107] The method may include the following steps:

[0108] Step S31: the general processor and the logical processor cooperate to decode at least one second encoded data to obtain second voice data, wherein the general processor is used to execute the non-cyclic calculation part of the decoding process, and the logical processor is used to execute the cyclic calculation part of the decoding process.

[0109] The second coded data is coded data, and after decoding, corresponding second voice data can be obtained. The description of this step can refer to step S11 in the above embodiment, and will not be repeated here.

[0110] See also Fig.16 , Fig.16 yes Fig.15 FIG. 1 is a flow chart of another implementation of step S31 in FIG. 1 . In this embodiment, the AMR-NB encoding and decoding algorithm is taken as an example for description.

[0111] Step S41: the general processor and the logic processor cooperate to execute the decoding part of the adaptive multi-rate speech decoding algorithm to decode at least one channel of second encoded data to obtain second speech data.

[0112] The description of this step can refer to step S31 in the above embodiment, which will not be repeated here. The difference is that in this embodiment, the decoding part of the multi-rate speech decoding algorithm (AMR) is executed.

[0113] See also Figures 17 to 21 , Fig.17 yes Fig.16A schematic flow chart of another implementation of step S41 in FIG. Fig.18 This is a schematic diagram of the format for decoding data transmitted from some general-purpose processors to logical processors. Fig.19 yes Fig.17 A flow chart of another implementation of step S412 in FIG. Fig. 20 yes Fig.17 A flow chart of another implementation of step S414 is shown in FIG. Fig.21 It is a schematic diagram of the format for decoding data passed from some logical processors to general processors.

[0114] In this embodiment, step S41 may further include steps S411 to S415.

[0115] Step S411: the general processor imports the second coded data, converts the second coded data into coding parameters, performs front-end processing on the coding parameters, and obtains third intermediate data.

[0116] The coding parameters may include but are not limited to retrieving coding parameters. The front-end processing of the coding parameters includes decoding Line Spectrum Frequency (LSF) parameters and generating interpolated LPC autocorrelation coefficients.

[0117] In some implementations, the third intermediate data includes some encoding parameters and other parameters.

[0118] like Fig.18 As shown, the format of data transmission from the general processor to the logical processor in the decoding part includes the second voice processing circuit (0-n), and the data transmission format corresponding to each second voice circuit is the partial coding parameter Parm[], other parameters (prev_lsf, past_lsf_q and A_t) arranged in order, and the padding data 16'h0000. Among them, prev_lsf and past_lsf_q are the current decoding parameters and the historical decoding parameters, and A_t is the fixed parameter in decoding. The number of partial coding parameters (Parm[]) is 14, prev_lsf and past_lsf_q are 10, A_t is 44, and the padding data is 2. As mentioned above, the number of all parameters can be selected according to the actual situation and is not limited here.

[0119] Step S412: the general processor transmits the third intermediate data to the logical processor, and notifies the logical processor to process the third intermediate data.

[0120] The general processor may send a third instruction to the logical processor, so that the logical processor processes the third data.

[0121] like Fig.19As shown, in some implementations, in step S412 , the general processor transmitting the third intermediate data to the logical processor may include sub-steps S4121 to S4123 .

[0122] Step S4121: The general processor stores the third intermediate data into the memory according to the third data address.

[0123] Step S4122: The general processor notifies the logical processor to read the third intermediate data.

[0124] The general processor may send a fourth instruction to the logical processor to notify the logical processor to read the third intermediate data. Here, the fourth instruction and the third instruction may be the same instruction or different instructions.

[0125] Step S4123: the logical processor reads the third intermediate data from the memory according to the third data address.

[0126] Since the logical processor has configured the third data address based on the management configuration unit, the logical processor can directly obtain the third data address, and then read the third intermediate data from the memory through the bus 40 based on the third data address.

[0127] Step S413: the logic processor performs a coding parameter loop calculation on the third intermediate data, updates the decoder characteristic parameters, and calculates the average line spectrum frequency to obtain fourth intermediate data.

[0128] Among them, in the decoding part of the AMR algorithm, the second voice processing circuit (amr_dec_calc) mainly includes the following steps:

[0129] 1) Delay of decoding pitch (pitch_lag value);

[0130] 2) Decoding algebraic codes;

[0131] 3) Decoding pitch and codebook;

[0132] 4) Find the stimulus and calculate the synthesized speech data (synthesis speech);

[0133] 5) Update decoder characteristic parameters;

[0134] 6) Calculate the LSF mean.

[0135] In the above, steps 1-4 implement the cyclic calculation of coding parameters, the purpose of which is to restore the coding parameters to the original voice data and realize the main process of decoding. In some implementations, the cyclic calculation of coding parameters can also be performed according to a preset number of subframes in the encoding process. The calculation steps for each subframe are the same, and steps 1-4 are performed, and the cycle is repeated 4 times. The decoding calculation steps of other algorithms in AMR are similar.

[0136] Step S414: the logical processor transmits the fourth intermediate data to the general processor.

[0137] like Fig. 20 As shown, in some implementations, step S414 may include sub-steps S4141 to S4143.

[0138] Step S4141: the logical processor stores the fourth intermediate data into the memory according to the fourth data address.

[0139] Step S4142: The logical processor notifies the general processor to read the fourth intermediate data.

[0140] Step S4143: The general processor reads the fourth intermediate data from the memory according to the fourth data address.

[0141] In one example, in the MR475 mode of AMR-NB, the encoding parameters calculated and input by each second voice processing circuit are 17 short data, of which the first 3 parameters are used in the general processor (PS end), and the remaining 14 parameters are used in the logic processor (PL end). The decoded data calculated and output by each second voice processing circuit are 160 integer (short) voice data and 88 short other data (lsf_hist and log_en_hist), a total of 248 operation results.

[0142] like Fig.21 As shown, the format of data transmitted from the logic processor to the general processor in the decoding part includes second voice processing circuits (0-n), and the data transmission format corresponding to each second voice circuit is voice data (recorded as synth) and other data (recorded as lsf_hist and log_en_hist) arranged in sequence, and the numbers are 160, 80 and 8 respectively. Among them, lsf_hist is a decoding parameter, and log_en_hist is an insertion table parameter, which is used to find the corresponding storage address.

[0143] Step S415: The general processor performs back-end filtering and back-end processing on the fourth intermediate data to obtain second voice data.

[0144] See also Fig. 22 , Fig. 22It is a comparison diagram of the first voice data and the second voice data in the time domain and frequency domain. That is, it is a comparison diagram of the original voice and the voice data restored by the encoding and decoding. Among them, the top is the comparison of the time domain signal, the bottom is the comparison of the frequency domain signal, the left side is the first voice data, and the right side is the second voice data. It can be seen that the voice data after encoding and decoding based on the method provided by this application has a high degree of restoration and a small degree of distortion.

[0145] As mentioned above, this application adopts a solution in which ARM and FPGA cooperate with each other. FPGA mainly provides powerful computing power, and ARM provides an efficient and convenient development method, which greatly reduces the processing time of the speech encoding and decoding algorithm and improves the encoding and decoding efficiency.

[0146] Furthermore, the speech processing device (ARM+FPGA architecture) based on the present application can also implement a more complex speech coding and decoding algorithm, thereby improving the efficiency and complexity of speech coding and decoding.

[0147] Furthermore, whether compared with the use of multiple vocoders or multiple MCUs to implement multi-channel voice encoding and decoding solutions at the same time, the ARM+FPGA architecture solution used in this application is far smaller than the existing implementation solutions in terms of size, power consumption and cost. In addition, multi-channel AMR encoding and decoding parallel processing can also be achieved, so that resources are fully utilized and hardware costs are reduced.

[0148] Furthermore, in this application, encoding and decoding are called in the form of a general IP core (Intellectual Property core, also known as IP core), which can realize the reuse of speech processing circuits in encoders and decoders, wherein the number of encoding and decoding paths is configured in the form of parameters. The IP core can be used arbitrarily on Xilinx, Altera or domestically produced FPGA platforms, or it can be integrated into a dedicated chip in the form of an IP core alone.

[0149] Furthermore, in some implementations, the ZYNQ7000 platform is used, which has strong scalability and flexibility and can be licensed to users in need in the form of IP core. At the same time, there are corresponding domestic alternatives to the ZYNQ7000 platform (FMQL PSOC series of Shanghai Fudan Microelectronics, domestic pure FPGA platforms of Fudan Microelectronics and Unigroup Guoxin), and the performance is almost the same, which can meet the requirements of military equipment.

[0150] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0151] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0152] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0154] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A speech coding method, characterized in that: The method is applied to a speech processing device, the speech processing device comprising a general processor and a logic processor coupled; The general processor and the logical processor cooperate to execute the encoding part of the adaptive multi-rate speech coding algorithm to encode at least one first speech data to obtain first coded data, wherein the general processor is used to execute the non-cyclic calculation part of the encoding process, and the logical processor is used to execute the cyclic calculation part of the encoding process; The general processor and the logic processor cooperate to execute the encoding part of the adaptive multi-rate speech coding algorithm, including: The general processor imports the first speech data, and analyzes and calculates linear prediction coding parameters to obtain first intermediate data; The general processor transmits the first intermediate data to the logical processor, and notifies the logical processor to process the first intermediate data; The logic processor cyclically calculates the pitch and innovation parameter of the first intermediate data, and updates the associated data to obtain second intermediate data; The logical processor transmits the second intermediate data to the general processor; The general processor packages the second intermediate data to obtain the first encoded data.

2. The method according to claim 1, characterized in that The first intermediate data includes the first speech data and other parameters, and the other parameters include the linear prediction coding parameters; wherein the first speech data and the other parameters are arranged in sequence.

3. The method according to claim 1, characterized in that The general processor transmits the first intermediate data to the logical processor, including: The general processor stores the first intermediate data into a memory according to a first data address; The general processor notifies the logical processor to read the first intermediate data; The logical processor reads the first intermediate data from the memory according to the first data address.

4. The method according to claim 1, characterized in that: The second intermediate data includes the tone and innovation parameters, and filling data; wherein the tone and innovation parameters, and filling data are arranged in sequence.

5. The method according to claim 1, characterized in that: The logical processor transmits the second intermediate data to the general processor, including: The logic processor stores the second intermediate data into the memory according to the second data address; The logical processor notifies the general processor to read the second intermediate data; The general processor reads the second intermediate data from the memory according to the second data address.

6. A speech decoding method, characterized in that: The method is applied to a speech processing device, the speech processing device comprising a general processor and a logic processor coupled; The general processor and the logical processor cooperate to execute a decoding part of an adaptive multi-rate speech decoding algorithm to decode at least one second coded data to obtain second speech data, wherein the general processor is used to execute a non-cyclic calculation part in the decoding process, and the logical processor is used to execute a cyclic calculation part in the decoding process; The general processor and the logic processor cooperate to execute the decoding part of the adaptive multi-rate speech decoding algorithm, including: The general processor imports the second coded data, converts the second coded data into coding parameters, performs front-end processing on the coding parameters, and obtains third intermediate data; The general processor transmits the third intermediate data to the logical processor, and notifies the logical processor to process the third intermediate data; The logic processor performs encoding parameter cyclic calculation, updates decoder characteristic parameters and calculates the average value of line spectrum frequency on the third intermediate data to obtain fourth intermediate data; The logical processor transmits the fourth intermediate data to the general processor; The general processor performs back-end filtering and back-end processing on the fourth intermediate data to obtain the second voice data.

7. The method according to claim 6, characterized in that The third intermediate data includes part of the encoding parameters, other parameters and filling data; wherein, part of the encoding parameters, other parameters and filling data are arranged in sequence.

8. The method according to claim 6, characterized in that The general processor transmits the third intermediate data to the logical processor, including: The general processor stores the third intermediate data into the memory according to the third data address; The general processor notifies the logical processor to read the third intermediate data; The logical processor reads the third intermediate data from the memory according to the third data address.

9. The method according to claim 6, characterized in that The fourth intermediate data includes voice data and other data, wherein the voice data and other data are arranged in sequence.

10. The method according to claim 6, characterized in that The logical processor transmits the fourth intermediate data to the general processor, including: The logic processor stores the fourth intermediate data into the memory according to the fourth data address; The logical processor notifies the general processor to read the fourth intermediate data; The general processor reads the fourth intermediate data from the memory according to the fourth data address.

11. A speech processing device, characterized in that: The apparatus includes a coupled general purpose processor, a logic processor, and a memory; The general processor and the logical processor are used to collaboratively execute the speech encoding method described in any one of claims 1 to 5 and / or collaboratively execute the speech decoding method described in any one of claims 6 to 10; The memory is used to store instructions and data required for the general processor and the logic processor to work.

12. The device according to claim 11, characterized in that The logic processor includes at least one parallel voice processing circuit, each of which is used to execute a cyclic calculation part in the encoding process of the first voice data and / or a cyclic calculation part in the decoding process of the second encoded data.

13. The device according to claim 12, characterized in that The speech processing circuit comprises a cyclic calculation unit, a cache unit, a lookup table unit and a management configuration unit, wherein the cyclic calculation unit is connected to the cache unit, the lookup table unit and the management configuration unit.

14. The device according to claim 11, characterized in that The device further includes a bus, wherein the bus is connected to the general processor, the logical processor and the memory respectively, and the general processor, the logical processor and the memory transmit data through the bus.

15. The device according to claim 11, characterized in that The logic processor includes a field programmable gate array and / or a digital signal processor.

Images