Decoding server and multi-channel decoding method suitable for multi-coding voice data

Abstract

The invention provides a decoding server suitable for multi-coding voice data, and the decoding server is provided with a plurality of decoders with different voice coding formats, and each decoder is optimized through the following method: a buffer which comprises a plurality of buffer queues and is used for receiving a voice decoding request, distributing the received voice decoding request to a cache queue for caching according to a dynamic distribution function; the voice decoding request comprises a coding format, a data frame length and a coding data stream; and the parallel processor comprises a plurality of decoding channels, each decoding channel corresponds to one buffer queue, and the parallel processor is used for reading the voice decoding request from the buffer, calling a corresponding decoder for decoding according to the encoding format in the voice decoding request, obtaining voice decoding data and transmitting the voice decoding data.

Description

Technical Field

[0001] The present invention relates to the field of coding and decoding, and in particular to a decoding server and a multi-channel decoding method suitable for multi-coded voice data. Background Art

[0002] Starting from 4G LTE, the voice implementation solution is no longer just 2G / 3G network, which simply provides voice services and other value-added services through circuit domain network. Instead, it is designed to carry voice services on IP network through IP Multimedia Subsystem (IMS), that is, VoLTE (Voice over LTE), to realize the "IPization" of all 2G / 3G circuit domain services on 4G network.

[0003] Existing devices such as call network element media gateways or base stations directly decode voice data according to the voice coding format, which occupies and consumes a lot of system resources such as CPU. In addition, due to the rapid development of communication technology, the original 2G era only supported basic PCMA / PCMU coding modes, and now 4 / 5G GSM / ILBC / G723.1 / G729 / AMR-NB / AMR-WB / EVS / G722 and other voice coding modes, MP4 / H264 / H263 / MPEG and other video coding modes, resulting in the need for call network element media gateways or base stations to support more and more coding modes. Due to the amount of data carried, the use of open source voice decoding tools such as ffmpeg, open source ITU-T or 3GPP codec source codes, etc., can only support serial decoding of voice data, which will lead to low decoding efficiency and waste of system resources such as CPU when batch concurrent decoding is performed.

[0004] At the same time, the data variables of the source code operations in the existing ITU-T and 3GPP voice decoding are all single-dimensional global variables, and there are no multi-dimensional variables. If cross-voice decoding or batch data decoding is performed simultaneously, it will cause noise in the decoded voice and cause the decoding operation to crash.

[0005] In order to solve the above problems, people have been seeking an ideal technical solution. Summary of the invention

[0006] The object of the present invention is to address the deficiencies of the prior art and thus provide a decoding server and a multi-channel decoding method suitable for multi-encoded voice data.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: A first aspect provides a decoding server suitable for multi-coded voice data, wherein the decoding server is deployed with: Decoders for several different speech coding formats, the decoders comprising decoding source codes for the corresponding coding formats; A buffer, comprising a plurality of buffer queues, for receiving voice decoding requests and distributing the received voice decoding requests to the buffer queues for caching according to a dynamic distribution function; the voice decoding requests include a coding format, a data frame length and a coded data stream; The parallel processor includes multiple decoding channels, each decoding channel corresponds to a cache queue, which is used to read the voice decoding request from the cache, and call the corresponding decoder for decoding according to the encoding format in the voice decoding request, obtain the voice decoding data and transmit it.

[0008] Furthermore, a corresponding decoder is created for each speech coding format, and configuration parameters are initialized; when configuring parameters, the global static single-dimensional variables in the decoding source code are modified to global static multi-dimensional variables, and then static variable memory is allocated; Each decoding channel calls the corresponding decoder to perform decoding according to the voice coding format in the voice decoding request, including: The decoding channel requests the memory space of the corresponding dimension from the decoder according to the data frame length in the speech decoding request, and modifies the overflow variables and carry variables in the basic operation file of the decoding source code from global static single-dimensional variables to global static multi-dimensional variables of the corresponding dimension; finally, the modified decoder is used to decode the speech encoding data to obtain the speech decoding data.

[0009] Furthermore, after receiving the data frame length sent by the decoding channel, the decoder allocates memory space of corresponding dimensions to the decoding channel, wherein the memory space allocated to different decoding channels is independent and non-repetitive.

[0010] Further, the parallel processor obtains channel configuration information, the channel configuration information includes a decoding channel type, a number of CPU cores, and a decoding interval, the decoding channel type includes a thread, a process, or a service; Generate an equal number of thread-level decoding channels, process-level decoding channels or service-level decoding channels according to the number of CPU cores and decoding channel types; Create an equal number of cache queues in the cache according to the number of CPU cores, and associate and bind a cache queue with a decoding channel; The speech decoding requests are obtained from the cache queue in batches according to the decoding interval.

[0011] Furthermore, the configuration information also includes resampling configuration information and channel configuration information; After receiving the voice decoding data, the decoding channel resamples the voice decoding data based on the resampling configuration information and the channel configuration information, and outputs the voice encoding data of the corresponding channel.

[0012] Furthermore, each decoding channel reads the voice decoding request from the buffer and also obtains the queue cache time of the voice decoding request, and determines whether the difference between the current time and the queue cache time exceeds the preset time threshold. If it exceeds, the voice decoding request is directly transmitted, and it is informed that it is undecoded original data; if it does not exceed, the corresponding decoder is called for decoding according to the encoding format in the voice decoding request, and the voice decoding data is obtained and transmitted.

[0013] A second aspect provides a multi-channel decoding method applicable to multi-coded voice data, applicable to the decoding server described in the first aspect, the decoding server is communicatively connected with a client network element device, and the multi-channel decoding method includes: Build a docking environment with customer network element equipment; After receiving multiple voice encoding requests sent by each customer network element device, calling the corresponding decoder for decoding according to the encoding format in the voice decoding request to obtain voice decoding data; The voice decoding data is encapsulated according to the preset transmission protocol, and a response data packet is generated and returned to the customer network element device.

[0014] A third aspect provides a multi-channel decoding method applicable to multi-coded voice data, applicable to a client network element device communicating with the decoding server according to the first aspect, the multi-channel decoding method comprising: Build the docking environment with the decoding server; When receiving data sent by the previous network element device, determine whether the data is encoded data. If it is encoded data, send a voice decoding request to the decoding server, and the voice decoding request includes the encoding format, data frame length and encoded data stream; if it is not decoded data, directly send the data to the target device; Receive the response data packet returned by the decoding server, obtain the decoded data from the corresponding data packet, reassemble the data and send it to the target device.

[0015] Furthermore, after setting up the docking environment with the decoding server, the running status of the decoding server is confirmed in real time through heartbeat interaction. When the running status of the decoding server is normal, it is determined whether the data is encoded data; when the running status of the decoding server is abnormal, the data is directly sent to the target device.

[0016] The present invention has outstanding substantive features and significant progress compared to the prior art. Specifically, 1. Set up multiple decoding channels, each corresponding to a decoding request, and call the corresponding decoder for decoding according to the voice coding format in the decoding request; process multiple decoding requests simultaneously through multiple decoding channels, which can avoid data intersection and reduce the occupation and consumption of system resources such as CPU by high concurrency; 2. Carry out secondary development of the decoding source code of the open source decoder, modify the single-dimensional variable into a multi-dimensional variable to enable the decoder to simultaneously process multiple decoding requests of the same type, and the secondary development source code can be fully disclosed to ensure the integrity, correctness, standardization and openness of data decoding; 3. Only one general server is needed to complete the deployment. Unlike the existing network element devices, it does not need to call network element media gateways or base stations and other devices, reducing manpower and costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of the decoding server described in Example 1.

[0018] Figure 2 This is a schematic diagram of the original EVS process.

[0019] Figure 3 It is the original EVS decode and its operation variables.

[0020] Figure 4 It is the original open source decoding serial process.

[0021] Figure 5 It is a flow chart of the EVS after secondary development of the present invention.

[0022] Figure 6 It is an overall flow chart of the multi-channel decoding method described in Example 2. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is further described in detail below through specific implementation methods.

[0024] Example 1 This embodiment provides a decoding server suitable for multi-coded voice data, such as Figure 1 As shown, the decoding server is deployed with: Decoders for several different speech coding formats, the decoders comprising decoding source codes for the corresponding coding formats; A buffer, comprising a plurality of buffer queues, for receiving voice decoding requests and distributing the received voice decoding requests to the buffer queues for caching according to a dynamic distribution function; the voice decoding requests include a coding format, a data frame length and a coded data stream; The parallel processor includes multiple decoding channels, each decoding channel corresponds to a cache queue, which is used to read the voice decoding request from the cache, and call the corresponding decoder for decoding according to the encoding format in the voice decoding request, obtain the voice decoding data and transmit it.

[0025] It is understandable that the decoding source codes of the decoder are all source codes that meet the ITU-T and 3GPP standards, and the encoding formats include but are not limited to G711 / G722 / G723 / G729 / AMR-NB / AMR-WB / EVS / GSM / ILBC / ILBC / SILK / H264 / H263.

[0026] Take the existing EVS original decoding process as an example. Figure 2-3 As shown in the figure, it can be seen that the variable in the middle is a global static single-dimensional variable. Therefore, if batch data is encountered, it can only be decoded serially, and the decoding time is long. Figure 4 As shown; when cross-voice decoding or batch data decoding at the same time, it is easy to cause noise in the decoded voice and cause the decoding operation to crash.

[0027] To this end, this embodiment conducts secondary development on the open source decoding source code, and modifies the single-dimensional variable into a multi-dimensional variable, so that it is suitable for cross-speech decoding or batch data decoding at the same time, such as Figure 5 As shown, the details are as follows: Create a corresponding decoder for each speech coding format and initialize the configuration parameters; when configuring the parameters, modify the global static single-dimensional variables in the decoding source code to global static multi-dimensional variables, and then allocate static variable memory; Each decoding channel calls the corresponding decoder to perform decoding according to the voice coding format in the voice decoding request, including: The decoding channel requests the memory space of the corresponding dimension from the decoder according to the data frame length in the speech decoding request, and modifies the overflow variables and carry variables in the basic operation file of the decoding source code from global static single-dimensional variables to global static multi-dimensional variables of the corresponding dimension; finally, the modified decoder is used to decode the speech encoding data to obtain the speech decoding data.

[0028] That is to say, the decoding channel requests the memory space of the corresponding dimension from the decoder according to the data frame length in the voice decoding request, and modifies the overflow variables and carry variables in the basic operation file of the decoding source code from global static single-dimensional variables to global static multi-dimensional variables of the corresponding dimensions. This is equivalent to dividing the original decoder into several small decoders, each of which corresponds to a decoding channel, for decoding the voice encoding data of the decoding channel; and multiple small decoders can work at the same time to realize the simultaneous decoding of multiple voice encoding data.

[0029] It should be noted that after the decoder receives the data frame length sent by the decoding channel, it allocates memory space of corresponding dimensions for the decoding channel, where the memory space allocated to different decoding channels is independent and non-repetitive, thereby ensuring that a decoder is virtually divided into several small decoders that do not interfere with each other.

[0030] In one embodiment, the parallel processor includes the following steps for obtaining a decoding channel: The parallel processor obtains channel configuration information, which includes decoding channel type, number of CPU cores, and decoding interval. The decoding channel type includes thread, process or service. The configuration information is pre-set. Customers can customize it according to their needs. It has the advantages of simple operation and fast upgrade. It can be deployed individually or in a clustered manner.

[0031] In specific implementation, at least one of multi-threading, multi-processing, and multi-service is used to decode multiple speech encoding data. Specifically, the same number of thread-level decoding channels, process-level decoding channels, or service-level decoding channels are generated according to the number of CPU cores and the type of decoding channels; for example, if the configuration information is multi-threading and 100 CPU cores, 100 process-level decoding channels will be created to decode data simultaneously; Create an equal number of cache queues in the cache according to the number of CPU cores, and associate and bind a cache queue with a decoding channel; the cache queue is a hash bucket structure, and the encoding format, data length, etc. in the voice decoding request are stored in different nodes of the hash bucket.

[0032] The speech decoding requests are obtained from the cache queue in batches according to the decoding interval. For example, each batch of decoding is performed with a storage length of 4 seconds or 200 packets of data are decoded as a batch.

[0033] Furthermore, the configuration information also includes resampling configuration information and channel configuration information; After receiving the voice decoding data, the decoding channel resamples the voice decoding data based on the resampling configuration information and the channel configuration information, and outputs the voice encoding data of the corresponding channel.

[0034] For example, if the resampling configuration information includes a sampling precision of 16 bits, and the channel configuration information is mono, then the speech decoding data is resampled based on the sampling precision of 16 bits, and mono speech encoding data is output.

[0035] It should be noted that, under normal circumstances, the jitter time of each frame of data is between 20 milliseconds and 150 milliseconds depending on the encoding. Affected by the network environment, the data transmission time is not fixed, and the normal decoding process takes some time. In order to reduce the perception of the target device, each decoding channel reads the voice decoding request from the buffer and also obtains the queue cache time of the voice decoding request, and determines whether the difference between the current time and the queue cache time exceeds the preset time threshold. If it exceeds, the voice decoding request is directly transmitted; if not, the corresponding decoder is called for decoding according to the encoding format in the voice decoding request, and the voice decoding data is obtained and transmitted.

[0036] Specifically, the preset time threshold can be set to 20 milliseconds. Once the difference between the current time and the queue cache time exceeds 20 milliseconds, no decoding processing is performed, and the data is directly sent to the target device, and it is notified that it is undecoded original data.

[0037] In this embodiment, multiple decoding channels are set, each decoding channel corresponds to a decoding request, and the corresponding decoder is called for decoding according to the voice coding format in the decoding request; multiple decoding requests are processed simultaneously through multiple decoding channels, which not only avoids data crossing, but also reduces the occupation and consumption of system resources such as CPU by high concurrency of codec transcoding; the decoding source code of the open source decoder is secondary developed, and the single-dimensional variable is modified into a multi-dimensional variable to enable the decoder to simultaneously process multiple decoding requests of the same type, and its secondary development source code can be fully disclosed, which can ensure the integrity, correctness, standardization and openness of data decoding.

[0038] Example 2 This embodiment provides a multi-channel decoding method applicable to multi-coded speech data, such as Figure 6 As shown, it is applicable to the decoding server described in Example 1, and the decoding server is connected to multiple client network element devices in communication, such as Figure 6 As shown, the multi-channel decoding method includes: Build a docking environment with customer network element equipment; due to business needs, this method runs in the Linux system, so customized data docking with customer network element equipment is performed through UDP socket / HTTP / KAFKA and other methods; Receive multiple voice encoding requests sent by each customer network element device, call the corresponding decoder for decoding according to the encoding format in the voice decoding request, and obtain voice decoding data; The voice decoding data is encapsulated according to the preset transmission protocol, and a response data packet is generated and returned to the customer network element device.

[0039] Specifically, the preset transmission protocols include GRPC / BRPC / UDP / KAFKA and other methods.

[0040] Example 3 This embodiment provides a multi-channel decoding method applicable to multi-coded voice data, applicable to a client network element device communicating with the decoding server described in Embodiment 1, and the multi-channel decoding method includes: Build a docking environment with the decoding server; due to business needs, the decoding server generally runs on the Linux system, so customized data docking with the decoding server is performed through UDP socket / HTTP / KAFKA and other methods.

[0041] When receiving data sent by the previous network element device, determine whether the data is encoded data. If it is encoded data, send a voice decoding request to the decoding server, and the voice decoding request includes the encoding format, data frame length and encoded data stream; if it is not decoded data, directly send the data to the target device; Receive the response data packet returned by the decoding server, obtain the decoded data from the corresponding data packet, reassemble the data, and send it to the target device, such as reassembling the data into UDP to generate a UDP message.

[0042] It is understandable that the response data packet also includes a decoded segment, wherein the decoded segment uses 2 bytes as a variable and the range is 0-65535 and is used cyclically.

[0043] It should be noted that after setting up the docking environment with the decoding server, the running status of the decoding server is confirmed in real time through heartbeat interaction. When the running status of the decoding server is normal, it is judged whether the data is encoded data; when the running status of the decoding server is abnormal, the data is directly sent to the target device. This ensures that decoding is performed only when the decoding server is normal, preventing long decoding time and decoding errors caused by abnormal decoding server.

[0044] Example 4 This embodiment provides a testing method: S1: Build and run the ring and database environment; Due to business needs, the decoding server generally runs in the Linux system. In order to verify the decoding system, a client and a server are established, where the server corresponds to the decoding server and has the decoding server function built in, and the client corresponds to the customer network element device and has the customer network element device function built in. The client and the server are run in the Linux system.

[0045] However, most of the existing staff's office environment is Windows system, so when testing, testers need to implement cross-platform testing operations. Therefore, the client and server software are pre-built on the Linux system, and the database environment and configuration software are deployed on the Windows system. The Windows system is required to be Windows Server 2012.

[0046] Specifically, deploy the Oracle server on the Windows system, create related tables and configure data. The configuration information includes the client's IP / PORT, the server's IP / PORT, the encoding format, the voice file data sent, the number of concurrent calls, resampling, mono or dual channels, etc.

[0047] S2: The client and server access the Oracle server, read the corresponding table, and configure and distribute the data.

[0048] S3: The server loads the decoding conversion system in memory according to the client IP / PORT, server IP / PORT, encoding format, sampling rate, and channel configuration read from the table, and waits for the client data to arrive and decode.

[0049] The server software will establish a long connection in the form of a TCP SOCKET server, waiting for the client to establish a TCP channel; since the voice encoding format is specified in the configuration, the content initiated by the client will be decoded according to the ITU-T or 3GPP specifications (the voice content initiated by the client should comply with the transmission specifications and encoding specifications of each encoding).

[0050] S4: The client generates corresponding data encoding verification test cases in memory according to the client IP / PORT, server IP / PORT, encoding format, and number of concurrent paths read from the table.

[0051] The client software will establish a persistent connection to the server in the form of a TCP SOCKET client. The voice encoding is specified in the configuration, and data decoding will be initiated to the server according to the ITU-T or 3GPP specifications (the client can send data according to the encoding format, number of concurrent channels and configured voice file path).

[0052] It should be noted that the client software receives the content of the voice file sent by the user.

[0053] S5: The client acts as the data decoding initiator and performs data decoding test according to the configured conditions and voice file content.

[0054] Through the above tests, it can be seen that the multi-channel decoding method proposed in the present invention can achieve high-concurrency processing of decoded data, which not only efficiently uses the CPU and memory resources in the system, but also integrates the mainstream encoding G711 / G722 / G723 / G729 / AMR-NB / AMR-WB / EVS / GSM / ILBC / SILK and other voice encoding formats and H264 / H263 / MP4 / MPEG and other video encoding formats, and has a complete solution for audio and video solutions.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present invention, which should be included in the scope of the technical solution for protection of the present invention.

Claims

1. A decoding server suitable for multi-encoded voice data, characterized in that: The decoding server is deployed with: Decoders for several different speech coding formats, the decoders comprising decoding source codes for the corresponding coding formats; A buffer, including a plurality of buffer queues, for receiving speech decoding requests and distributing the received speech decoding requests to the buffer queues for caching according to a dynamic distribution function; The speech decoding request includes a coding format, a data frame length and a coded data stream; The parallel processor includes multiple decoding channels, each decoding channel corresponds to a cache queue, which is used to read the voice decoding request from the cache, and call the corresponding decoder for decoding according to the encoding format in the voice decoding request, obtain the voice decoding data and transmit it.

2. A decoding server suitable for multi-coded voice data according to claim 1, characterized in that: Create a corresponding decoder for each speech coding format and initialize the configuration parameters; when configuring the parameters, modify the global static single-dimensional variables in the decoding source code to global static multi-dimensional variables, and then allocate static variable memory; Each decoding channel calls the corresponding decoder to perform decoding according to the voice coding format in the voice decoding request, including: The decoding channel requests the memory space of the corresponding dimension from the decoder according to the data frame length in the speech decoding request, and modifies the overflow variables and carry variables in the basic operation file of the decoding source code from global static single-dimensional variables to global static multi-dimensional variables of the corresponding dimension; finally, the modified decoder is used to decode the speech encoding data to obtain the speech decoding data.

3. A decoding server suitable for multi-coded voice data according to claim 2, characterized in that: After the decoder receives the data frame length sent by the decoding channel, it allocates memory space of corresponding dimensions to the decoding channel, where the memory space allocated to different decoding channels is independent and non-repetitive.

4. A decoding server suitable for multi-coded voice data according to claim 1, characterized in that: The parallel processor obtains channel configuration information, where the channel configuration information includes a decoding channel type, a number of CPU cores, and a decoding interval, where the decoding channel type includes a thread, a process, or a service; Generate an equal number of thread-level decoding channels, process-level decoding channels or service-level decoding channels according to the number of CPU cores and decoding channel types; Create an equal number of cache queues in the cache according to the number of CPU cores, and associate and bind a cache queue with a decoding channel; The speech decoding requests are obtained from the cache queue in batches according to the decoding interval.

5. A decoding server suitable for multi-coded voice data according to claim 4, characterized in that: The configuration information also includes resampling configuration information and channel configuration information; After receiving the voice decoding data, the decoding channel resamples the voice decoding data based on the resampling configuration information and the channel configuration information, and outputs the voice encoding data of the corresponding channel.

6. A decoding server suitable for multi-coded voice data according to any one of claims 1 to 5, characterized in that: Each decoding channel reads the voice decoding request from the buffer and also obtains the queue cache time of the voice decoding request, and determines whether the difference between the current time and the queue cache time exceeds the preset time threshold. If it exceeds, the voice decoding request is directly transmitted and it is informed that it is the undecoded original data; if it does not exceed, the corresponding decoder is called for decoding according to the encoding format in the voice decoding request, and the voice decoding data is obtained and transmitted.

7. A multi-channel decoding method applicable to multi-coded voice data, applicable to the decoding server according to any one of claims 1 to 6, wherein the decoding server is communicatively connected to a plurality of client network element devices, characterized in that: Build a docking environment with customer network element equipment; Receive multiple voice encoding requests sent by each customer network element device, call the corresponding decoder for decoding according to the encoding format in the voice decoding request, and obtain voice decoding data; The voice decoding data is encapsulated according to the preset transmission protocol, and a response data packet is generated and returned to the customer network element device.

8. A multi-channel decoding method for multi-coded voice data, applicable to a client network element device communicating with a decoding server according to any one of claims 1 to 6, characterized in that: Build the docking environment with the decoding server; When receiving data sent by the previous network element device, determine whether the data is encoded data, and if it is encoded data, send a voice decoding request to the decoding server, wherein the voice decoding request includes the encoding format, the data frame length and the encoded data stream; If it is not decoded data, the data is directly sent to the target device; Receive the response data packet returned by the decoding server, obtain the decoded data from the corresponding data packet, reassemble the data and send it to the target device.

9. A multi-channel decoding method applicable to multi-coded speech data according to claim 8, characterized in that: After setting up the docking environment with the decoding server, the running status of the decoding server is confirmed in real time through heartbeat interaction. When the running status of the decoding server is normal, it is determined whether the data is encoded data; When the operation status of the decoding server is abnormal, the data is directly sent to the target device.

Images