A multi-channel RTP delay balanced sending system and method based on FPGA

Abstract

A multi-channel RTP delay balanced sending system and method based on FPGA, relates to the technical field of data transmission, FPGA includes an audio acquisition module, a first-level cache, a second-level cache, an RTP packaging module, a UDP / IP protocol stack module and a Gigabit Ethernet interface; the audio data of the audio channel is written into the first-level cache multiple times at the end of each channel, and is read out from the first-level cache and written into the second-level cache multiple times at the end of each sampling, and RTP message packaging starts each time the writing into the second-level cache is completed; RTP packaging starts the first packaging after the 48th sampling point is completed, and then the latest 48 sampling point data are taken in each cycle. The present invention realizes the decentralized packaging and sending of RTP stream data through the temporary storage of the first-level cache and the cyclic update of the second-level cache, realizes the equalization of RTP stream delay and the equalization of network bandwidth occupation, reduces the peak forwarding performance requirements of the switch in a multi-device environment, and effectively optimizes the problem of network congestion.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a multi-channel RTP delay equalization sending system and method based on FPGA. Background Art

[0002] In the field of network media, FPGA has inherent advantages over ARM processors in multi-channel, large-capacity data acquisition, multi-channel network data stream generation and protocol packaging transmission due to its flexible and high-speed interface and efficient concurrent data processing capabilities.

[0003] Especially for the application scenario of sending multiple RTPs of network audio, since the same media source will be subscribed and played by multiple terminal devices, multi-channel multicast is usually used for streaming, which can reduce the network bandwidth usage in this scenario. However, in the scenario of sending multiple RTP streams, if all streams are sent at the same time, it may cause network congestion, resulting in uneven delays between streams; and if all RTP streams are sent at the same time, extremely high network bandwidth requirements will be generated in a short period of time, causing network peak bandwidth pressure. Summary of the invention

[0004] In view of the above technical problems in the related art, the present invention proposes a multi-channel RTP delay equalization sending system and method based on FPGA.

[0005] In order to achieve the above-mentioned invention object, the present invention adopts the following technical scheme:

[0006] In a first aspect, the present invention provides a multi-channel RTP delay balanced sending system based on FPGA, wherein the FPGA includes an audio acquisition module, a cache module, an RTP packaging module, a UDP / IP protocol stack module and a Gigabit Ethernet interface:

[0007] The audio acquisition module is used to collect audio data of M audio channels;

[0008] The cache module is connected to the audio acquisition module and the RTP packaging module. The cache module includes a primary cache and a secondary cache. The audio data of the audio channel is written into the primary cache multiple times at the end of each channel, and is read from the primary cache multiple times and written into the secondary cache at the end of each sampling. RTP message packaging starts each time the writing into the secondary cache is completed;

[0009] The RTP packaging module is used to package the audio data in the secondary cache according to the audio transmission routing table to obtain multiple RTP streams;

[0010] When the RTP packaging module takes data, the first packaging starts when the 48th sampling point is completed. The starting address of the data source is the starting address of the 1st sampling point, and the ending address is the ending address of the 48th sampling point. When the 49th sampling point is completed, the second packaging starts. The starting address of the data source is the starting address of the 2nd sampling point, and the ending address is the ending address of the 1st sampling point. This is equivalent to moving the read pointer back one sampling point, and the cycle is repeated according to this rule.

[0011] The UDP / IP protocol stack module is connected to the RTP packaging module, which realizes the packet packaging and parsing of the MAC layer, IP layer, and UDP layer, and is used to perform UDP / IP encapsulation on the packaged RTP stream and then send it out through the Gigabit Ethernet interface;

[0012] The Gigabit Ethernet interface is connected to the Ethernet physical layer transceiver chip for sending and receiving Ethernet data packets;

[0013] Specifically, the audio acquisition interface of the audio acquisition module is a TDM interface, which is composed of multiple TDM data lines, a bit clock, and a sampling start pulse; the sampling start pulse in the TDM interface is the starting time of each sampling point.

[0014] Specifically, the data of the secondary cache is updated when each sample data is written. The time when each sample data is written is the RTP packaging time. The data storage order from low address to high address is: the first channel data of the first sampling point, the second channel data of the first sampling point, ..., the 64th channel data of the first sampling point, the first channel data of the second sampling point, the second channel data of the second sampling point, ..., the 64th channel data of the second sampling point, ..., the first channel data of the 48th sampling point, the second channel data of the 48th sampling point, ..., the 64th channel data of the 48th sampling point. After storing the data of the 48th sampling point, the data of the 49th sampling point is overwritten to the position of the first sampling point, and this rule is repeated.

[0015] Specifically, the first-level cache is a RAM with a width of 24 bits and a depth of 64 bits, which can accommodate data of 1 sampling point of 64 channels; the second-level cache is a RAM with a width of 24 bits and a depth of 4096 bits, which can accommodate data of 48 sampling points of 64 channels.

[0016] Specifically, the TDM clock domain is used to write to the first-level cache, and the writing time points are staggered according to the order of each TDM data line, that is, the writing is delayed backward in sequence, and the time period from the bit count 31 of the previous channel to the bit count 6 of the next channel using the TDM bit count is divided into 8 writes; the TDM bit count is 0-31.

[0017] Specifically, the reading from the primary cache and writing into the secondary cache in multiple times at the end of each sampling is specifically: using the Ethernet clock domain to read the primary cache and writing into the secondary cache in 64 times.

[0018] Specifically, the TDM clock domain frequency is 12.288 MHz; the Ethernet clock domain frequency is 125 MHz.

[0019] Specifically, the audio acquisition module has 8 TDM data lines.

[0020] Specifically, the system further includes an audio transmission routing table, in which the corresponding relationship between the audio data and the audio channels in the RTP stream is stored, and is used to determine the transmission path of the audio data.

[0021] Specifically, the Gigabit Ethernet interface is a RGMII interface.

[0022] In a second aspect, the present invention provides a multi-channel RTP delay equalization sending method based on FPGA, based on the multi-channel RTP delay equalization sending system based on FPGA described in any one of the first aspects, comprising the following steps:

[0023] S1, collecting audio data of multiple audio channels through an audio collection module;

[0024] S2, the audio data of each audio channel is written into the primary cache multiple times according to the TDM data line sequence at the end of each channel, and is read out from the primary cache multiple times according to the channel sequence and written into the secondary cache at the end of each sampling, and when each writing into the secondary cache is completed, the audio data in the secondary cache is packaged into multiple RTP streams according to the mapping relationship specified in the routing table according to the audio sending routing table;

[0025] When the RTP packaging module takes data, the first packaging starts when the 48th sampling point is completed. The starting address of the data source is the starting address of the 1st sampling point, and the ending address is the ending address of the 48th sampling point. When the 49th sampling point is completed, the second packaging starts. The starting address of the data source is the starting address of the 2nd sampling point, and the ending address is the ending address of the 1st sampling point. This is equivalent to moving the read pointer back one sampling point, and the cycle is repeated according to this rule.

[0026] S3, encapsulate multiple RTP streams into UDP / IP, and then send them out through the Gigabit Ethernet interface.

[0027] The present invention realizes decentralized packaging and sending of RTP stream data through temporary storage in the first-level cache and cyclic update of the second-level cache, realizes equalization of RTP stream delay and network bandwidth occupancy, reduces the peak forwarding performance requirements of switches in a multi-device environment, and effectively optimizes the problem of network congestion. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 This is a schematic diagram of a multi-channel RTP delay equalization sending system based on FPGA provided according to an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a cache module workflow according to an embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of a multi-channel RTP delay equalization sending method based on FPGA provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0033] In the description of the present invention, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0034] In the description of the present invention, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

[0035] Embodiment 1

[0036] refer to Figure 1 , This embodiment provides a multi-channel RTP delay balanced sending system based on FPGA, wherein the FPGA includes an audio acquisition module, a cache module, an RTP packaging module, a UDP / IP protocol stack module and a Gigabit Ethernet interface;

[0037] An audio acquisition module, used to acquire audio data of multiple audio channels;

[0038] The audio acquisition interface of the audio acquisition module is a TDM interface, which is composed of multiple time division multiplexing (TDM) data lines, a bit clock (BCLK), and a sampling start pulse (LRCK).

[0039] TDM (Time-Division Multiplexing) is a data transmission and sampling technology that allows multiple signals to be transmitted over the same transmission medium, but they are separated in time.

[0040] Preferably, in this embodiment, there are 8 TDM data lines in the audio acquisition module; each TDM data line can accommodate the transmission and reception of up to 8 audio channels, and the 8 TDM data lines can accommodate up to 64 channels of audio data;

[0041] In this embodiment, the 8 TDM data lines are in order. In order to better distinguish them, the TDM data lines can be named in order, namely TDM0, TDM1, TDM2, TDM3, TDM4, TDM5, TDM6 and TDM7.

[0042] The cache module is connected to the audio acquisition module and the RTP packaging module. The cache module includes a primary cache and a secondary cache. The audio data of the audio channel is written into the primary cache multiple times at the end of each channel, and is read from the primary cache multiple times and written into the secondary cache at the end of each sampling. RTP message packaging starts each time the writing into the secondary cache is completed;

[0043] In the prior art, the conventional sending method of multi-channel RTP is to set an AB double buffer interval, where the A buffer and the B buffer are generally set to the same size, and all audio data collected for a unit time (such as 1ms) is written into the A buffer, and then all audio data for the next unit time is written into the B buffer. At the same time, the audio data in the A buffer is packaged into multiple RTP streams according to the content of the routing table and sent in sequence. Since multiple RTP streams must be sent in sequence, each RTP stream has a maximum of 1152 bytes of audio data, and the maximum length of the entire packet is 1218 bytes (including the minimum frame interval of 12 bytes). The time it takes to send is about 10us. The sending delay of the first RTP stream is 1ms (starting from the first audio sample in the packet), and the sending delay of the 64th RTP stream is 1.63ms. In order to balance the delay of each RTP stream and reduce the peak network bandwidth demand, multiple RTP streams are not all sent at the same unit time position. There are 48 sampling points in 1ms. 0-2 RTP streams are sent at each sampling point. The interval between sampling points is about 20.8us. At most two largest RTP streams can be sent. 0-96 RTP streams can be sent at 48 sampling points. The data source of the sent RTP stream is the audio data of a total of 48 sampling points from this sampling point. In this way, the packaging delay of each RTP stream is 1ms-1.01ms, which is more balanced than the delay packaged at the same time and has a lower average delay. It also balances the network bandwidth usage, and when multiple devices are sending, lower requirements are placed on the switch's peak forwarding performance.

[0044] refer to Figure 2 A two-level cache is set between the audio acquisition module and the RTP packaging module, including a first-level cache and a second-level cache; the first-level cache is a RAM with a width of 24 bits and a depth of 64, which can accommodate the data of 1 sampling point of all 64 channels; the second-level cache is a RAM with a width of 24 bits and a depth of 4096, which can accommodate the data of 48 sampling points of all 64 channels.

[0045] It is understandable that the complete audio data is 24 bits, and the common sampling frequency of audio data is 48KHz, that is, there are 48 sampling points in 1ms as mentioned above, and 48 data are collected in 1 second. In order to accommodate the data of 48 sampling points of 64 channels, the depth of the secondary cache needs to be at least 3072, but because the depth of RAM must be 2 to the power of n, the secondary cache is a RAM with a depth of 4096.

[0046] A sampling point refers to a discrete point in time of an audio signal. In digital audio processing, a continuous audio signal is sampled at a certain frequency (sampling rate). Each sampling point represents the amplitude value of the audio signal at a specific moment. In a TDM interface, the sampling start pulse (LRCK) is the starting time of each sampling point.

[0047] The RTP packaging module takes data from the secondary cache, and the data in the secondary cache is written by the primary cache. The reason why it is not directly written by the collected data is that the data of each channel of a sampling point is collected at different times (the TDM interface has a channel arrangement order), but the cached data cannot be changed during the sampling time from each RTP packaging moment to the completion of the transmission. Therefore, a primary cache is required to cache the data of the next sampling point first, and write the data of the primary cache to the secondary cache at one time when this sampling point ends; write the primary cache using the TDM clock domain (frequency is 12.288MHz), staggered according to the order of each TDM data line, that is, delayed writing in sequence, and writing in 8 times using the time period of TDM bit count 31-6;

[0048] In this embodiment, the complete audio data is 24 bits, each TDM channel is 32 bits, the bit count is counted from 0 to 31, and it starts counting from 0 again after exceeding 31, and 31-6 is from the moment when the bit count of the previous channel is 31 to the moment when the bit count of the next channel is 6. In fact, each TDM line has obtained the complete 24-bit audio data when the bit count is 23, and the TDM0 data is written into the first-level cache when the bit count is 31, the TDM1 data is written into the first-level cache when the bit count is 0, ..., and the TDM7 data is written into the first-level cache when the bit count is 6 (because the cache is the same, the 8 TDM lines cannot be written at the same time);

[0049] The time point for writing to the L2 cache is after all the 64 channels of each sample have been written to the first level. The Ethernet clock domain (frequency is 125MHz) is used to read the L1 cache and write to the L2 cache. This needs to be written 64 times, and the time is about 6 clock cycles of the TDM clock domain, which is equivalent to the time period of 7-12 TDM bit counts of the first channel data of the next sample, which is written to the L2 cache 64 times.

[0050] The period of the TDM clock domain is 1 / 12.288MHz ≈ 81.3ns, and the period of the Ethernet clock domain is 1 / 125MHz = 8ns. Therefore, the total time required to write 64 times in the 125MHz clock domain is 64 * 8ns = 512ns.

[0051] 512ns (the time required to write 64 times in the 125MHz clock domain) is equivalent to the number of cycles in the TDM clock domain: 512ns / 81.3ns ≈ 6.296 cycles. Due to approximations in calculations and slight differences in actual situations, this time period can be considered to be equivalent to 6 clock cycles in the TDM clock domain.

[0052] The data in the secondary cache is updated when each sample data is written (i.e., the RTP packaging moment). The data storage order from low address to high address is: the 1st channel data of the 1st sampling point, the 2nd channel data of the 1st sampling point, ..., the 64th channel data of the 1st sampling point, the 1st channel data of the 2nd sampling point, the 2nd channel data of the 2nd sampling point, ..., the 64th channel data of the 2nd sampling point, ..., the 1st channel data of the 48th sampling point, the 2nd channel data of the 48th sampling point, ..., the 64th channel data of the 48th sampling point. After storing the data of the 48th sampling point, the data of the 49th sampling point will be overwritten to the position of the 1st sampling point, and the cycle will be repeated according to this rule. When the RTP packaging module takes data, the first packaging starts when the 48th sampling point is completed (0-2 RTP streams are packaged each time), and the start address of the read data source is the start address of the 1st sampling point, and the end address is the end address of the 48th sampling point; when the 49th sampling point is completed, the second packaging starts, and the start address of the read data source is the start address of the 2nd sampling point, and the end address is the end address of the 1st sampling point, which is equivalent to moving the read pointer back one sampling point, and the cycle continues in this way.

[0053] The data in the secondary cache is stored continuously in the order of sampling points and channels. The 64 channel data of each sampling point are arranged in sequence. After all channel data of a sampling point are completed, the data storage of the next sampling point begins. When the data of the 48th sampling point is stored, the data of the 49th sampling point will overwrite the position of the 1st sampling point, and the cycle continues.

[0054] It is understandable that when the 49th sampling point is completed, according to the logic of circular storage, the new data (49th sampling point) has covered the position of the 1st sampling point in the buffer. At this time, in order to pack the next RTP packet, the read pointer needs to start reading from the start address of the 2nd sampling point, and the end of the buffer is the 48th sampling point. Since the 49th sampling point is stored at the position of the original 1st sampling point, that is, "the end address is the end address of the 1st sampling point" in this application actually refers to the position of the "new" 49th sampling point after the loop, that is, the position before the 1st sampling point is covered, but considering that it is a circular storage, it can be understood as the logical "end" of the data packaged this time, that is, the latest data written.

[0055] This embodiment provides a method for packaging RTP streams at sampling points: the first 16 sampling points are packaged with 2 RTPs each time, and the next 32 sampling points are packaged with 1 RTP each time;

[0056] That is, 2 RTPs are packaged each time from the 1st sampling point to the 16th sampling point, and 1 RTP is packaged each time from the 17th sampling point to the 48th sampling point; 2 RTPs are packaged each time from the 49th sampling point to the 64th sampling point, and 1 RTP is packaged each time from the 65th sampling point to the 96th sampling point, and so on.

[0057] The conventional system requires that a maximum of 64 RTPs be packaged. 64 / 48 is rounded up to 2, and the remainder is 16. That is, 2 RTPs can be packaged at each of the first 16 sampling points, and 1 RTP can be packaged at each of the next 32 sampling points.

[0058] In another possible implementation, the first 16 sampling points may be packaged into one RTP each time, and the following 32 sampling points may be packaged into two RTPs each time.

[0059] To summarize, TDM data is written into the L1 cache multiple times at the end of each channel, and is read out from the L1 cache multiple times and written into the L2 cache at the end of each sampling. RTP message packaging starts each time writing into the L2 cache is completed. The packaged 48 sampling point data are the 48 sampling points before the current packaging time. Through cyclic access to the cache, only the latest 48 sampling point data can be retained.

[0060] The RTP packaging module is used to encapsulate the audio data in the secondary cache of the cache module according to the RTP protocol to obtain an RTP data packet;

[0061] Before the audio data enters the UDP / IP protocol stack module, the RTP packaging module fills the header data in front of a frame of audio data according to the RTP protocol to obtain an RTP data packet. Similarly, the RTP header data is parsed from the data received from the UDP protocol stack to obtain the audio data.

[0062] The system further comprises an audio transmission routing table, in which the corresponding relationship between audio data and audio channels in the RTP stream is stored, and is used to determine the transmission path of the audio signal (audio data).

[0063] Assuming there are 64 RTP streams in total, the audio transmission routing table sets 64 64-bit data, each 64-bit consists of 8 bytes, corresponding to the 8 channels of each RTP stream, where each byte indicates which audio input this channel comes from, 1-64 is valid, 0 means no source, that is, the data of this channel is invalid. The audio transmission routing table is stored in non-volatile memory, such as Flash memory. Every time the system is powered on, the audio transmission routing table will be re-read from Flash to ensure that the system uses the latest configuration.

[0064] According to the audio sending routing table, the audio data in the secondary cache is packaged into multiple RTP streams according to the mapping relationship specified in the routing table.

[0065] The UDP / IP protocol stack module is connected to the RTP packaging module to realize the message packaging and parsing of the MAC layer, IP layer, and UDP layer, and is used to encapsulate the packaged RTP stream into UDP / IP and then send it out through the Gigabit Ethernet interface; this is the existing technology and will not be repeated here.

[0066] An RTP data packet can contain 1 to 8 audio channels. The UDP / IP protocol stack module can send multiple different Ethernet data packets at the same time, and the audio data, destination IP address, and destination port can be different from each other. Such multiple Ethernet data packets are usually called multiple RTP streams.

[0067] The Gigabit Ethernet interface is an RGMII interface, which is connected to an Ethernet physical layer transceiver chip (PHY) and is used to send and receive Ethernet data packets.

[0068] Embodiment 2

[0069] refer to Figure 3 This embodiment provides a multi-channel RTP delay equalization sending method based on FPGA, based on the multi-channel RTP delay equalization sending system based on FPGA in Embodiment 1, including the following steps:

[0070] S1, collecting audio data of multiple audio channels through an audio collection module;

[0071] S2, the audio data of each audio channel is written into the primary cache multiple times according to the TDM data line sequence at the end of each channel, and is read out from the primary cache multiple times according to the channel sequence and written into the secondary cache at the end of each sampling, and when each writing into the secondary cache is completed, the audio data in the secondary cache is packaged into multiple RTP streams according to the mapping relationship specified in the routing table according to the audio sending routing table;

[0072] The conventional way to send multi-channel RTP is to set the AB double buffer interval. The A buffer and the B buffer are generally set to the same size. All audio data collected in a unit time (such as 1ms) is written into the A buffer, and then all audio data in the next unit time is written into the B buffer. At the same time, the audio data in the A buffer is packaged into multiple RTP streams according to the content of the routing table and sent in sequence. Since multiple RTP streams must be sent in sequence, each RTP stream has a maximum of 1152 bytes of audio data, and the maximum length of the entire packet is 1218 bytes (including the minimum frame interval of 12 bytes). The time it takes to send is about 10us. The sending delay of the first RTP stream is 1ms (starting from the first audio sample in the packet), and the sending delay of the 64th RTP stream is 1.63ms. In order to balance the delay of each RTP stream and reduce the peak network bandwidth demand, multiple RTP streams are not all sent at the same unit time position. There are 48 sampling points in 1ms. 0-2 RTP streams are sent at each sampling point. The interval between sampling points is about 20.8us. At most two largest RTP streams can be sent. 0-96 RTP streams can be sent at 48 sampling points. The data source of the sent RTP stream is the audio data of a total of 48 sampling points from this sampling point. In this way, the packaging delay of each RTP stream is 1ms-1.01ms, which is more balanced than the delay packaged at the same time and has a lower average delay. It also balances the network bandwidth usage, and when multiple devices are sending, lower requirements are placed on the switch's peak forwarding performance.

[0073] Specifically, the first-level cache is a RAM with a width of 24 bits and a depth of 64 bits, which can accommodate data of 1 sampling point of all 64 channels at most; the second-level cache is a RAM with a width of 24 bits and a depth of 4096 bits, which can accommodate data of 48 sampling points of all 64 channels at most.

[0074] A sampling point refers to a discrete point in time of an audio signal. In digital audio processing, a continuous audio signal is sampled at a certain frequency (sampling rate), and each sampling point represents the amplitude value of the audio signal at a specific moment.

[0075] The RTP packaging module takes data from the secondary cache, and the data in the secondary cache is written by the primary cache. The reason why it is not written directly by the collected data is that the data of each channel of a sampling point is collected at different times (the TDM interface has a channel arrangement order), but the cached data cannot be changed during the sampling time from each RTP packaging moment to the completion of the transmission. Therefore, a primary cache is required to cache the data of the next sampling point first, and write the data of the primary cache to the secondary cache at one time when this sampling point ends. The time points of writing the primary cache are staggered for each TDM data line, that is, the writing is delayed backwards in sequence, and the writing is divided into 8 times using the time period of TDM bit count 31~6.

[0076] The time point for writing to the L2 cache is after all the 64 channels of each sample have been written to the first level. The Ethernet clock domain (frequency is 125MHz) is used to read the L1 cache and write to the L2 cache. This needs to be written 64 times, and the time is about 6 clock cycles of the TDM clock domain, which is equivalent to the time period of 7 to 12 TDM bit counts of the first channel data of the next sample, which is written to the L2 cache 64 times.

[0077] The data in the secondary cache is updated when each sample data is written (i.e., the RTP packaging moment). The data storage order from low address to high address is: the 1st channel data of the 1st sampling point, the 2nd channel data of the 1st sampling point, ..., the 64th channel data of the 1st sampling point, the 1st channel data of the 2nd sampling point, the 2nd channel data of the 2nd sampling point, ..., the 64th channel data of the 2nd sampling point, ..., the 1st channel data of the 48th sampling point, the 2nd channel data of the 48th sampling point, ..., the 64th channel data of the 48th sampling point. After storing the data of the 48th sampling point, the data of the 49th sampling point will be overwritten to the position of the 1st sampling point, and the cycle will be repeated according to this rule. When the RTP packaging module takes data, the first packaging starts when the 48th sampling point is completed (0-2 RTP streams are packaged each time), and the start address of the read data source is the start address of the 1st sampling point, and the end address is the end address of the 48th sampling point; when the 49th sampling point is completed, the second packaging starts, and the start address of the read data source is the start address of the 2nd sampling point, and the end address is the end address of the 1st sampling point, which is equivalent to moving the read pointer back one sampling point, and the cycle continues in this way.

[0078] To summarize, TDM data is written into the L1 cache multiple times at the end of each channel, and is read out from the L1 cache multiple times and written into the L2 cache at the end of each sampling. RTP message packaging starts each time writing into the L2 cache is completed. The packaged 48 sampling point data are the 48 sampling points before the current packaging time. Through cyclic access to the cache, only the latest 48 sampling point data can be retained.

[0079] S3, encapsulate multiple RTP streams into UDP / IP and send them out through the Gigabit Ethernet interface;

[0080] The present invention realizes decentralized packaging and sending of RTP stream data through temporary storage in the first-level cache and cyclic update of the second-level cache, realizes equalization of RTP stream delay and network bandwidth occupancy, reduces the peak forwarding performance requirements of switches in a multi-device environment, and effectively optimizes the problem of network congestion.

[0081] It will be appreciated by those skilled in the art that all or some of the steps and systems in the methods provided above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components may be implemented as a microprocessor, such as a central processing unit, a digital signal processor, or software executed by a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-channel RTP delay equalization sending system based on FPGA, characterized in that: The FPGA includes an audio acquisition module, a cache module, an RTP packaging module, a UDP / IP protocol stack module and a Gigabit Ethernet interface: The audio acquisition module is used to collect audio data of M audio channels; The cache module is connected to the audio acquisition module and the RTP packaging module. The cache module includes a primary cache and a secondary cache. The audio data of the audio channel is written into the primary cache multiple times at the end of each channel, and is read from the primary cache multiple times and written into the secondary cache at the end of each sampling. RTP message packaging starts each time the writing into the secondary cache is completed; The RTP packaging module is used to package the audio data in the secondary cache according to the audio transmission routing table to obtain multiple RTP streams; When the RTP packaging module takes data, the first packaging starts when the 48th sampling point is completed. The starting address of the data source is the starting address of the 1st sampling point, and the ending address is the ending address of the 48th sampling point. When the 49th sampling point is completed, the second packaging starts. The starting address of the data source is the starting address of the 2nd sampling point, and the ending address is the ending address of the 1st sampling point. This is equivalent to moving the read pointer back one sampling point, and the cycle is repeated according to this rule. The UDP / IP protocol stack module is connected to the RTP packaging module, which realizes the packet packaging and parsing of the MAC layer, IP layer, and UDP layer, and is used to perform UDP / IP encapsulation on the packaged RTP stream and then send it out through the Gigabit Ethernet interface; The Gigabit Ethernet interface is connected to the Ethernet physical layer transceiver chip and is used to send and receive Ethernet data packets.

2. The multi-channel RTP delay equalization sending system according to claim 1, characterized in that: The audio acquisition interface of the audio acquisition module is a TDM interface, which is composed of multiple TDM data lines, a bit clock, and a sampling start pulse; the sampling start pulse in the TDM interface is the starting time of each sampling point.

3. The multi-channel RTP delay equalization sending system according to claim 1, characterized in that: The data of the secondary cache is updated when each sampling data is written. The time when each sampling data is written is the RTP packaging time. The data storage order from low address to high address is: the first channel data of the first sampling point, the second channel data of the first sampling point, ..., the 64th channel data of the first sampling point, the first channel data of the second sampling point, the second channel data of the second sampling point, ..., the 64th channel data of the second sampling point, ..., the first channel data of the 48th sampling point, the second channel data of the 48th sampling point, ..., the 64th channel data of the 48th sampling point. After storing the data of the 48th sampling point, the data of the 49th sampling point is overwritten to the position of the first sampling point, and this rule is repeated.

4. The multi-channel RTP delay equalization sending system according to claim 2, characterized in that: The first-level cache is a RAM with a width of 24 bits and a depth of 64 bits, which can accommodate data of 1 sampling point of 64 channels; the second-level cache is a RAM with a width of 24 bits and a depth of 4096 bits, which can accommodate data of 48 sampling points of 64 channels.

5. The multi-channel RTP delay equalization sending system according to claim 4, characterized in that: The TDM clock domain is used for writing to the first-level cache, and the writing time points are staggered according to the order of each TDM data line, that is, the writing is delayed backward in sequence, and the TDM bit count is used to write 8 times from the bit count 31 of the previous channel to the bit count 6 of the next channel; the TDM bit count is 0-31.

6. The multi-channel RTP delay equalization sending system according to claim 5, characterized in that: The method of reading from the primary cache and writing into the secondary cache for multiple times at the end of each sampling is specifically: reading the primary cache using the Ethernet clock domain and writing into the secondary cache for 64 times.

7. The multi-channel RTP delay equalization sending system according to claim 6, characterized in that: The TDM clock domain frequency is 12.288 MHz; the Ethernet clock domain frequency is 125 MHz.

8. The multi-channel RTP delay equalization sending system according to claim 2, characterized in that: The audio acquisition module has 8 TDM data lines.

9. The multi-channel RTP delay equalization sending system according to claim 1, characterized in that: The system further comprises an audio transmission routing table, in which the corresponding relationship between the audio data and the audio channels in the RTP stream is stored, and is used to determine the transmission path of the audio data.

10. A multi-channel RTP delay equalization sending method based on FPGA, based on the multi-channel RTP delay equalization sending system based on FPGA according to any one of claims 1-9, characterized in that: The following steps are involved: S1, collecting audio data of multiple audio channels through an audio collection module; S2, the audio data of each audio channel is written into the primary cache multiple times according to the TDM data line sequence at the end of each channel, and is read out from the primary cache multiple times according to the channel sequence and written into the secondary cache at the end of each sampling, and when each writing into the secondary cache is completed, the audio data in the secondary cache is packaged into multiple RTP streams according to the mapping relationship specified in the routing table according to the audio sending routing table; When the RTP packaging module takes data, the first packaging starts when the 48th sampling point is completed. The starting address of the data source is the starting address of the 1st sampling point, and the ending address is the ending address of the 48th sampling point. When the 49th sampling point is completed, the second packaging starts. The starting address of the data source is the starting address of the 2nd sampling point, and the ending address is the ending address of the 1st sampling point. This is equivalent to moving the read pointer back one sampling point, and the cycle is repeated according to this rule. S3, encapsulate multiple RTP streams into UDP / IP, and then send them out through the Gigabit Ethernet interface.

Images