Method for receiving and transmitting big data by TSI721 under Linux system

Abstract

According to the method for receiving and transmitting the big data through the TSI721 under the Linux system, DMA data sent by other nodes are received or sent to other nodes through a continuous reserved memory, the limitation of the 4MB size of a kernel is broken through, the whole space can be accessed only by opening a window mapping relation and a virtual address, and the operation is simple and convenient. The reserved memory receives DMA data sent by other nodes or sends the DMA data to other nodes, and the physical address is continuous.

Description

Technical Field

[0001] The present invention relates to the technical field of computer communications, and in particular to a method for TSI721 to send and receive big data under a Linux system. Background Art

[0002] TSI721 is a high-performance PCI Express (PCIe) to RapidIO Gen2 protocol conversion bridge product launched by Integrated Device Technology, Inc. (IDT). It supports 16Gbps data transmission speed, provides 8 direct memory access (DMA) and 4 information engines / channels, each channel can achieve 16Gbps data transmission speed. This design enables TSI721 to realize the allocation of multiple engines in a single core in a multi-core multi-threaded system, thereby simplifying the development of system-level software to the greatest extent. In addition, TSI721 also provides a system-level interconnection function based on Intel processors in a distributed multi-core processing environment. Compared with other interconnection products (such as 10GigE), it has obvious advantages in processing power, latency and overall system-level energy management.

[0003] TSI721 is designed to overcome the difficulty of using and expanding PCIe non-transparent bridge technology, while maintaining the high performance, low latency and high flexibility of RapidIO systems in wireless infrastructure, defense, imaging and industrial markets. By realizing the conversion between PCIe protocol and RapidIO, TSI721 enables RapidIO-based systems to be applied in a wider range of scenarios, while taking advantage of RapidIO's advantages in backplane interconnection.

[0004] In addition, TSI721 supports multiple operating systems, including Linux and Windows, and provides corresponding software support for these systems. For Linux systems, the driver of TSI721 is an essential software component that allows the operating system to recognize and effectively interact with the hardware. The correct writing and optimization of the driver is essential to ensure the efficient operation of the TSI721 bridge chip in a specific environment. For the real-time operating system VxWorks, TSI721 also shows good compatibility, which is suitable for environments with strict requirements on real-time performance, such as aerospace, industrial automation and other fields. Summary of the invention

[0005] The technical solution of the present invention aims at: 1. Using continuous reserved memory to receive DMA data sent by other nodes or to send DMA data to other nodes, with continuous physical addresses, and only one virtual address is needed to access the entire space as long as a window mapping relationship is opened, and an efficient and convenient interface is provided; 2. User memory exceeds the kernel 4MB size limit and can reach 1GB in size, and the sending rate is higher than that of user-state memory allocated by malloc.

[0006] The technical solution of the present invention provides a method for TSI721 to send and receive big data under Linux system, comprising the following steps:

[0007] A reserved memory of a preset size is reserved in the tail Linux system memory, and a mapping relationship is established with a preset function;

[0008] Call the preset function to map the reserved memory space to the virtual address;

[0009] When receiving DMA data sent by other nodes: a mapping between a virtual address and a RapidIO address is established through the interface. The DMA data sent by other nodes is stored in the virtual address through the interface, and the DMA data is sent from the virtual address to the RapidIO address.

[0010] When sending DMA data to other nodes and the destination address of RapidIO address: use the virtual address as the sending user buffer, obtain the data to be sent from the user or directly generate the data to be sent, and use the physical address and offset corresponding to the virtual address as parameters to call the interface and send it to the destination address of other nodes and RapidIO address.

[0011] Preferably, the preset size does not exceed 1 GB.

[0012] Preferably, the tail Linux system memory is stored in a Linux system end address.

[0013] Preferably, after the DMA data is sent from the virtual address to the RapidIO address, the user queries the DMA data stored in the virtual address to verify whether the DMA data sent by other nodes, the DMA data stored in the virtual address and the DMA data received by the RapidIO address are consistent. If they are consistent, the mapping between the virtual address and the RapidIO address is successful.

[0014] The technical solution of the present invention proposes a method for TSI721 to send and receive big data under a Linux system. By using continuous reserved memory to receive DMA data sent by other nodes or send DMA data to other nodes, it is possible to break through the kernel's 4MB size limit. As long as a window mapping relationship and a virtual address are opened, the entire space can be accessed. The reserved memory receives DMA data sent by other nodes or sends DMA data to other nodes, and has continuous physical addresses. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of TSI721 configuration and receiving process;

[0016] Figure 2 Write a flowchart for TSI721 to use the reserved memory as the user data area to send DMA data. DETAILED DESCRIPTION

[0017] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0018] The method of reserving memory under Linux is as follows:

[0019] First, after starting the system, you need to use the command cat / proc / iomem | grep System RAM to find the tail of the system memory: 100000000-10dffffff: System RAM.

[0020] Then, 1GB of memory is reserved at the end. The end address is chosen because this address is basically not used when the system starts. The reserved starting address is 0x10dffffff-0x40000000=0xcdffffff.

[0021] Add memmap=1024M\$0xcdffffff at the end of the kernel startup item in the kernel startup file / boot / efi / BOOT / grub.cfg. The verification method is: after restarting the system, run cat / proc / iomem|grep System RAM. If the memory allocation at the end is c0000000-cdffffff:System RAM:, it is successful.

[0022] The user state can open the TSI721 device, map the reserved memory space through the mmap function, and give the virtual address for use. The pseudo code is as follows:

[0023] int fd=rio_mport_open(mport_id,0);

[0024] void*ibmap=mmap(NULL,0x40000000,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0xce000000);

[0025] If the ibmap virtual address is not empty, it means that the mapping is successful; the user can copy or read user data to this address.

[0026] The TSI721 receive window uses the reserved memory as follows:

[0027] Before TSI721 receives DMA data from other nodes, it needs to map the window first and map the reserved memory address (virtual address) and RapidIO address. For example, if the RapidIO address is 0xB0000000, and the reserved memory first address is 0xce000000, the relationship between the two is mapped through the interface rio_ibwin_map. The RapidIO address is the destination address for DMA transmission. After TSI721 receives the data, it directly moves the data at address 0xB0000000 to the reserved memory at 0xce000000. The user can query the ibmap virtual address to verify whether the DMA data received by the RapidIO address and the DMA data sent by other nodes are consistent; if the data is consistent, it means that the mapping window is successful.

[0028] The method of using reserved memory to send data by TSI721 is as follows:

[0029] Before sending data, TSI721 needs to use the 1GB memory starting from the reserved memory address 0xce000000 as the sending user buffer. Users can copy data into it first, or directly generate data in this memory; then call the rio_dma_write_d function interface to directly use the reserved memory physical address and offset as parameters to send to the destination device and RapidIO destination address. For tasks with high real-time requirements and tasks that do not allow frequent fragmentation and splitting of data packets, using reserved memory can ensure that large amounts of data are sent at one time.

[0030] The embodiment of the present invention is applicable to Linux systems: Ubuntu, Kylin system, Centos system and Tongxin system, etc. The official TSI721 driver package is kernel-rapidio.tar.gz.

[0031] like Figure 1As shown in the figure, after loading the official TSI721 driver, you can write a test case that uses the reserved memory to open a receiving window. First, call rio_mport_open to open the TSI721 device

[101] , then call rio_ibwin_map to bind the reserved memory first address 0xce000000 and the RapidIO address 0xB0000000 as a mapping window

[102] ; then the mmap function converts the reserved memory physical address into a virtual address for the user

[103] ; at this time, other nodes can send DMA data to the RapidIO address 0xB0000000, and the virtual address content is printed regularly at this time, and it will be found that the data is consistent with the sending end

[105] ;

[0032] like Figure 2 As shown, the user can write and use the reserved memory as the user data area to send DMA data. First, call rio_mport_open to open the TSI721 device

[201] . Then, the mmap function converts the reserved memory physical address 0xce000000 into a virtual address for the user to use

[202] . The user can copy data into the virtual address or directly generate data in this memory

[203] . Then, call the key function rio_dma_write_d to send the data opened at any offset position in this memory to the specified node. If the return value is not 0, it means that there is an error in the transmission. If it is 0, it is successful

[204] .

[0033] Embodiment 1:

[0034] TSI721, as the receiving end, opens up 1GB memory as a window. The transmitting end radar FPGA needs to send 32MB of data at a time. This application requires dividing the 1GB window into 32 queues of 32MB in size. The FPGA sends data to the first address of queues 0-31 in a loop. After each transmission, it sends a doorbell to notify the user to take the data away. The rate can reach the full bandwidth.

[0035] Embodiment 2:

[0036] As the transmitter, TSI721 uses 256MB of reserved memory as a user buffer. Radar data requires a packet of 256MB. The packet type is Swrite+Nwrite_R. Except for the last packet which is of Nwrite_R type, the previous ones are all of Swrite type, which makes it easier for FPGA to determine when the data ends.

[0037] The embodiment of the present invention proposes a method for TSI721 to send and receive big data under a Linux system. By using continuous reserved memory to receive DMA data sent by other nodes or send DMA data to other nodes, it is possible to break through the kernel's 4MB size limit. As long as a window mapping relationship and a virtual address are opened, the entire space can be accessed. The reserved memory receives DMA data sent by other nodes or sends DMA data to other nodes, and has continuous physical addresses.

Claims

1. A method for sending and receiving big data using TSI721 under Linux system, characterized in that: The following steps are involved: A reserved memory of a preset size is reserved in the tail Linux system memory, and a mapping relationship is established with a preset function; Call the preset function to map the reserved memory space to the virtual address; When receiving DMA data sent by other nodes: a mapping between a virtual address and a RapidIO address is established through the interface. The DMA data sent by other nodes is stored in the virtual address through the interface, and the DMA data is sent from the virtual address to the RapidIO address. When sending DMA data to other nodes and the destination address of RapidIO address: use the virtual address as the sending user buffer, obtain the data to be sent from the user or directly generate the data to be sent, and use the physical address and offset corresponding to the virtual address as parameters to call the interface and send it to the destination address of other nodes and RapidIO address.

2. A method for sending and receiving big data by TSI721 under Linux system as claimed in claim 1, characterized in that: The preset size does not exceed 1 GB.

3. The method for sending and receiving big data by TSI721 under Linux system as claimed in claim 1, characterized in that: The tail Linux system memory is stored in the Linux system end address.

4. The method for sending and receiving big data by TSI721 under Linux system as claimed in claim 1, characterized in that: After the DMA data is sent from the virtual address to the RapidIO address, the user queries the DMA data stored in the virtual address to verify whether the DMA data sent by other nodes, the DMA data stored in the virtual address and the DMA data received by the RapidIO address are consistent. If they are consistent, the mapping between the virtual address and the RapidIO address is successful.

Images