Universal interface architecture design method of 40G Ethernet for radar signal processing

Abstract

The invention relates to the technical field of radar signal processing, in particular to a universal interface architecture design method of a 40G Ethernet for radar signal processing, a bottom driver of the architecture is a DPDK, a universal interface design is provided, and a user can realize definition and play only according to a definition declaration port. During operation, cross-thread direct connection of the functional entity is realized through the shared memory annular queue, and the functional entity autonomously screens required data, so that intermediate links are eliminated. The architecture supports unpacking and restructuring of oversized and giant data packets and realizes receiving and transmitting of data with any byte length. The method comprises universal function interface design, application protocol header design, data group packet sending design, receiving memory design, receiving single packet data processing design, receiving giant packet data processing design and receiving multi-node synchronous data processing design. According to the method, through protocol packaging, zero-copy optimization and two-stage cache design, the difficulty of synchronization of giant packet processing and multiple computing nodes under the 40G Ethernet is solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of radar signal processing, in particular to a general interface architecture design method for radar signal processing oriented 40G Ethernet. BACKGROUND

[0002] At present, the antenna array scale of newly developed radar products is expanding, the number of channels is increasing, the data volume and algorithm complexity of radar signal processing have also been significantly improved, and various high-performance computing based on data complex scientific computing has put forward higher and higher requirements for the rapid exchange capacity of a large amount of data. After years of continuous development, the standard of 40G Ethernet has been very mature, and the industry chain from optical modules, switches to network cards has been very mature, and the device compatibility is very strong, which can effectively reduce the threshold of network deployment. For the application scene of radar signal processing machine data transmission "high density, large bandwidth, low delay", 40G Ethernet as a high-speed network technology can meet the large-scale data storage requirements of radar signal processing machine and the bandwidth and low delay requirements of massive data exchange of each computing node, and can improve the data processing efficiency and transmission speed. However, the link layer of 40G Ethernet has the challenges of traffic scheduling and congestion control. In the application of radar signal processing, large data communication often presents "burst traffic" characteristics. When the peak traffic of multiple nodes is superimposed, the link buffer space will be quickly exhausted, which is easy to cause "congestion collapse", resulting in a sharp drop in data transmission throughput.

[0003] Therefore, the application of 40G Ethernet in the field of radar signal processing still faces three major challenges. First, when facing super large data packet computing, distributed computing scheme is often needed, and the current network driven passive IP and data packet information cannot meet the synchronization of multiple computing nodes in the distributed computing case. Second, when large data "burst traffic" occurs, it is difficult to meet the high-speed, low-delay stable transmission of each computing node in the radar signal processing machine. Third, the maximum limit of the Ethernet UDP application layer driving protocol for single sending is 64KB, and when large data packet transmission is performed, packet disassembly and packet assembly need to be performed at the application layer, which has poor universality. SUMMARY

[0004] In order to meet the requirements of high-speed, low-delay communication of massive data in radar signal processing applications, the embodiment of the present application proposes a general interface architecture design method for radar signal processing oriented 40G Ethernet, which solves the technical implementation difficulties of giant packet processing and multi-computing node synchronization under 40G Ethernet through protocol encapsulation, zero-copy optimization and two-level cache design.

[0005] To achieve the above object, embodiments of the present application propose a general interface architecture design method for radar signal processing oriented 40G Ethernet, the bottom layer driver of the architecture is DPDK, the method comprises: S1, general functional interface design, address, packet sequence number, node sequence number information involved each time is placed in the initialization process, CPU consumption of information transmission when calling sending function multiple times is reduced; S2, application protocol header design, considering the unpacking, recombination and synchronous summary of super large data packets; S3, data packet sending design, the sending of data packets uses large page memory, only needs to fill data and call DPDK for sending; S4, receiving memory design, when receiving data, an independent receiving thread is used to extract data from the large page memory at full speed to prevent packet loss; S5, receiving single packet data processing design, when receiving single packet data, a queue is applied according to a fixed length, single packet data is copied to a ring queue, and the write index of the ring queue is updated; S6, receiving giant packet data processing design, when receiving giant packet data, the storage of communication frames is performed by fragmentation, the node number is detected first, and then the ring queue is realized from the node; S7, receiving multi-node synchronous data processing design, when receiving multi-node synchronous data, the packet sequence number is used as a key value, and the data of the same packet sequence number of multiple nodes is recombined into a continuous data packet.

[0006] To achieve the above object, embodiments of the present application also propose a computer readable storage medium storing a computer program, the computer program can implement the general interface architecture design method for radar signal processing oriented 40G Ethernet when executed by a processor.

[0007] Optionally, in the DPDK driver, the sending function needs to use integer IP address and port number each time, therefore, these information is provided by a string during initialization, to avoid increasing CPU consumption caused by converting the string to integer each time, the address, packet sequence number, node number information involved each time is placed in the initialization process when designing the general functional interface.

[0008] Optionally, the maximum limit of single sending data of the UDP application layer driver protocol of the 40G Ethernet is 64KB, when the required sending data is greater than 64KB, a general application protocol header needs to be designed to implement unpacking and packet operation, the application protocol header comprises ID number of a target node, ID number of a source node, packet length, fragment length, fragment number, fragment sequence number and data payload.

[0009] Optionally, the process of sending data specifically comprises: calculating the data fragment number according to the size of the data packet to be sent; obtaining sending memory; filling a data frame header; filling data payload; calling a general data sending interface function to complete the sending of the data packet.

[0010] Optionally, since the data packet sending uses large page memory, large page memory needs to be applied in advance during initialization, and DPDK is called for sending after data filling is completed.

[0011] Optionally, the receiving memory is a ring queue data receiving memory, including a packet information storage area, a single packet data storage area, and a jumbo packet data storage area; when receiving data, an independent receiving thread is used to extract data from the DPDK driver at full speed to prevent packet loss, and after receiving the data, it is necessary to judge whether to unpack, if the fragment is 0, it indicates single packet data, and unpacking is not needed, if the fragment is greater than 0, it indicates that unpacking is needed.

[0012] Optionally, for single packet data, it is directly written into the ring queue memory; when single packet data is received, a queue is applied according to the fixed length MTU, the data is copied to the ring queue, the write index of the ring queue is updated, and the frame header contained in the data packet is still retained in the data storage area; when the user calls the general receiving interface function, the data address points to the actual data area; in order to meet the application scenario of the user needing to obtain frame header information, the general receiving interface function is overloaded to provide a function interface capable of obtaining the frame header.

[0013] Optionally, in the case that the received data is jumbo packet data, the source key value of the jumbo packet needs to be obtained, the jumbo packet storage area is searched according to the source key value, when the source key value matches successfully, the jumbo packet is inserted, and the packet information is updated to the ring queue.

[0014] Optionally, the source key value is Key; Key = NodeId + PackId; wherein NodeId is a node number, and PackId is a packet serial number.

[0015] The embodiment of the application provides a general interface architecture design method for radar signal processing of 40G Ethernet. In a radar signal processing application scenario with large data and strong real-time, through encapsulation of a driving protocol, zero-copy optimization and two-level cache design, technical implementation difficulties of jumbo packet processing and multi-computing node synchronization of DPDK under 40G Ethernet are solved, user operation is simplified through the same interface, high performance and ease of use are considered, and bottom layer technical support is provided for radar signal processing and other high-speed data flow scenes. Compared with the prior art, the application has the following innovations.

[0016] First, large packet data transmission and reception are supported, automatic unpacking / packing is realized, a receiving end recombines frame sequences through shared memory+ring queue, the number of copying times is reduced, and transmission efficiency is improved.

[0017] Second, multi-computing node same frame data synchronization is supported, and output results are combined into a continuous memory block.

[0018] Third, based on the zero-copy design, the DPDK memory pool is directly filled when sending, and the transmission delay is reduced through independent threads and two-level cache.

[0019] Fourth, multiple buffers are reserved for each computing node to balance the conflict between resources and real-time performance.

[0020] Fifth, a unified interface package is provided to shield the complexity of the underlying DPDK driver interface. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technical fields, the drawings needed to be used in the description of the embodiments of the present application or the related technology will be briefly introduced below. Obviously, the following drawings are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings. The drawings described herein are only used to explain the present application and do not limit the present application.

[0022] Figure 1 is a flowchart of a general interface architecture design method for radar signal processing of 40G Ethernet provided in an embodiment of the present application; Figure 2 is a schematic diagram of the general interface design provided in an embodiment of the present application; Figure 3 is a schematic diagram of the application protocol header structure provided in an embodiment of the present application; Figure 4 is a schematic diagram of the packet sending provided in an embodiment of the present application; Figure 5 is a schematic diagram of the storage memory structure of single packet data provided in an embodiment of the present application; Figure 6 is a schematic diagram of the storage memory structure of giant packet data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be described in detail below with reference to the drawings. Those skilled in the art can understand that in the embodiments of the present application, many technical details are proposed in order to make the reader better understand. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The following embodiments are divided for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the following embodiments can be combined with each other under the premise of no contradiction.

[0024] One embodiment of the present application provides a general interface architecture design method for radar signal processing of 40G Ethernet, the bottom layer driver of the architecture is DPDK, the following will specifically describe the implementation details of the general interface architecture design method for radar signal processing of 40G Ethernet proposed by the embodiment, the following implementation details are provided for the convenience of understanding, and are not essential for implementing the present solution.

[0025] The general interface architecture design proposed by the embodiment is a high-speed low-latency general solution based on 40G Ethernet DPDK driver for radar signal processing, and the user only needs to define the port according to the definition to achieve the defined use. In runtime, the cross-thread straight-through function entity is realized through the shared memory ring queue, the function entity autonomously selects the required data, and eliminates the intermediate link. It can support super large packet disassembly and reassembly, and realize arbitrary byte length data transmission.

[0026] The specific process of the general interface architecture design method for radar signal processing of 40G Ethernet proposed by the embodiment can be as shown in Figure 1 , which includes: S1, general function interface design, the address, packet number, node number information involved each time is placed in the initialization process, reducing the CPU consumption of information transmission when calling the sending function multiple times.

[0027] In S1, the sending function in the DPDK driver needs to use the integer IP address and port number each time, the user provides these information through a string in initialization, in order to avoid the increase of CPU consumption caused by converting the string to integer each time, the address, packet number, node number and other information involved each time can be placed in the initialization process when designing the interface. The interface design is as shown in Figure 2 .

[0028] S2, application protocol header design, considering the disassembly, reassembly and synchronous summary of super large data packets.

[0029] Figure 3 The general application protocol header structure of the general interface architecture design is shown.

[0030] In S2, since the 40G Ethernet UDP application layer driver protocol has a maximum limit of 64KB for single data transmission, when the required data to be sent is greater than 64KB, a general application protocol header needs to be designed to realize the disassembly and packet operation. The application protocol header should include the target node ID number, the source node ID number, the packet length, the fragment length, the fragment number, the fragment sequence number and the data payload.

[0031] S3, data packet sending design, the data packet sending uses the large page memory, only needs to fill the data and calls the DPDK to send.

[0032] Figure 4 The process of data packet sending is shown.

[0033] In an example, the process of sending data specifically includes: calculating the data fragment number according to the size of the data packet to be sent; obtaining the sending memory; filling the data frame header; filling the data payload; and calling the general data sending interface function to complete the sending of the data packet. Since the data packet sending uses the large page memory, the large page memory needs to be applied in advance during initialization, and the DPDK is called to send after the data filling is completed.

[0034] S4, receiving memory design, when receiving data, an independent receiving thread is used to extract data from the large page memory at full speed to prevent packet loss.

[0035] In S4, the receiving memory is a ring queue data receiving memory, including a packet information storage area, a single packet data storage area and a jumbo packet data storage area. When receiving data, an independent receiving thread is used to extract data from the DPDK driver at full speed to prevent packet loss, and after receiving the data, it is necessary to judge whether to unpack, if the fragment is 0, it indicates single packet data, and unpacking is not needed, if the fragment is greater than 0, it indicates that unpacking is needed.

[0036] S5, receiving single packet data processing design, when receiving single packet data, a queue is applied according to a fixed length, the single packet data is copied to the ring queue, and the write index of the ring queue is updated.

[0037] Figure 5 The storage memory structure of single packet data is shown.

[0038] In S5, for single packet data, the data is directly written into the ring queue memory. When receiving single packet data, a queue is applied according to a fixed length MTU, the data is copied to the ring queue, and the write index of the ring queue is updated. The frame header contained in the data packet is still retained in the data storage area. When the user calls the general receiving interface function, the data address points to the actual data area. In order to meet the application scenario of the user needing to obtain frame header information, the general receiving interface function is overloaded to provide a function interface capable of obtaining the frame header.

[0039] S6, receiving jumbo packet data processing design, when receiving jumbo packet data, the fragments are communicated for frame storage, the node number is detected first, and then the ring queue is realized from the node.

[0040] Figure 6 The storage memory structure of jumbo packet data is shown.

[0041] In S6, in the case that the received data is giant packet data, the source key value of the giant packet needs to be obtained, the source key value is searched in the giant packet storage area, and when the source key value is matched successfully, the giant packet is inserted, and the packet information is updated to the ring queue.

[0042] In one example, the source key value is Key; Key = NodeId + PackId; wherein NodeId is the node number, and PackId is the packet sequence number.

[0043] In S7, a multi-node synchronous data processing design is received, when the multi-node synchronous data is received, the packet sequence number is used as the key value, and the data of the same packet sequence number of the multi-node is recombined into a continuous data packet.

[0044] The embodiment provides a general interface architecture design method of a 40G Ethernet for radar signal processing. In a radar signal processing application scene with large data and strong real-time, through encapsulation of a driving protocol, zero-copy optimization and two-level cache design, technical implementation difficulties of giant packet processing and multi-computing node synchronization of DPDK under the 40G Ethernet are solved, user operation is simplified through the same interface, high performance and ease of use are considered, and bottom layer technical support is provided for high-speed data flow scenes such as radar signal processing. Compared with the prior art, the embodiment has the following innovations.

[0045] First, large packet data transmission and reception are supported, automatic unpacking / packing is implemented, frame sequence is recombined by a receiving end through shared memory+ring queue, the number of copying times is reduced, and transmission efficiency is improved.

[0046] Second, multi-computing node same frame data synchronization is supported, and output results are combined into a continuous memory block.

[0047] Third, based on zero-copy design, DPDK memory pool is directly filled when sending, and transmission delay is reduced through an independent thread+two-level cache when receiving.

[0048] Fourth, multiple buffer areas are reserved for each computing node to balance the conflict between resources and real-time performance.

[0049] Fifth, a general unified interface encapsulation is provided to shield the complexity of the bottom layer DPDK driving interface.

[0050] The step division of the above various methods is only for the purpose of clearly describing the technical scheme, and in specific implementation, one step can be combined or some steps can be divided into multiple steps, as long as the same logical relationship is included, and all are within the protection scope of the application. Irrelevant modifications or irrelevant designs added in the algorithm or the flow are within the protection scope of the application as long as the core of the algorithm and the flow is not changed.

[0051] Another embodiment of the present application provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program, when executed by a processor, enables a radar signal processing oriented general interface architecture design method of 40G Ethernet to be implemented.

[0052] That is, a person skilled in the art can understand that all or part of the steps in the above method embodiments can be completed by programs instructing relevant hardware, the programs are stored in a storage medium, and the programs include a plurality of instructions to make a device (such as a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method described in the method embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0053] A person skilled in the art can understand that the above embodiments are specific embodiments of the present application, and in actual application, various changes can be made in form and details without departing from the spirit and scope of the present application. For those skilled in the art, a number of improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements are also regarded as the protection scope of the present application.

Claims

1. A general interface architecture design method for 40G Ethernet for radar signal processing, wherein the underlying driver of the architecture is DPDK, characterized in that, The method includes: S1, a generalized functional interface design, places the address, packet sequence number, and node sequence number information that are needed each time in the initialization process, reducing the CPU consumption of information transmission when the send function is called multiple times; S2, application protocol header design, takes into account the disassembly, reassembly and synchronous summarization of ultra-large data packets; S3, data packet sending design, the sending of data packets uses large pages of memory, only needs to fill the data and call DPDK to send; S4, the receive memory design, uses an independent receive thread to retrieve data from the large page memory at full speed when receiving data to prevent packet loss; S5 is designed for receiving single-packet data. When receiving single-packet data, a queue is requested according to a fixed length, the single-packet data is copied to the circular queue, and the write index of the circular queue is updated. S6 is designed for receiving jumbo packet data. When receiving jumbo packet data, it segments the communication frames for storage, first detects the node number, and then implements a circular queue from the nodes. S7 is designed for receiving multi-node synchronous data. When receiving multi-node synchronous data, the packet sequence number is used as the key value to reassemble data with the same packet sequence number from multiple nodes into a continuous data packet.

2. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, In the DPDK driver, the send function needs an integer IP address and port number for each send. Therefore, this information is provided as a string during initialization. To avoid increasing CPU consumption by converting the string to an integer every time a send operation is performed, the address, packet sequence number, and node number information required each time are placed in the initialization process when designing the generalized functional interface.

3. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, The maximum data transmission limit for a single UDP application layer driver protocol in 40G Ethernet is 64KB. When the required data transmission exceeds 64KB, a generalized application protocol header needs to be designed to implement packet disassembly and reassembly operations. The application protocol header includes the target node ID, the source node ID, packet length, fragment length, number of fragments, fragment sequence number, and data payload.

4. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, The process of sending data specifically includes: Calculate the number of data fragments based on the size of the data packets to be sent; Get the sending memory; Fill in the data frame header; Fill the data payload; Call the general data sending interface function to complete the data packet sending.

5. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, Since the data packet is sent using large pages of memory, large pages of memory need to be allocated in advance during initialization. After the data is filled, DPDK is called to send the data.

6. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, The receiving memory is a circular queue of data receiving memory, including a packet information storage area, a single packet data storage area, and a giant packet data storage area; When receiving data, an independent receiving thread is used to extract data from the DPDK driver at full speed to prevent packet loss. After receiving the data, it is necessary to determine whether to unpack the packet. If the fragmentation value is 0, it means that it is a single packet of data and does not need to be unpacked. If the fragmentation value is greater than 0, it means that the packet needs to be unpacked.

7. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, For single-packet data, it is directly written into the memory of the circular queue; When a single data packet is received, a queue is requested according to the fixed-length MTU, the data is copied to the circular queue, the write index of the circular queue is updated, and the frame header contained in the data packet is still retained in the data storage area. When a user calls the general receiving interface function, the data address is pointed to the actual data area; To meet the application scenarios where users need to obtain frame header information, the general receiving interface function is overloaded to provide a function interface that can obtain the frame header.

8. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, When the received data is jumbo packet data, it is necessary to obtain the source key value of the jumbo packet, search in the jumbo packet storage area according to the source key value, and insert the jumbo packet when the source key value is successfully matched, and at the same time update the packet information to the circular queue.

9. The method for designing a general interface architecture for 40G Ethernet for radar signal processing according to claim 1, characterized in that, The source key value is Key; Key = NodeId + PackId; Where NodeId is the node number and PackId is the packet sequence number.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can implement a general interface architecture design method for 40G Ethernet for radar signal processing as described in any one of claims 1 to 9.

Images