Beamforming system and method based on GPU distributed cluster

Abstract

The present application provides a beamforming system and method based on a GPU distributed cluster. The system according to the present application includes: a signal acquisition system, a switching network, a distributed beamforming system, and an astronomical signal processing backend; the signal acquisition system includes M FPGA boards, which collect astronomical data and pre-process the astronomical data through the M FPGA boards, and the astronomical data is data including P frequency points of N antennas; the distributed beamforming system receives N antenna data through G GPU nodes and performs GPU beamforming parallel calculations to synthesize B beams, and sends the B beams as beamforming results to the astronomical signal processing backend; the astronomical signal processing backend receives the beamforming results through L server nodes. The technical solution provided by the present application can reduce the system cost while improving efficiency and ensuring real-time performance.

Description

Technical Field

[0001] This document relates to the field of digital beamforming technology, and more particularly to a beamforming system and method based on a GPU distributed cluster. Background Art

[0002] Digital beamforming is a very important technology in astronomical signal processing. It combines data from multiple antennas or sensors to improve the resolution and signal-to-noise ratio of the observed target. This technology can be used to observe the sky in all directions.

[0003] In the existing technology, GPU has become the first choice for high-performance computing due to its highly parallel computing capabilities. However, with the upgrade of modern astronomical equipment, the volume and complexity of observation data have increased significantly, and traditional GPU computing methods have become difficult to meet the needs of real-time processing. Summary of the invention

[0004] The present invention provides a beamforming system and method based on a GPU distributed cluster, which combines the high-performance parallel computing capability of the GPU and the scalability of the distributed cluster to achieve signal beamforming using a distributed GPU cluster architecture, thereby solving the problem that traditional GPU computing methods are difficult to meet the needs of real-time processing.

[0005] The present invention provides a beamforming system based on a GPU distributed cluster, comprising:

[0006] Signal acquisition systems, switching networks, distributed beamforming systems, and astronomical signal processing backends;

[0007] The signal acquisition system includes M FPGA boards, and astronomical data is collected and preprocessed through the M FPGA boards. The astronomical data is data including P frequency points of N antennas, where N and P are positive integers greater than or equal to 1.

[0008] The switching network is used to send the pre-processed astronomical data to the distributed beamforming system;

[0009] The distributed beamforming system includes G GPU nodes, which receive N antenna data through the G GPU nodes and perform GPU beamforming parallel calculation to synthesize B beams, and send the B beams as beamforming results to the astronomical signal processing backend, where B and G are both positive integers greater than or equal to 1;

[0010] The astronomical signal processing backend includes L server nodes, and beam synthesis results are received through the server nodes, where L is a positive integer greater than or equal to 1.

[0011] The present invention provides a beamforming method based on a GPU distributed cluster, comprising:

[0012] Collecting astronomical data through M FPGA boards of a signal acquisition system and preprocessing the astronomical data, wherein the astronomical data includes data of P frequency points of N antennas, wherein N and P are positive integers greater than or equal to 1;

[0013] The pre-processed astronomical data is sent to the distributed beamforming system through a switching network;

[0014] Receiving N antenna data through G GPU nodes of a distributed beamforming system and performing GPU beamforming parallel calculation to synthesize B beams, and sending the B beams as beamforming results to the astronomical signal processing backend, where G is a positive integer greater than or equal to 1;

[0015] The beamforming result is received by L server nodes of the astronomical signal processing backend, wherein L and B are both positive integers greater than or equal to 1.

[0016] The present invention provides a beamforming system and method based on a GPU distributed cluster, which utilizes the parallel computing advantages of GPUs and the scalability of cluster architecture, can process a large number of parallel tasks, realize signal beamforming using a distributed GPU cluster architecture, distribute a large amount of observation data to multiple nodes for parallel processing, realize beamforming of part of the GPU data on each self-organizing node, and design the format and sending order of the beamforming results according to the load balancing principle, thereby reducing the system cost while improving the efficiency, ensuring real-time performance, and significantly improving the data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 is a schematic diagram of a beamforming system based on a GPU distributed cluster according to an embodiment of the present invention;

[0019] Figure 2 It is a schematic diagram of a signal acquisition system, a distributed beamforming system and a signal processing backend that are fully connected according to an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a self-organizing structure of each node in a distributed beamforming system according to an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of functions in each node of a distributed beamforming system according to an embodiment of the present invention;

[0022] Figure 5 Schematic diagram of a data transmission and caching method for each node of a distributed beamforming system of the present invention according to an embodiment of the present invention;

[0023] Figure 6 The figure is a flowchart of a beamforming method based on a GPU distributed cluster according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0025] System Example

[0026] According to an embodiment of the present invention, a beamforming system based on a GPU distributed cluster is provided. Figure 1 is a schematic diagram of a beamforming system based on a GPU distributed cluster according to an embodiment of the present invention. Figure 1 As shown, the GPU distributed cluster-based beamforming system of the embodiment of the present invention specifically includes:

[0027] Signal acquisition system 10, switching network 12, distributed beamforming system 14 and astronomical signal processing backend 16;

[0028] The signal acquisition system 10 includes M FPGA boards, which collect astronomical data and pre-process the astronomical data. The astronomical data includes data of N antennas and P frequency points, where N and P are positive integers greater than or equal to 1.

[0029] The embodiment of the present invention divides data and constrains the format of the signal acquisition system 10 according to common requirements in the field of astronomical radio. The common requirement is that astronomical signals are usually divided into dual-polarization signal data of a frequency band, and digital beam synthesis and astronomical data preprocessing between dual polarizations are performed on the dual-polarization data.

[0030] The signal acquisition system 10 completes signal acquisition and preprocessing, and packages the preprocessed data into a VDIF format, and sends it to the distributed beamforming system through the 100G network port. The signal preprocessing algorithm includes delay compensation, PFB, FFT and dynamic truncation.

[0031] The astronomical data in the embodiment of the present invention is generally composed of N antennas, P frequency points, dual polarization or single polarization, and the signal acquisition system is usually collected by M FPGA boards. j FPGA board acquisition The data of the signal acquisition system in the embodiment of the present invention is connected to the distributed beamforming system 14 through the switching network 12 in a fully connected manner, such as Figure 2 FIG. 1 is a schematic diagram showing a signal acquisition system 10 , a distributed beamforming system 14 , and an astronomical signal processing backend 16 that are fully connected according to an embodiment of the present invention.

[0032] In the embodiment of the present invention, j FPGA board acquisition The P frequency point data of the antenna is preprocessed, and the preprocessed data is packaged and encapsulated into VDIF format and sent to the distributed beamforming system through the switching network, where: Obtained by the following formula:

[0033] (1);

[0034] Wherein, M is a positive integer greater than or equal to 1.

[0035] The switching network 12 is used to send the pre-processed astronomical data to the distributed beamforming system;

[0036] The distributed beamforming system 14 includes: G GPU nodes, which receive N antenna data through the G GPU nodes and perform GPU beamforming parallel calculation to synthesize B beams, and send the B beams as beamforming results to the astronomical signal processing backend, where B and G are positive integers greater than or equal to 1;

[0037] The distributed beamforming system 12 in the embodiment of the present invention is composed of G GPU servers, which synthesize B beams. The embodiment of the present invention uses frequency point division to decompose the tasks served by each GPU, that is, the first of the G GPU servers h GPU processing Q h Dual-polarization or single-polarization data at each frequency point, Q h Obtained by the following formula:

[0038] (2);

[0039] Under massive data bandwidth, the number of GPU nodes required will be relatively large, while general commercial servers are relatively expensive. In the embodiment of the present invention, each node of the distributed beamforming system adopts a self-organizing architecture, and each node is composed of a motherboard, a CPU, a memory, an input / output network card and a GPU, forming a set of cost-effective distributed clusters, which greatly reduces the system cost.

[0040] Each node of the GPU-based distributed beamforming system in the embodiment of the present invention is assigned a number 1, 2, 3, ..., G-1, G. Each node is interconnected through a 10 Gigabit network, and the node number is automatically identified according to the local area network IP address.

[0041] The astronomical signal data is transmitted to the distributed beamforming system through the signal acquisition system and the switching network. The embodiment of the present invention completes the hth GPU node of the distributed beamforming system. Q h The steps of receiving data from N antennas at different frequencies, parallel computing of GPU beamforming, astronomical data preprocessing and result transmission are as follows. In the embodiment of the present invention, each node completes Q h The B beams of the frequency points are synthesized as follows:

[0042] Each GPU server is equipped with at least one dual-port network card for data reception. Figure 2 It can be seen that the data of the signal acquisition system 10 in the embodiment of the present invention is connected to the distributed beam synthesis system 14 through the switching network 12 in a fully connected manner, that is, the data of all FPGA boards will transmit part of the frequency point data to each GPU node. The connection method of the two network ports of the FPGA board and a single GPU node is: the N / 2 channels Q frequency point data composed of the first half of the FPGA board are transmitted to the network port 1 of the server, and the N / 2 channels Q frequency point data composed of the second half of the FPGA board are transmitted to the network port 2 of the server.

[0043] After each server receives data, two memory buffer pools are designed to cache the data. Each buffer pool has a one-to-one correspondence with the network port, that is, buffer pool 1 is used to cache the data received by network port 1, and buffer pool 2 is used to cache the data received by network port 2. The data of the two buffer pools is combined to form a complete N-way antenna. Q h Frequency data.

[0044] After receiving and caching data for a period of time, the data is transferred to the GPU memory, and the parallel calculation of beamforming and astronomical signal preprocessing are completed in the GPU, that is, the Stokes integral parallel calculation is performed on the beamforming results.

[0045] After completing the parallel calculation of beamforming and Stokes integral in the GPU, the results are packaged into a fixed format and sent to the astronomical signal processing backend, thus realizing the real-time operation of beamforming of the entire system.

[0046] The astronomical signal processing backend 16 includes: L server nodes, through which beam synthesis results are received, where L is a positive integer greater than or equal to 1.

[0047] The astronomical signal processing backend 16 in the embodiment of the present invention is usually composed of multiple server nodes, and the i-th server node usually processes P frequency points. beam data, where Obtained by the following formula:

[0048] (3).

[0049] Where L is the number of astronomical signal processing backend server nodes.

[0050] When the distributed beamforming system 14 in the embodiment of the present invention sends the results, it rearranges the results in each node and divides the beamforming results into L groups according to the number of server nodes L of the astronomical signal processing backend 16. Each group includes Q h Frequency points The L groups of data are sent to the L servers at the astronomical signal processing backend in sequence.

[0051] Taking into account the load balancing of the switch and ensuring that each server at the back end of astronomical signal processing can receive the required data at the same time, the distributed beamforming system of the embodiment of the present invention determines to start sending from the i-th group of result data according to its own node number when sending the result, and the range of i is 1, 2, ..., L.

[0052] A specific implementation of the embodiment of the present invention is as follows:

[0053] The signal acquisition system in the embodiment of the present invention is composed of M FPGA boards for acquisition, and each j-th board processes K j The embodiment of the present invention stipulates that the data of the collection end is connected to the distributed beamforming system through a switching network in a fully connected manner.

[0054] ;

[0055] Among them, N is 192 antennas, P is 1024 frequency points, M is 24 FPGA boards, K is jis the number of antenna lines processed by the jth board. According to this formula, each acquisition board processes 1024 frequency point data of 8 antennas.

[0056] The distributed beamforming system of the embodiment of the present invention is composed of G GPU servers, which synthesize B beams in total. h Dual-polarization or single-polarization data of each frequency point.

[0057] ;

[0058] Among them, G is 24 GPU nodes, B is 96 beams, and Q is the number of frequencies processed by each GPU node. According to the formula, each GPU server node processes 192 channels of 43 frequency data. Since 1024 frequency points cannot be divided by 24 GPU nodes, the last GPU node only needs 35 frequency point data, and the remaining 8 frequency points are set to 0, and the total number of valid frequency points is still 1024.

[0059] Each node of the distributed beamforming system in the embodiment of the present invention adopts a self-organizing architecture. Figure 3 It can be seen that each node consists of a motherboard, CPU, memory, input / output network card and GPU.

[0060] A specific implementation of the present invention is that the memory size is 16GB, the GPU model is a general graphics card RTX4090, the input network card is a dual-port 10G network card, and the output network card is a single-port 10G network card. The GPU is inserted into the motherboard PCIe x16 slot, the input network card is inserted into the motherboard PCIe x8 slot, and the output network card is inserted into the PCIe x1 slot.

[0061] Each node of the GPU-based distributed beamforming system in the embodiment of the invention is assigned a number 1, 2, 3, ..., G-1, G. Each node is interconnected through a 10 Gigabit network, and the node number is automatically identified according to the local area network IP address.

[0062] The astronomical signal data in the embodiment of the present invention is transmitted to the distributed beamforming system through the signal acquisition system and the switching network, such as Figure 4 As shown, the embodiment of the present invention completes the steps of receiving 192 antenna data of 43 frequencies, GPU beamforming parallel calculation, astronomical data preprocessing and result sending in each node of the distributed beamforming system. In the embodiment of the present invention, each node completes 96 beamforming of 43 frequencies.

[0063] At least one dual-port network card is equipped on each GPU server for data reception. The data of the signal acquisition system in the embodiment of the present invention is connected to the distributed beamforming system through a switching network in a fully connected manner, that is, the data of all FPGA acquisition boards will transmit part of the frequency data to each GPU server. The connection method between the FPGA board and the two network ports of a single server is: the 96 channels of 43 frequency data composed of the first 12 FPGA boards are transmitted to the server's network port 1, and the 96 channels of 43 frequency data composed of the last 12 FPGA boards are transmitted to the server's network port 2.

[0064] like Figure 5 As shown in the figure, after each server receives the data, two memory buffer pools are designed to cache the data. Each buffer pool has a one-to-one correspondence with the network port, that is, buffer pool 1 is used to cache the data received by network port 1, and buffer pool 2 is used to cache the data received by network port 2. The data of the two buffer pools together is the complete N-way antenna Q h Frequency data.

[0065] After receiving and caching data for a period of time, the data is transferred to the GPU memory, and the parallel calculation of beamforming and astronomical signal preprocessing are completed in the GPU, that is, the Stokes integral parallel calculation is performed on the beamforming results.

[0066] After completing the parallel calculation of beamforming and Stokes integral in the GPU, the results are packaged into a fixed format and sent to the astronomical signal processing backend, thus realizing the real-time operation of beamforming of the entire system.

[0067] The astronomical signal processing backend usually consists of multiple server nodes. Usually, the i-th server node processes 1024 frequency points of S i beam data. i The calculation formula is as follows:

[0068] ;

[0069] Among them, L is 24 astronomical signal processing backend server nodes, S i is the number of beams processed by each server in the astronomical signal processing backend. According to the formula, each server in the astronomical signal processing backend processes 4 beams and 1024 frequency point data.

[0070] When the distributed beamforming system in the embodiment of the present invention sends the result, it rearranges the result in each node, and divides the beamforming result into 24 groups according to the number of server nodes 24 of the astronomical signal processing backend, each group contains data of 43 frequency points and 4 beams. The 24 groups of data are sent to the 24 servers of the astronomical signal processing backend in sequence, and the connection mode with the distributed beamforming system adopts a fully connected mode, such as Figure 2 shown.

[0071] Considering the load balancing of the switch and ensuring that each server of the astronomical signal processing backend can receive the required data at the same time, the distributed beamforming system of the embodiment of the present invention determines to start sending from the i-th group of result data according to its own node number when sending the result. i The range is 1, 2, ..., 24. That is, the first GPU node starts sending the first group of result data to the network card in sequence, and the result sending order is the first group, the second group, ..., the 24th group; the second GPU node starts sending the second group of result data to the network card in sequence, and the result sending order is the second group, the third group, ..., the 24th group, the first group, and the subsequent GPU nodes send the results in the same order.

[0072] By adopting the embodiments of the present invention, the following beneficial effects are achieved:

[0073] The embodiment of the present invention utilizes the parallel computing advantages of GPU and the scalability of cluster architecture, can process a large number of parallel tasks, realize signal beam synthesis by using distributed GPU cluster architecture, distribute a large amount of observation data to multiple nodes for parallel processing, realize beam synthesis of part of GPU data on each self-organizing node, and design the format and sending order of beam synthesis results according to the load balancing principle, thereby reducing system cost while improving efficiency, ensuring real-time performance, and significantly improving data processing efficiency.

[0074] Method Embodiment

[0075] According to an embodiment of the present invention, a beamforming method based on a GPU distributed cluster is provided. Figure 6 is a flowchart of a beamforming method based on a GPU distributed cluster according to an embodiment of the present invention. Figure 6 As shown, the beamforming method based on GPU distributed cluster in the embodiment of the present invention specifically includes:

[0076] S61, collecting astronomical data through M FPGA boards of a signal acquisition system and preprocessing the astronomical data, wherein the astronomical data includes data of N antennas and P frequency points, wherein N and P are positive integers greater than or equal to 1;

[0077] S62, sending the pre-processed astronomical data to the distributed beamforming system through the switching network;

[0078] S63, receiving N antenna data through G GPU nodes of the distributed beamforming system and performing GPU beamforming parallel calculation to synthesize B beams, and sending the B beams as beamforming results to the astronomical signal processing backend, where B and G are positive integers greater than or equal to 1;

[0079] S64. Receive beamforming results through L server nodes of the astronomical signal processing backend, where L is a positive integer greater than or equal to 1.

[0080] This method embodiment is a method embodiment that corresponds one-to-one to the above-mentioned system embodiment. Please refer to the above-mentioned system embodiment for specific implementation.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A beamforming system based on GPU distributed cluster, characterized in that include: Signal acquisition systems, switching networks, distributed beamforming systems, and astronomical signal processing backends; The signal acquisition system includes M FPGA boards, which are used to collect astronomical data through the M FPGA boards and pre-process the astronomical data, wherein the astronomical data is data including P frequency points of N antennas, wherein N and P are positive integers greater than or equal to 1; The switching network is used to send the pre-processed astronomical data to the distributed beamforming system; The distributed beamforming system includes G GPU nodes, which are used to receive N antenna data through the G GPU nodes and perform GPU beamforming parallel calculation to synthesize B beams, and send the B beams as beamforming results to the astronomical signal processing backend, where B and G are positive integers greater than or equal to 1; The astronomical signal processing backend includes L server nodes, which are used to receive beamforming results through the server nodes, where L is a positive integer greater than or equal to 1; The signal acquisition system is specifically used for: Through the j FPGA board acquisition The P frequency point data of the antenna is preprocessed, and the preprocessed data is packaged and encapsulated into VDIF format and sent to the distributed beamforming system through the switching network, where: Obtained by the following formula: (1)； Wherein, M is a positive integer greater than or equal to 1; The GPU node includes: a motherboard, a CPU, a GPU, a memory, an input network card, and an output network card; The distributed beamforming system is specifically used for: Each GPU node processes the data of N antennas, among which h GPU processing Q h Frequency points, Q h Obtained by the following formula: (2)； The specific process of each GPU node processing N antenna data is as follows: One or more dual-port network cards are set on each GPU node to receive data sent by the switching network; After receiving the data, each GPU node caches the received data through two pre-set memory buffer pools, and combines the data of the two buffer pools to form complete N-way antenna Q frequency point data; wherein each buffer pool corresponds one-to-one to the network port of the dual-port network card; The two memory buffer pools transfer data to the GPU at preset time intervals, and the GPU completes the beamforming parallel calculation and astronomical signal preprocessing; The results are packaged into a fixed format and sent to the astronomical signal processing backend.

2. The system according to claim 1, characterized in that The method for preprocessing the astronomical data by the signal acquisition system includes: delay compensation, PFB, FFT and dynamic truncation.

3. The system according to claim 1, characterized in that The signal acquisition system is connected to the distributed beamforming system through a switching network in a fully connected manner.

4. The system according to claim 1, characterized in that The distributed beamforming system is further used to: assign different numbers to each GPU node, interconnect each GPU node through a 10 Gigabit network, and automatically identify the number according to the IP address of the local area network where the GPU node is located.

5. The system according to claim 1, characterized in that The astronomical signal processing backend i server nodes handle P nodes beam data, where Obtained by the following formula: (3)。 6. The system according to claim 5, characterized in that The Bournard beamforming system is further used to: rearrange the results in each GPU node when sending the beamforming results to the astronomical signal processing backend, specifically including: According to the number of server nodes L at the backend of astronomical signal processing, the beamforming results are divided into L groups. i The group contains Q frequency points The L groups of data are sent to the L servers of the astronomical signal processing backend in sequence; The distributed beamforming system sends the results according to the GPU node number from the i The group data starts to be sent. i The range is 1, 2, …, L.

7. A beamforming method based on a GPU distributed cluster, applied to the beamforming system based on a GPU distributed cluster as claimed in any one of claims 1 to 6, characterized in that include: Collecting astronomical data through M FPGA boards of a signal acquisition system and preprocessing the astronomical data, wherein the astronomical data includes data of P frequency points of N antennas, wherein N and P are positive integers greater than or equal to 1; The pre-processed astronomical data is sent to the distributed beamforming system through a switching network; Receive N antenna data through G GPU nodes of the distributed beamforming system and perform GPU beamforming parallel calculation to synthesize B beams, and send the B beams as beamforming results to the astronomical signal processing backend, where B and G are both positive integers greater than or equal to 1; The beamforming result is received by L server nodes of the astronomical signal processing backend, where L is a positive integer greater than or equal to 1.

Images