Coherent de-dispersion method for pulsar baseband data

Abstract

The invention provides a coherent de-dispersion method for pulsar baseband data. The method comprises the following steps: establishing a data transmission mode among a data reading thread, an input buffer area, a multi-thread parallel coherent de-dispersion thread, an output buffer area and a file writing thread based on a HASHPIPE architecture; preprocessing data frames of the VDIF data and storing the data frames into an input buffer area; executing a multi-thread parallel coherent achromatic thread by using data of the input buffer area, and writing a result into the output buffer area, wherein data splitting, FFT, frequency conversion, complex multiplication, frequency conversion, IFFT, filling detection power removal and Stokes parameter calculation are realized; and calling a file writing thread to write the data in the output buffer area into the file. According to the method disclosed by the invention, while the coherent de-dispersion process is accelerated by applying multiple threads in parallel, the acceleration processing process of the GPU is further optimized by using the HASHPIPE architecture for parallel processing, so that the processing performance is remarkably improved.

Description

Technical Field

[0001] The invention belongs to the field of radio telescopes, and in particular relates to a coherent de-dispersion method for pulsar baseband data, which is based on a GPU and a HASHPIPE architecture. Background Art

[0002] Pulsars are one of the four great discoveries in astronomy in the 1960s. Their discovery confirmed the existence of neutron stars. Pulsars have extremely strong magnetic fields. While their rotation axes rotate at high speed, they emit radio radiation from the magnetic polar caps. When these radiations sweep across radio telescopes on Earth, the radiation of pulsars is detected and presents as periodic pulse signals. By measuring pulsar signals, the arrival time (TOA) and pulse profile of the pulses can be obtained, so as to study the physical processes under extreme conditions in the universe, such as pulsar magnetosphere dynamics, internal structure, cosmic background gravitational waves, and super-dense matter. Pulsars have a stable period and extremely slow period change rate, and their stability is comparable to that of atomic clocks, so they can be used as a reference source for time calibration. In addition, the radiation characteristics of pulsars can also be used to detect the average magnetic field distribution of the interstellar medium and the Milky Way. These extreme physical properties make pulsars extremely valuable astrophysical laboratories.

[0003] However, the electromagnetic wave signals emitted by pulsars will be affected by the dispersion, scattering and scintillation effects of the interstellar medium during propagation, causing the average pulse profile width and shape of the pulsar signals received by the radio telescope to change. The dispersion effect will broaden the pulse profile, the scattering will further blur the signal, and the scintillation effect will cause fluctuations in pulse intensity. Among these effects, the time delay caused by the dispersion effect is much greater than the scattering delay. Therefore, in order to obtain an accurate pulse shape, the pulsar observation data must be de-dispersed.

[0004] At present, there are two main methods for de-dispersion: incoherent de-dispersion and coherent de-dispersion. Incoherent de-dispersion is an earlier method that uses a multi-channel filter to divide the pulsar baseband observation signal into multiple independent channels, and applies an appropriate time delay to the data of each frequency band, so that the received pulse signals are aligned and added. However, although incoherent de-dispersion eliminates the dispersion effect to a certain extent, it has certain limitations in high-precision observations because the dispersion within each small channel cannot be completely eliminated.

[0005] Coherent de-dispersion technology was first proposed by Hankins and Rickett in 1975. Although it was not initially implemented due to computer performance limitations, its theoretical approach is unique. Coherent de-dispersion effectively eliminates the dispersion effect within the passband by converting the time domain signal into a frequency domain signal, multiplying it with the transmission function of the interstellar medium, and then converting it back to the time domain. With the advancement of computer and data storage technology, especially the parallel computing power of graphics processing units (GPUs), coherent de-dispersion technology can achieve real-time processing when processing massive observation data, greatly improving the observation sensitivity and time resolution.

[0006] In other words, incoherent dedispersion cannot completely eliminate the residual dispersion in the frequency channel, while coherent dedispersion will not have this problem. Therefore, the pulsar baseband data after sampling and processing by the existing radio telescope terminal is more inclined to undergo coherent dedispersion processing to remove the dispersion effect introduced by the pulsar signal during the propagation of the interstellar medium, thereby improving the time resolution and signal quality of the data.

[0007] In recent years, the in-depth study of pulsars has promoted the development of research such as millimeter-wave pulsar observations and gravitational wave detection, and has put forward higher requirements on the performance of observation equipment.

[0008] Internationally, countries such as Australia and the United States have gradually formed a heterogeneous pulsar digital terminal system architecture of FPGA+CPU+GPU. In this architecture, FPGA is used for ADC control and signal preprocessing, CPU is responsible for data transmission and logic control, and GPU is used to execute core algorithms such as coherent de-dispersion. This architecture has the ability to process wide bandwidth and high-speed signals in real time, and has been applied in many pulsar observation projects.

[0009] For example, the Parkes Telescope in Australia has installed multiple pulsar terminal systems, such as BPSR, CASPER, and UWL. These systems use the above-mentioned heterogeneous architecture, in which the CASPER system processes dual-polarization signals through IBOB, CPU, and multiple GPU nodes to achieve full-bandwidth coherent de-dispersion from low frequency to high frequency. Similarly, the GUPPI pulsar terminal of the Green Bank Telescope (GBT) in the United States also provides a wide-bandwidth coherent de-dispersion observation mode, and the VEGAS system was developed on this basis.

[0010] In China, pulsar observation equipment has been gradually developed. The Tianma 65-meter radio telescope of the Shanghai Astronomical Observatory, the Nanshan 26-meter telescope of the Xinjiang Astronomical Observatory, the 40-meter telescope of the Yunnan Astronomical Observatory, the 40-meter telescope of the National Time Service Center, and the FAST 500-meter telescope of the National Astronomical Observatory have all applied incoherent or coherent dispersion elimination technology to varying degrees. Since the Xinjiang Nanshan Observatory introduced the analog multi-channel filter group dispersion elimination system from the University of Manchester in 1999, it has continuously improved its pulsar observation equipment and introduced the Australian pulsar digital filter group PDFB system and incoherent dispersion elimination system in 2009.

[0011] The Tianma 65-meter radio telescope is currently equipped with multiple terminal systems, among which the DIBAS (Digital Backend System) digital terminal is designed based on the VEGAS system developed by the National Radio Astronomy Observatory of the United States, and has a pulsar observation mode similar to GUPPI, supporting two data processing methods: incoherent and coherent de-dispersion. The DIBAS digital terminal is an unpublished technology. The DIBAS hardware system consists of a digital acquisition unit, a high-performance computing unit, and a storage unit. It adopts a GPU-based heterogeneous computing mode to further optimize and improve the coherent de-dispersion technology. Since the terminal is jointly developed by China and the United States, the data obtained from DIBAS after observation are all data files that have been processed according to scientific goals, not raw baseband data. The post-processing technology of pulsar data is not mastered by us, and if the equipment stops working unexpectedly, it will be impossible to perform subsequent observation and processing on the pulsar data.

[0012] In other words, since the pulsar equipment used by Shanghai Tianma is imported equipment, the pulsar data obtained after observation is already processed data, the technical content has not been made public, and the company does not have the coherent de-dispersion technology. In addition, most domestic equipment uses incoherent de-dispersion.

[0013] In summary, there are indeed some obvious gaps in the current pulsar data processing technology at home and abroad, especially in the calculation of massive data, especially in the independent research and development of pulsar terminals and processing technology. The specific shortcomings of pulsar data processing technology include:

[0014] 1. High computing resource requirements: In the application of high-precision data processing and de-dispersion algorithms, existing technologies often require a large amount of computing resources. Traditional computing methods may require high hardware investment and may cause system overload when processing large amounts of data.

[0015] 2. Insufficient processing accuracy: Some existing technologies may not strictly control the accuracy of data during the processing process, resulting in errors when performing complex processing such as coherent dedispersion of pulsar signals. This will affect the final data quality and analysis results.

[0016] 3. Poor flexibility: Traditional data processing methods usually lack sufficient flexibility and adaptability, and are difficult to adjust according to different data characteristics and processing requirements. When modifying algorithms and adapting to new requirements, a lot of manual adjustments and optimizations may be required.

[0017] GPU accelerated processing: In order to improve processing efficiency, GPU is usually used for accelerated computing in the prior art, but the processing speed is still not fast enough. Summary of the invention

[0018] The purpose of the present invention is to provide a coherent dedispersion method for pulsar baseband data, so as to accelerate the coherent dedispersion process by applying multi-threaded parallelism, and further optimize the accelerated processing process of GPU by using HASHPIPE architecture parallel processing, so as to significantly improve the processing performance.

[0019] In order to achieve the above object, the present invention provides a coherent de-dispersion method for pulsar baseband data, comprising:

[0020] S1: Based on the HASHPIPE architecture, a data transmission method is established between the data reading thread net_thread.c, the input buffer inputbuf, the multi-threaded parallel coherent de-dispersion thread, the output buffer outputbuf and the file writing thread output_thread.c;

[0021] S2: In the data reading thread net_thread.c, the data frame of the VDIF data is checked, the time information is read, and the frame header is deleted to obtain the data in the required arrangement format and store it in the input buffer inputbuf;

[0022] S3: Execute multi-threaded parallel coherent de-dispersion threads using data in the input buffer inputbuf and write the results into the output buffer outputbuf. In this process, data splitting, FFT, frequency conversion, complex multiplication, frequency conversion, IFFT, de-filling detection power and Stokes parameter calculation are implemented in sequence through function calls;

[0023] S4: Call the file writing thread output_thread.c to write the processed data in the output buffer outputbuf into the file.

[0024] The step S1 specifically includes: step S11: using the buffer setting programs databuf.h and databuf.c to complete the configuration of the input buffer inputbuf and the output buffer outputbuf, and define the data structure related to the frame number of the VDIF data.

[0025] The size of the input buffer inputbuf is three times the amount of data processed each time, which is nvalid×nfft×nsub, where nsub refers to the number of subbands of VDIF data, nfft refers to the number of FFT data set to be processed simultaneously in GPU parallel processing, and nvalid is the valid data length of a single subband in one fft;

[0026] The size of the output buffer outputbuf is three times the amount of data written each time Msamp×mchan, where mchan is the total number of channels generated in the end and Msamp is the length of the data after processing for the number of channels set by the user; and / or

[0027] The number of input buffers inputbuf and output buffers outputbuf is multiple.

[0028] The step S2 specifically includes:

[0029] S21: Use the data reading thread net_thread.c to read the VDIF baseband data and read the frame header information;

[0030] S22: Parse the time in the frame header of the data frame and calculate the corresponding UTC time and MJD time;

[0031] S23: Remove the frame header and store the processed data into the input buffer inputbuf that has been set.

[0032] The step S22 further includes: by detecting changes in the second number second and the frame number frame per second in the data frame header, checking the frame number within a second in the frame header, and filling the gap by generating an equal amount of white noise frames when data frames are found to be lost.

[0033] The step S3 specifically includes:

[0034] S31: Execute the main program module gpu_thread.cu to implement the initialization of the multi-threaded parallel coherent de-dispersion thread;

[0035] S32: After the main program module gpu_thread.cu implements the initialization of the multi-threaded parallel coherent de-dispersion thread, the thread state of the hashpipe framework is determined to ensure the correct operation of the thread; next, the data state of the input buffer inputbuf and the output buffer outputbuf is checked to see if they are available; if they are determined to be available, step S33 is executed;

[0036] S33: Execute the main program module gpu_thread.cu to implement the process of multi-threaded parallel coherent de-dispersion threads.

[0037] The step S33 specifically includes:

[0038] S331: For each data processing, the main program module gpu_thread.cu reads a specified amount of data from the input buffer inputbuf, and enters the format conversion function separatevdifData, and calls the format conversion function separatevdifData to implement data conversion and data arrangement processing of VDIF data;

[0039] S332: calling the data splitting function unpack_and_padd to split the data into nfft×nsub data blocks with a valid length of nvalid, where nfft is the number of FFT data to be performed simultaneously in GPU parallel processing, nsub refers to the number of subbands of VDIF data, and nvalid is the valid data length of a single subband in one fft;

[0040] S333: calling the FFT plan to perform a fast Fourier transform based on multi-thread parallel processing on the nfft×nsub group of data blocks;

[0041] S334: calling the complex multiplication function PointwiseComplexMultiply to perform a complex multiplication operation on the data block and the chirp function generated in step S31 to obtain a data block after dispersion elimination;

[0042] S335: calling the IFFT plan to perform an inverse Fourier transform based on multi-threaded parallel processing on the data block that finally generates the total number of channels mchan, so as to convert the de-dispersed data back to the time domain;

[0043] S336: calling a de-pad detection power function transpose_unpadd_and_detect to remove the overlapping part of the time domain data and calculate the power sum on the two polarization channels;

[0044] S337: Calculate and quantify the Stokes parameters, save them in the output buffer outputbuf, and then terminate the current multi-threaded parallel coherent de-dispersion thread.

[0045] In step S334, before performing the complex multiplication operation, it also includes: deleting the negative spectrum part and retaining the positive frequency part; and / or before step S334 and after step S334, it also includes: calling the channel spectrum positive and negative frequency conversion function swap_spectrum_halve to flip the positive and negative frequencies of the data.

[0046] The multi-threaded parallel processing includes one of GPU parallel processing, FPGA accelerated parallel processing, multi-core CPU parallel processing and dedicated hardware processor parallel processing.

[0047] The file is a filterbank file; and the coherent de-dispersion method further comprises step S5: performing folding drawing in the dspsr pulsar software according to the generated filterbank file.

[0048] Compared with the existing domestic pulsar baseband data processing technology, the coherent de-dispersion method of pulsar baseband data of the present invention combines GPU acceleration technology with HASHPIPE architecture to achieve efficient real-time processing of pulsar signals. In particular, in the application of the coherent de-dispersion algorithm, the parallel processing capability of the GPU greatly improves the calculation speed. This improvement significantly reduces the processing delay, allowing the system to complete data parsing more quickly. In addition, the present invention introduces VDIF data frame parsing and lost frame repair technology to ensure the integrity and continuity of data during transmission, solving the problem of difficulty in recovering data frames after loss in the prior art.

[0049] When processing pulsar observation data, the existing technology often encounters problems such as insufficient processing speed and difficulty in repairing lost frames when facing large-scale data. The present invention combines GPU acceleration with multi-threaded processing, which not only greatly improves the processing efficiency, but also ensures the integrity of the data through an intelligent lost frame detection and repair mechanism. This improvement improves the real-time performance and accuracy of pulsar signal processing and better meets the processing requirements of radio telescope observation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the data frame format of existing VDIF data.

[0051] Figure 2 This is a schematic diagram of the data frame header format of existing VDIF data.

[0052] Figure 3 It is the frequency distribution diagram of the eight left-hand or right-hand channels.

[0053] Figure 4 It is a schematic diagram of the principle of processing one of the two data primitive data types.

[0054] Figure 5 It is an overall structural block diagram of the coherent de-dispersion method of pulsar baseband data of the present invention.

[0055] Figure 6 It is a structural block diagram of the coherent de-dispersion algorithm part of the coherent de-dispersion method for pulsar baseband data of the present invention.

[0056] Figure 7 It is a schematic diagram of the principle of adding and removing overlapping parts of the coherent de-dispersion method of pulsar baseband data of the present invention.

[0057] Figure 8-Figure 10 This is a result diagram of three pulsars obtained by the coherent de-dispersion method of pulsar baseband data of the present invention. DETAILED DESCRIPTION

[0058] The present invention is further described below in conjunction with specific examples. It should be understood that the following examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0059] The basic principle of coherent de-dispersion based on the coherent de-dispersion method of pulsar baseband data of the present invention is as follows:

[0060] The following will introduce the basic theory of coherent dedispersion and the derivation process of the core algorithm in detail. The pulsar radiation emitted by the pulsar will pass through the interstellar medium ISM before being received by the radio telescope and will be affected by its refraction, i.e. dispersion. The refractive index μ is:

[0061]

[0062] where f p is the plasma frequency, f is the electromagnetic wave frequency, when f<f p When the electromagnetic waves cannot pass through the interstellar medium, this is also the key condition that constrains whether the pulsar radiation signal can be received by the radio telescope on the ground. The plasma frequency f p It can be expressed by the following formula. The other dispersion constant D can be expressed by the following formula:

[0063]

[0064] n e The electron number density is approximately equal to 0.03 cm -3 , e represents the charge, m e It represents the charge mass. The values ​​in the dispersion constant formula are all known, where c is the speed of light.

[0065] So according to the above two formulas, the plasma frequency f can be calculated p The values ​​of and dispersion constant D are:

[0066]

[0067] The pulsar radiation signal travels through the interstellar medium at a group velocity v g =cμ, from the formula we can know that v g <c, when the distance d from the pulsar to the earth is d, the time delay t caused by the interstellar medium can be calculated according to the following formula:

[0068]

[0069] By substituting the dispersion constant D and setting the dispersion measure The result of the last row in the above formula can be deduced.

[0070] In coherent dedispersion, the dispersion delay of the interstellar medium on the pulsar radiation signal appears as a convolution of the time domain signal, while in the frequency domain, this is equivalent to the product of the pulse radiation signal and the dispersion delay effect caused by the interstellar medium.

[0071] S(v0+v)=S initial (v0+v)×H(v0+v)

[0072] Among them, S is the signal received by the radio telescope, S initial is the original signal of pulse radiation, v0 is the center frequency of the channel, ν is the frequency offset within the channel, and H(ν0+ν) represents the transmission function of the interstellar medium in the frequency domain.

[0073] The delay effect caused by the interstellar medium can be expressed as a phase rotation that depends on the frequency and the propagation path length. In the frequency domain, the interstellar medium only has a delay effect on the frequency domain phase of the signal. The wave number is the wavelength, d is the distance the pulsar radiation signal travels in the interstellar medium, and c = λv.

[0074] By the formula Performing Taylor expansion, we can get:

[0075] So the transmission function of the interstellar medium in the frequency domain is H(ν0+ν):

[0076]

[0077] Taylor expansion of the above formula and taking the first three terms yields:

[0078]

[0079] The first of these three is a constant phase shift, the second term According to the shift theorem, it corresponds to the delay in the time domain, and the third term is the dispersion effect of the in-band signal, which is the part we need to eliminate.

[0080] Therefore, the present invention uses the previously derived dispersion DM to replace the distance d of the pulsar radiation from the pulsar to the earth and the plasma frequency f p , the final transfer function can be obtained, so that the formula can be used to perform coherent de-dispersion processing on the radio telescope received signal S.

[0081] The final transfer function is:

[0082]

[0083] Where DM represents the dispersion, v0 is the center frequency of the channel, ν is the frequency offset within the channel, and k DM Represents the dispersion constant.

[0084] The basic principle of VDIF format data decoding based on the coherent de-dispersion method of pulsar baseband data of the present invention is as follows:

[0085] The baseband data generated by the VLBI terminal is in VDIF (VLBI Data Interchange Format) format, and the VLBI data packet consists of countless data frames.

[0086] The data frame format of the VLBI data packet belongs to the prior art. The VLBI Data Interchange Format (VDIF) Specification document explains the VDIF data format in detail. Each data frame is as follows Figure 1 As shown, it includes a data frame header and a data array. A data frame includes sub-band data of one or more frequencies, and the frame header contains information about the data array in the data frame. The purpose of introducing the data frame format in this section is to explain the specific effect of the data decoding part program designed by the present invention for the data frame format.

[0087] Among them, the basic format of the data frame header is as follows Figure 2 As shown, the information in the data frame header is the basis for subsequent data processing.

[0088] exist Figure 2In the VDIF, a word represents 4 bytes, and the frame header consists of 8 words, which is equivalent to 32 bytes. This is the standard 32-byte frame header. Words are just a way of division, and multiple words are transmitted at the same time. VDIF data transmission is carried out according to the structure of data frames. The data collected in real time by the data acquisition terminal will be transmitted in the form of data frames and saved in the data file. It should be noted that the digital acquisition and transmission parts are both existing technologies.

[0089] Word 0 (4 bytes):

[0090] Bit 31: invalid data flag I1 (valid = 0, invalid = 1);

[0091] Bit 30: legacy mode L1 (0 indicates a standard 32-byte frame header; 1 indicates a 16-byte frame header); in the present invention, the data frame header format used for processing data is a standard 32-byte frame header, so when reading the frame header information, it is necessary to read the legacy mode L1 to confirm that it is a standard 32-byte frame header;

[0092] Bits 29-0: Seconds from reference epoch 30 .

[0093] Word 1 (4 bytes):

[0094] Bits 31-30: Unassigned (should be set to all 0) Un-assigned2;

[0095] Bits 29-24: Reference epoch Ref Epoch6 (starting from 00:00 UTC on January 1, 2000, and increasing by one every six months); take the data frame header information read by the program designed by the present invention as an example, where the epoch below is the data of the reference epoch from bits 29 to 24 here, and the value is 44, which is equivalent to increasing by 1 every six months starting from January 1, 2000, and increasing by 1 44 times, which is equivalent to January 1, 2022 as the reference time, starting from this time as a reference to record the seconds, and the seconds information is recorded in bits 23-0;

[0096] Bits 23-0: Data Frame # within second 24(starting from zero, an integer number of data frames per second); the observation bandwidth of the telescope is very wide, and the amount of data generated per second is also very large. A single VDIF data frame cannot accommodate all the data. Taking the frame header information read by the program designed by the present invention as an example, the data volume of a single data frame is 8192 bytes, but according to the bandwidth and sampling rate of the telescope, the data volume of 1 second is 8*64000000 bytes, which means that multiple data frames are needed to store the data per second. The data frame number within the second records the amount of data in the current second. If the data frame number within the second is a, it means that the data currently recorded is the data starting from (a-1)*8192 bytes in the current second.

[0097] Word 2 (4 bytes):

[0098] Bits 31-29: VDIF version number V3 (used to support up to seven future VDIF frame header formats);

[0099] Bits 28-24: log2(#chns)5 of the number of channels in the data array (base 2; the number of channels must be a power of 2); the existing telescope observation data is divided into channels. According to the following frame header information, the value of bits 28-24 is 4, so the number of channels is expressed as 2 to the fourth power, that is, 16 channels. This VDIF file records the starting frequency of 2268Mhz, the bandwidth of each frequency channel is 64Mhz, there are a total of eight channels, left-right rotation data, that is, 2×8=16 channels;

[0100] Bit 23-0: Data frame length (including header) 24 , the unit is 8 bytes; the data frame number within the second above records the total number of data frames per second, and this part records the number of data stored in the data array in a single data frame.

[0101] Word 3 (4 bytes):

[0102] Bit 31: Data type C1 (0 indicates real data; 1 indicates complex data);

[0103] Bits 30-26: bits per sample (bits / sample - I5) (maximum 32 bits / sample);

[0104] Bits 25-16: Thread ID 10 , 0 to 1023);

[0105] Bit 15-0: Station ID 16 , a 2-character ASCII standard ID or a numeric ID assigned by a global site);

[0106] The number of bits per sample indicates the number of bytes occupied by a single data in the data array. The thread ID and site ID are not used later, but these two items are included in the data frame header in VDIF, so they are briefly introduced here.

[0107] Word 4-7 (4 bytes): Extended User Data 32 .

[0108] In the present invention, the data type processed by the coherent de-dispersion method of pulsar baseband data is a VDIF multi-channel data type. Therefore, in the present invention, the VDIF data is pre-processed, specifically including:

[0109] Step A1: Read the data frame header information, and determine the exact time point of data observation through the three parameters of seconds, legacymode, and epoch in the data frame header information. Seconds is the number of seconds from the reference epoch, epoch is the reference epoch, and legacymode is the legacy mode.

[0110] Step A2: Use three functions UTCconvertToMJD, UTC2JD, and VDIFconvertToMJD to calculate the information of the data frame header of the VDIF data to UTC time and MJD time;

[0111] Among them, UTC and MJD are two ways of recording astronomical time. According to the existing method described above, UTC time can be calculated. There is an existing conversion formula between UTC time and MJD time. MJD time can be calculated according to the formula. MJD time will be used when generating pulsar processing files later.

[0112] Step A3: Check and supplement the data frame by detecting the change of the second number and the frame number per second in the data frame header.

[0113] Among them, four functions are used: data frame header check function check_vdif_header, data frame header check function only_check_vdif_header, white noise generation function generate_gaussian_noise and data frame supplement function generate_and_write_vdif_frame, where check_vdif_header implements the function of checking whether the data frame is lost. If there is a loss, the white noise generation function generate_gaussian_noise and the data frame supplement function generate_and_write_vdif_frame are called to use white noise to fill the lost data frame. White noise can only be used to fill the lost data array, not the data frame header. The data frame header check function only_check_vdif_header is used to check the data frame again after completing the above operations.

[0114] For example, for the data file information of the pulsar B0329+54 observed by the Tianma telescope, seconds=68133904, legacymode=0, frame=0, epoch=44, framelength=1028, nhan=4, version=0, stationid=0, treadid=0, nbits=7. Therefore, the observed UTC time and the abbreviated Julian day time are accurately calculated, and the frame length can also be used to calculate the frame header plus data array bytes of a single data frame of the VDIF file, which is 8224 bytes (the framelength read data is 1028, then the size of the entire data frame is 1028×8=8224 bytes, excluding the 32 bytes of the frame header, the data array length is 8224-32=8192 bytes, of which the frame header size is 32 frame headers and the data array length is 8192 bytes. By outputting data through nhan, the total data channel can be calculated to be 2 4 = 16 channels, 16 channels include eight channels of left-handed signals and eight channels of right-handed channels. A single channel has 64,000,000 samples per second, but the vdif data is arranged as the first channel right-handed sample 1, the first channel left-handed sample 1, the second channel right-handed sample 1, the second channel left-handed sample 1, etc., and sample 2 is arranged after sample 1. The specific arrangement is as follows: Figure 3 and Figure 4 As shown in the figure, this is one of the arrangements. The size of a single data in the channel is 8 bits, that is, a char type integer data.

[0115] The frequency distribution of the eight left-hand or right-hand channels is as follows: Figure 3 As shown, the starting frequency of the first channel is 1968MHz, the bandwidth of a single channel is 64MHz, and the final cutoff frequency of the eight channels is 2480MHz.

[0116] In the VDIF data array, there are two ways to arrange the original data. The system includes processing of two data original data types, and the schematic diagram of processing one of the two data original data types is as follows: Figure 4 Specifically, before processing, the data is first arranged in the order of left and right rotation signals, and then arranged in eight channels in sequence. Another arrangement is the opposite, first arranged in eight channels in sequence and then arranged in the order of left and right rotation signals.

[0117] In order to perform coherent dedispersion processing of pulsars, the data must first be preprocessed. Therefore, the VDIF data is preprocessed, specifically including: removing the data frame header information of the VDIF data and rearranging the data to prepare for subsequent processing. Among them, rearranging the data includes storing the left-handed and right-handed VDIF data in two independent data buffers, and adjusting the VDIF data according to the logic of channel first and time later.

[0118] Therefore, the two kinds of raw data are finally processed to form the following Figure 4 The arrangement format shown on the right. After rearrangement, the header information of each data frame is removed, leaving only valid data. Secondly, the left and right rotation data are stored in two independent data buffers as shown in the right figure below. In the new arrangement format, the first time point of all channels is the first, and then the second time point of all channels is arranged. Such a data structure is more suitable for subsequent processing operations, especially when performing coherent de-dispersion, which helps to improve processing efficiency and continuity of data access.

[0119] The coherent de-dispersion method of pulsar baseband data of the present invention is implemented by a coherent de-dispersion system based on GPU and HASHPIPE architecture. The coherent de-dispersion system based on GPU and HASHPIPE architecture is mainly divided into two parts, namely, the HASHPIPE architecture part and the coherent de-dispersion algorithm part. The HASHPIPE architecture part is mainly as follows: Figure 5 As shown in Figure 2, the coherent de-dispersion algorithm runs through the HASHPIPE architecture. Its specific structure is as follows: Figure 6 shown.

[0120] The coherent de-dispersion method for pulsar baseband data of the present invention can realize coherent de-dispersion processing of multi-channel baseband pulsar data (multi-channel data here refers to the number of channels of baseband data and the number of channels in VDIF data), and the final total number of channels generated in the final pulsar filterbank file can be selected according to user needs (the final total number of channels generated here refers to the number of channels obtained by splitting each single channel in the program, which will be explained in detail below). The generated filterbank file can be directly used to complete the final folding drawing in the dspsr software, and due to the channel splitting function of the program, the user can see the details of the pulsar pattern in each frequency channel, and can also compare the gap between pulsar signals under different frequency channel frequency bandwidths in more detail.

[0121] The implementation process of the coherent de-dispersion method for pulsar baseband data of the present invention is described in detail below.

[0122] The coherent de-dispersion method of pulsar baseband data of the present invention comprises:

[0123] Step S1: establishing a data transmission mode among a data reading thread net_thread.c, an input buffer inputbuf, a multi-threaded parallel coherent de-dispersion thread, an output buffer outputbuf and a file writing thread output_thread.c based on a HASHPIPE architecture;

[0124] Among them, HASHPIPE is a high-availability shared pipeline engine, which accelerates multi-threaded parallel operation and completes the function of fast data transmission between threads through shared buffers. Based on the HASHPIPE architecture, the data transmission method is that data is transmitted in sequence along the data reading thread net_thread.c, the input buffer inputbuf, the multi-threaded parallel coherent de-dispersion thread, the output buffer outputbuf and the file writing thread output_thread.c. Therefore, through the HASHPIPE architecture, the cyclic reading and writing of the two shared buffers are realized, thereby ensuring efficient data processing.

[0125] The step S1 specifically includes:

[0126] Step S11: using the buffer setting programs databuf.h and databuf.c, the configuration of the input buffer inputbuf and the output buffer outputbuf is completed, and the data structure related to the frame number of the VDIF data is defined.

[0127] Among them, the size of the input buffer inputbuf is set to three times the amount of data processed each time, which is nvalid×nfft×nsub, where nsub refers to the number of subbands of VDIF data. In this embodiment, nsub takes 8 channels as an example, nfft refers to the number of FFT data performed simultaneously in GPU parallel processing, and nvalid is the valid data length of a single subband in an fft, that is, the amount of data processed each time is nvalid data amount corresponding to the number of subband channels that can be performed nfft times.

[0128] The values ​​of nvalid, nfft, and nsub are set manually.

[0129] The number of input buffers inputbuf is multiple to improve the data processing speed. In this embodiment, the number of input buffers inputbuf is 3, and the purpose of buffers of the same size is to achieve parallel processing.

[0130] The input buffer inputbuf works as follows:

[0131] 1) Data reading: The data reading thread net_thread.c is responsible for reading data; after the received data frame header is removed, the remaining data will be stored in the first input buffer.

[0132] 2) Filling the input buffer inputbuf: When the first input buffer inputbuf is full, the data reading thread net_thread.c will store the newly received data in the second input buffer inputbuf; this process is performed in a cycle, and the data is stored in turn among the three input buffers inputbuf to form a circular queue.

[0133] 3) The role of input buffer inputbuf: If the reading thread processes data faster and the GPU processes it slower, the input buffer inputbuf can effectively prepare the data. In this way, when the GPU reads data from the first input buffer inputbuf, if its processing speed is slower, the second input buffer can provide subsequent data, thereby avoiding the GPU waiting for data and improving the overall processing efficiency.

[0134] The size of the output buffer outputbuf is set to three times the amount of data written each time, Msamp×mchan, where mchan is the total number of channels generated in the end, which is the product of the number of baseband data sub-band channels and the number of user-split channels, and Msamp is the data length after processing of the number of single channels set by the user. The values ​​of Msamp and mchan are also set manually.

[0135] There are multiple output buffers outputbuf, which also play a role in data preparation for subsequent reading of the file writing thread output_thread.c. In this embodiment, there are three output buffers outputbuf.

[0136] Therefore, the size of the data block after the final data processing will change from nvalid×nfft×nsub to Msamp×mchan data size. The same as the input data format, three output buffers of the same size are set to achieve parallel processing.

[0137] Important data parameters in the coherent de-dispersion system are also defined in the buffer setting programs databuf.h and databuf.c, which is convenient for subsequent programs to call.

[0138] Step S2: VDIF data preprocessing, including: in the data reading thread net_thread.c, checking the data frame of the VDIF data, reading the time information and deleting the frame header, obtaining the data in the required arrangement format and storing it in the input buffer inputbuf.

[0139] Step S2 specifically includes:

[0140] Step S21: using the data reading thread net_thread.c to read the VDIF baseband data and read the frame header information;

[0141] Step S22: Parse the time in the frame header of the data frame to calculate the corresponding UTC time and MJD time;

[0142] Since pulsar data has extremely high requirements for time accuracy, step S22 also includes checking the frame number within a second in the frame header by detecting changes in the second number and the frame number per second in the data frame header to ensure the continuity of the frame; and when data frames are found to be lost, they are filled by generating an equal amount of white noise frames to avoid data missing problems in subsequent pulsar folding and drawing.

[0143] Step S23: Remove the frame header and store the processed data in the input buffer inputbuf that has been set. The system will give a prompt each time the processed data is successfully stored. After this step is completed, the data is saved in the input buffer inputbuf and waits for further processing.

[0144] Step S3: using the data in the input buffer inputbuf to execute the multi-threaded parallel coherent de-dispersion thread and write the result into the output buffer outputbuf, in which data splitting, FFT, frequency conversion, complex multiplication, frequency conversion, IFFT, de-filling detection power and Stokes parameter calculation are sequentially realized through function calls;

[0145] Step S3 is the core step. The multi-threaded parallel coherent de-dispersion thread is composed of the coherent de-dispersion kernel function gpu_kernel.cu, the function declaration module gpu_thread.h and the main program module gpu_thread.cu.

[0146] The coherent de-dispersion kernel function gpu_kernel.cu defines all the functions required for coherent de-dispersion, that is, Figure 6 The definitions of the rest of the functions in the middle column except FFT and IFFT (including format conversion function separatevdifData, data splitting function unpack_and_padd, channel spectrum positive and negative frequency conversion function swap_spectrum_halve, complex multiplication function PointwiseComplexMultiply, de-filling detection power function transpose_unpadd_and_detect, and stokes parameter calculation and quantization function) are all in the coherent de-dispersion kernel function gpu_kernel.cu. The function declaration module gpu_thread.h declares the functions defined in the coherent de-dispersion kernel function gpu_kernel.cu.

[0147] The principle of coherent dedispersion is explained in detail above. The time delay caused by the pulsar radiation signal passing through the interstellar medium can be expressed as the product of the interstellar medium effect in the frequency domain and the pulsar signal in the frequency domain, as shown below:

[0148] S initial (v+v0)=H -1 (v+v0)S(v+v0)

[0149] The inverse of the transfer function H -1 (v+v0) is called the chirp function.

[0150] Therefore, the overall process of step S3 is as follows: Figure 6 The main program module gpu_thread.cu is used to complete the initialization of the multi-threaded parallel coherent de-dispersion thread, the process of the multi-threaded parallel coherent de-dispersion thread (ie, the main thread), and the calling of all functions.

[0151] Step S3 specifically includes:

[0152] Step S31: executing the main program module gpu_thread.cu to implement the initialization of the multi-threaded parallel coherent de-dispersion thread;

[0153] The initialization includes variable initialization, reading parameter file para, CPU and GPU memory allocation (memory needs to be allocated in advance during initialization for each part of the data after being processed by the function), FFT plan generation and chirp function calculation.

[0154] Because the cufft library in CUDA is called in the function, the FFT in this cufft library needs to generate an FFT plan in advance. The FFT plan records the number of data points processed by a single fft and the form of fft processing. The setting here is CUFFT_C2C. Subsequent FFT only needs to call this FFT plan and can run directly without setting too many parameters.

[0155] The parameter file para includes N bin (single processing data size), noverlap (filling length), outputfilename (output fillterbank file name), source_name (processing pulsar source name, such as B0329+54), dm_start (dispersion start value), dm_step (chromatic step length), ndm (number of dispersions), nchan (number of user split channels), etc. The para file is provided each time according to the user's processing requirements.

[0156] It can be seen from the formula that the chirp function is related to the frequency bandwidth of the pulsar and the number of single data processing points. The frequency bandwidth of the pulsar baseband data is fixed, and the number of single data processing points is also set as a fixed value in the Para parameter file, so the chirp function calculation can be completed according to the known parameters in the initialization stage.

[0157] The calculation of the chirp function is realized by the calculation formula of the transfer function, and the transfer function is:

[0158]

[0159] Where DM represents the dispersion, k DM represents the dispersion constant, v0 is the center frequency of the subband or channel, the bandwidth is Δv (all in MHz), and v is the frequency offset within the channel or subband, satisfying -Δv / 2<ν<Δν / 2.

[0160] The initialization process of the main program module gpu_thread.cu can be completed after the cache area sets the programs databuf.h and databuf.c to initialize the input buffer inputbuf and the output buffer outputbuf and the data reading thread net_thread.c for thread initialization, without waiting for the data reading thread net_thread.c to complete the preprocessing of the first data block. This is because the initialization steps of the HASHPIPE architecture program can be implemented in parallel according to the current running status.

[0161] That is to say, both the data reading thread net_thread.c and the main program module gpu_thread.cu have the functions of thread initialization and the main program of the thread. The operating logic in the HASHPIPE architecture is to first complete the cache setting programs databuf.h and databuf.c and then complete the initialization of the data reading thread net_thread.c and the main program module gpu_thread.cu.

[0162] Step S32: After the main program module gpu_thread.cu implements the initialization of the multi-threaded parallel coherent de-dispersion thread, the thread state of the hashpipe framework is determined to ensure the correct operation of the thread; next, check whether the data status of the input buffer inputbuf and the output buffer outputbuf is available to ensure the continuity of data processing; if it is determined to be available, execute step S33.

[0163] Step S33: Execute the main program module gpu_thread.cu to implement the multi-threaded parallel coherent de-dispersion thread process.

[0164] Step S33 specifically includes:

[0165] Step S331: For each data processing, the main program module gpu_thread.cu reads a specified amount of data from the input buffer inputbuf, and enters the format conversion function separatevdifData, calling the format conversion function separatevdifData to implement data conversion and data arrangement processing of VDIF data;

[0166] Step S332: calling the data splitting function unpack_and_padd to split the data into nfft×nsub data blocks with a valid length of nvalid, where nfft is the number of FFT data to be performed simultaneously in GPU parallel processing, nsub refers to the number of subbands of VDIF data, and nvalid is the valid data length of a single subband in one fft;

[0167] Due to the characteristics of the discrete convolution theorem, there will be confusion at the edges of the output time series. To avoid this phenomenon, we use the overlap preservation method.

[0168] The specific operation process of calling the data splitting function unpack_and_padd is as follows Figure 6 shown.

[0169] like Figure 7 As shown, in step S332, in the data splitting function unpack_and_padd, the data is split into nfft×nsub data blocks of length nvalid, where nvalid is the valid data length of a single subband in an fft, and an overlapping part of length noverlap is added at both ends of each data block. After adding the overlapping part, the total length of the data reaches nfft×nsub data points of length nbin, where nbin is the number of calculation points of FFT, and the size of nbin is nvalid+2×noverlap.

[0170] It should be noted that the data content of the overlapping part also comes from the input data.

[0171] Step S333: Call the FFT plan to perform GPU parallel processing-based Fast Fourier Transform (FFT) on the nfft×nsub group of data blocks.

[0172] like Figure 6 As shown, the FFT program has been initialized in the initialization step of step S31, so it can be directly operated here. In this process, the number of calculation points of FFT is nbin, and Fourier transform is performed on nfft×nsub group data blocks each time.

[0173] After FFT, the data is converted from the time domain to the frequency domain, and the data aliasing is reduced by the overlapping preservation method.

[0174] Step S334: calling the complex multiplication function PointwiseComplexMultiply to perform a complex multiplication operation on the data block and the chirp function generated in step S31, thereby obtaining a data block after dispersion elimination.

[0175] In the complex multiplication function PointwiseComplexMultiply, complex multiplication operation is performed on the pulsar baseband data (ie, data block) in the frequency domain and the chirp function generated during initialization, thereby completing the dispersion elimination in the channel.

[0176] In this embodiment, since the format type of the VDIF data is a real number type, the imaginary part can be filled with zeros to perform Fourier transform in a complex format. According to the principle of signal system, a symmetrical spectrum will appear after the real signal is transformed through Fourier transform. Therefore, in step S334, before performing the complex multiplication operation, it also includes: deleting the negative spectrum part and retaining the positive frequency part.

[0177] Since the fft plan calls the cufft library, the positive and negative frequencies of the spectrum in the Fourier transform result will be swapped. Therefore, in the present invention, a channel spectrum positive and negative frequency conversion function swap_spectrum_halve is also designed to restore the shape of the spectrum to avoid subsequent data processing problems caused by library functions. This channel spectrum positive and negative frequency conversion function swap_spectrum_halve needs to be performed before and after the complex multiplication function PointwiseComplexMultiply, that is, after FFT and before IFFT; that is, before step S334 and after step S334, it includes: calling the channel spectrum positive and negative frequency conversion function swap_spectrum_halve to flip the positive and negative frequencies of the data.

[0178] Among them, after step S334 (before IFFT), while calling the channel spectrum positive and negative frequency conversion function swap_spectrum_halve to restore the spectrum channel, the user's required channel division is performed to convert the number of channels on the frequency from the number of sub-bands nsub of the VDIF data to the final total number of channels mchan. Among them, nchan is the number of user split channels, that is, a single sub-band of the VDIF data is split into nchan channels, and the final total number of channels mchan and the number of user split channels nchan satisfy mchan=nsub×nchan.

[0179] Step S335: Call the IFFT plan to perform an inverse Fourier transform (IFFT) based on GPU parallel processing on the data block that ultimately generates the total number of channels mchan, so as to convert the de-dispersed data back to the time domain.

[0180] Similar to the FFT plan, the IFFT plan also completes the initialization in the initialization step. However, unlike FFT, in this process, the length of each single data processed by IFFT is 1 / (2×nchan) of the data length during Fourier transform, where the 2 in the denominator is due to the deletion of half of the data, and 1 / nchan is due to the frequency channel splitting of a single subband when converting from the frequency domain to the time domain, and nchan is the number of channels split by the user. Thus, the user's demand for channel segmentation is met.

[0181] Step S336: Call the unpadd detection power function transpose_unpadd_and_detect to Figure 7 As shown, the overlapping part of the time domain data is removed and the power sum on the two polarization channels is calculated;

[0182] For the data that has been restored to the time domain, completed coherent dedispersion and completed data channel splitting, it is necessary to process the single data in the de-pad detection power function transpose_unpadd_and_detect, remove the overlapping parts added at both ends of the previous valid data, and calculate the power sum of the data on the two polarization channels to prepare for subsequent quantization operations.

[0183] Step S337: Calculate and quantize the Stokes parameters and save them in the output buffer outputbuf, and then terminate the current multi-threaded parallel coherent de-dispersion thread.

[0184] In this embodiment, the compute_block_sums, compute_channel_statistics and redigitize kernel functions are called to perform Stokes parameter calculation and data quantization.

[0185] Since the power sum obtained in the previous step is a floating point number, and the data to be stored in the end is of unsigned char type, the compute_block_sums, compute_channel_statistics and redigitize kernel functions quantize the data to complete the conversion from floating point numbers to char type data, where compute_block_sums completes the data summation, compute_channel_statistics part completes the calculation of the mean and standard deviation, and redigitize quantizes the data according to the mean and standard deviation calculated previously, and converts the floating point number to unsigned char type data. The processed data will be saved in the output buffer outputbuf, that is, written to the output block, and the input buffer outputbuf and output buffer outputbuf status are finally updated, terminating the current multi-threaded parallel coherent de-dispersion thread. The data after redigitize is restored to be available and can be written to the output buffer outputbuf.

[0186] Step S4: calling the file writing thread output_thread.c to write the processed data in the output buffer outputbuf into a file, especially into a filterbank file that meets the processing requirements of the dspsr pulsar software.

[0187] In order to ensure the correctness of the file format and the integrity of the data, before calling the file writing thread output_thread.c to perform the writing operation, the file header of the filterbank file is initialized and written according to the data information of the parameter file obtained; after the file header is written, the data in the output buffer outputbuf is written to this filterbank file.

[0188] Among them, the data information of the parameter file includes key information such as the name of the observation source, the right ascension and declination of the observation source, the sampling rate, the number of channels, the starting frequency, and the ending frequency.

[0189] Step S5 (optional): Perform folding drawing in the dspsr pulsar software according to the generated filterbank file.

[0190] Among them, dspsr is a mature pulsar software. Using the corresponding instructions, you can draw the filterbank file that has completed coherent dedispersion, so that the data can be visualized in front of the user.

[0191] Experimental results:

[0192] After the coherent de-dispersion system of pulsar baseband data based on GPU and HASHPIPE architecture was completed, a large amount of data was used to test the technology. The main pulsar data tested were the following pulsars: B0329+54, B0355+54, J0248+6021

[0193] like Figure 8 The data shown are for pulsar B0329+54, with a flux of 203 mjy, right ascension 03:32:59.4096, declination 54:34:43.329, DM value 26.7641, and a rotation period frequency of about 1.399;

[0194] like Fig. 9 Shown are the data of pulsar B0355+54, with a flux of 23mjy, right ascension 03:58:53.7238, declination 54:13:13.784, DM value 57.1420, and a rotation period frequency of about 6.394;

[0195] like Fig.10 Shown are the data of pulsar J0248+6021, with a flux of 14mjy, right ascension 02:48:18.617, declination 60:21:34.72, DM value 370, and a rotation period frequency of approximately 4.606.

[0196] Figure 8-Figure 10The results of three pulsars are shown in Figure 1. The initial pulsar baseband data are all VDIF data of 8 sub-bands. After the designed coherent de-dispersion system, a single sub-band is divided into 64 sub-channels, with a total of 8*64 channels. The pulse pattern of each single channel can be seen in the dspsr folded drawing. Since the Tianma telescope currently uses the S and X bands for pulsar observations, the previous data acquisition terminal of the system only contains the observation frequency of the S band, so in the final generated graph, there are only pulsar signals around 2200Mhz to 2300Mhz. The system can cover a wide frequency range, laying the foundation for subsequent ultra-wideband data processing.

[0197] In other embodiments, the GPU can be replaced by other types of processors capable of implementing multithreading, so the GPU parallel processing can be replaced by other types of multithreaded parallel processing. The following is an enumeration of multithreaded parallel processing technologies.

[0198] FPGA accelerates parallel processing: FPGA (field programmable gate array) can be used as an alternative to GPU, especially in applications that require high throughput and low latency. By implementing the coherent de-dispersion algorithm and data frame parsing algorithm on FPGA, it can provide real-time data processing capabilities and optimize at the hardware level to improve processing speed and efficiency.

[0199] Multi-core CPU parallel processing: Similar processing acceleration can also be achieved by using a CPU with multiple processing cores. By optimizing multi-core processing and task scheduling, parallel processing capabilities similar to those of a GPU can be achieved. Although this method may be slightly inferior to a GPU in performance, it is still effective for certain application scenarios.

[0200] Dedicated hardware processor parallel processing: Design and use custom hardware processors (application specific integrated circuits) specifically for pulsar data processing. These processors can be optimized for specific processing requirements, thereby increasing processing speed and efficiency.

[0201] The coherent de-dispersion method of pulsar baseband data of the present invention can achieve the following beneficial effects:

[0202] 1. Improve processing speed: The present invention realizes efficient data processing by utilizing GPU acceleration and HASHPIPE architecture. GPU has powerful parallel computing capability and can significantly improve the speed of pulsar data processing. HASHPIPE architecture can add parallel processing channels on the basis of the original GPU to improve the speed.

[0203] GPU accelerated computing technology uses the parallel computing power of GPU to accelerate complex coherent de-dispersion algorithms during pulsar data processing, thereby improving data processing efficiency. It has significant performance advantages, especially when processing large-scale observation data.

[0204] The multi-threaded data processing architecture based on HASHPIPE adopts the HASHPIPE framework and ensures the real-time performance of data processing through multi-threaded parallel processing. It is suitable for application scenarios of large-scale data processing, especially the real-time observation and processing needs of radio telescopes.

[0205] 2. Optimize computing resource utilization: By optimizing data processing algorithms and using modern hardware acceleration technology, the present invention reduces the demand for computing resources and improves the processing efficiency of the system. This can reduce hardware investment costs while improving processing performance. Among them, for VDIF data format decoding: adopt an efficient decoding algorithm to reduce the CPU burden. Coherent de-dispersion: Through parallel processing and efficient algorithms, maximize the use of GPU resources.

[0206] 3. Improve processing accuracy: The present invention designs precise algorithms and data processing procedures to ensure that the accuracy of data can be controlled when processing pulsar signals, reduce errors, and improve data reliability and analysis accuracy.

[0207] Coherent de-dispersion processing technology converts the pulsar signal received by the radio telescope into the frequency domain and applies the de-dispersion algorithm in the frequency domain to eliminate the pulse profile broadening problem caused by the interstellar medium, thereby improving the time resolution and accuracy of the signal.

[0208] 4. Increase flexibility and adaptability: The technical solution of the present invention has high flexibility and can be adjusted and optimized according to different data characteristics and requirements, adapt to various processing scenarios, and improve the adaptability and scalability of the system. Among them, the present invention adopts dynamic parameter adjustment: the characteristics of the input data through the parameter file are adjusted to process different types of data; and modular design: the system adopts a modular architecture, and each processing module can be replaced or adjusted according to needs. These steps enhance the adaptability and scalability of the system, enabling it to adapt to various processing scenarios.

[0209] 5. The decoding process of baseband VDIF format data is more accurate and efficient: For baseband data collected by radio telescopes, the VDIF (VLBI Data Interchange Format) format is currently mainly used. VDIF is a general term for a format. The data in VDIF format is generally the same in format. The frame header information filled in different VDIF files is different, and the data arrangement format in different VDIF files is also different. The existing VDIF decoding method is only suitable for current VDIF data. The baseband data used in the coherent de-dispersion method of pulsar baseband data of the present invention comes from other digital acquisition terminals of Tianma. The format of the baseband data is VDIF format, and a special decoding method is designed, and GPU is used for accelerated decoding, which is faster and more accurate.

[0210] VDIF data parsing and lost frame repair technology determines the integrity of the data by detecting the seconds and frame numbers of VDIF format data frames. When data frames are lost, it fills the lost parts by generating Gaussian noise to ensure data continuity and integrity.

[0211] The above-mentioned technical points jointly ensure the real-time data processing capability and high efficiency of the present invention.

[0212] In summary, compared with the existing domestic pulsar baseband data processing technology, the present invention combines GPU acceleration technology with HASHPIPE architecture to achieve efficient real-time processing of pulsar signals. In particular, in the application of coherent de-dispersion algorithm, the parallel processing capability of GPU greatly improves the calculation speed. This improvement significantly reduces the processing delay, allowing the system to complete data parsing more quickly. In addition, the present invention introduces VDIF data frame parsing and lost frame repair technology to ensure the integrity and continuity of data during transmission, solving the problem of difficulty in recovering data frames after loss in the prior art.

[0213] When processing pulsar observation data, the existing technology often encounters problems such as insufficient processing speed and difficulty in repairing lost frames when facing large-scale data. The present invention combines GPU acceleration with multi-threaded processing, which not only greatly improves the processing efficiency, but also ensures the integrity of the data through an intelligent lost frame detection and repair mechanism. This improvement improves the real-time performance and accuracy of pulsar signal processing and better meets the processing requirements of radio telescope observation data.

[0214] The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. The above embodiments of the present invention can also be modified in various ways. All simple, equivalent changes and modifications made according to the claims and the description of the present invention fall within the scope of protection of the claims of the present invention. The contents not described in detail in the present invention are all conventional technical contents.

Claims

1. A coherent de-dispersion method for pulsar baseband data, characterized in that: include: Step S1: establishing a data transmission mode among a data reading thread net_thread.c, an input buffer inputbuf, a multi-threaded parallel coherent de-dispersion thread, an output buffer outputbuf and a file writing thread output_thread.c based on a HASHPIPE architecture; Step S2: in the data reading thread net_thread.c, the data frame of the VDIF data is checked, the time information is read, and the frame header is deleted to obtain the data in the required arrangement format and store it in the input buffer inputbuf; Step S3: using the data in the input buffer inputbuf to execute the multi-threaded parallel coherent de-dispersion thread and write the result into the output buffer outputbuf, in which data splitting, FFT, frequency conversion, complex multiplication, frequency conversion, IFFT, de-filling detection power and Stokes parameter calculation are sequentially realized through function calls; Step S4: calling the file writing thread output_thread.c to write the processed data in the output buffer outputbuf into the file.

2. The coherent de-dispersion method for pulsar baseband data according to claim 1, characterized in that: The step S1 specifically includes: step S11: using the buffer setting programs databuf.h and databuf.c to complete the configuration of the input buffer inputbuf and the output buffer outputbuf, and define the data structure related to the frame number of the VDIF data.

3. The coherent de-dispersion method for pulsar baseband data according to claim 2, characterized in that: The size of the input buffer inputbuf is three times the amount of data processed each time, which is nvalid×nfft×nsub, where nsub refers to the number of subbands of VDIF data, nfft refers to the number of FFT data set to be processed simultaneously in GPU parallel processing, and nvalid is the valid data length of a single subband in one fft; The size of the output buffer outputbuf is three times the amount of data written each time Msamp×mchan, where mchan is the total number of channels generated in the end and Msamp is the length of the data after processing for the number of channels set by the user; and / or The number of input buffers inputbuf and output buffers outputbuf is multiple.

4. The coherent de-dispersion method for pulsar baseband data according to claim 1, characterized in that: Step S2 specifically includes: Step S21: using the data reading thread net_thread.c to read the VDIF baseband data and read the frame header information; Step S22: Parse the time in the frame header of the data frame to calculate the corresponding UTC time and MJD time; Step S23: remove the frame header and store the processed data into the input buffer inputbuf that has been set.

5. The coherent de-dispersion method for pulsar baseband data according to claim 4, characterized in that: The step S22 further includes: by detecting changes in the second number second and the frame number frame per second in the data frame header, checking the frame number within a second in the frame header, and filling the gap by generating an equal amount of white noise frames when data frames are found to be lost.

6. The coherent de-dispersion method for pulsar baseband data according to claim 1, characterized in that: The step S3 specifically includes: Step S31: executing the main program module gpu_thread.cu to implement the initialization of the multi-threaded parallel coherent de-dispersion thread; Step S32: After the main program module gpu_thread.cu completes the initialization of the multi-threaded parallel coherent de-dispersion thread, the thread state of the hashpipe framework is determined to ensure the correct operation of the thread; next, check whether the data state of the input buffer inputbuf and the output buffer outputbuf is available; if it is determined to be available, execute step S33; Step S33: Execute the main program module gpu_thread.cu to implement the multi-threaded parallel coherent de-dispersion thread process.

7. The coherent de-dispersion method for pulsar baseband data according to claim 6, characterized in that: The step S33 specifically includes: Step S331: For each data processing, the main program module gpu_thread.cu reads a specified amount of data from the input buffer inputbuf, and enters the format conversion function separatevdifData, calling the format conversion function separatevdifData to implement data conversion and data arrangement processing of VDIF data; Step S332: calling the data splitting function unpack_and_padd to split the data into nfft×nsub data blocks with a valid length of nvalid, where nfft is the number of FFT data to be performed simultaneously in GPU parallel processing, nsub refers to the number of subbands of VDIF data, and nvalid is the valid data length of a single subband in one fft; Step S333: calling the FFT plan to perform FFT based on multi-thread parallel processing on the nfft×nsub group of data blocks; Step S334: calling the complex multiplication function PointwiseComplexMultiply to perform a complex multiplication operation on the data block and the chirp function generated in step S31 to obtain a data block after dispersion elimination; Step S335: calling the IFFT plan to perform IFFT based on multi-thread parallel processing on the data block that finally generates the total number of channels mchan, so as to convert the de-dispersed data back to the time domain; Step S336: calling the unpadd detection power function transpose_unpadd_and_detect to remove the overlapping part of the time domain data and calculate the power sum on the two polarization channels; Step S337: Calculate and quantize the Stokes parameters and save them in the output buffer outputbuf, and then terminate the current multi-threaded parallel coherent de-dispersion thread.

8. The coherent de-dispersion method for pulsar baseband data according to claim 7, characterized in that: In step S332, in the data splitting function unpack_and_padd, the data is split into nfft×nsub data blocks of length nvalid, where nvalid is the valid data length of a single subband in one fft, and overlapping parts of length noverlap are added at both ends of each data block; and / or In step S334, before performing the complex multiplication operation, the method further includes: deleting the negative frequency spectrum part and retaining the positive frequency part; and / or Before step S334 and after step S334, the process includes: calling the channel spectrum positive and negative frequency conversion function swap_spectrum_halve to flip the positive and negative frequencies of the data.

9. The coherent de-dispersion method for pulsar baseband data according to claim 7, characterized in that: The multi-threaded parallel processing includes one of GPU parallel processing, FPGA accelerated parallel processing, multi-core CPU parallel processing and dedicated hardware processor parallel processing.

10. The coherent de-dispersion method for pulsar baseband data according to claim 1, characterized in that: In the step S4, the file is a filterbank file; and the coherent de-dispersion method further comprises a step S5: performing folding drawing in the dspsr pulsar software according to the generated filterbank file.

Images