A data processing method and device for radio frequency interference

Abstract

The invention discloses a data processing method and device for reducing radio frequency interference (RFI) signals in astronomical observation data. The method not only greatly improves the reliability and accuracy of RFI signal detection, but also reduces the time consumption for processing a large amount of data. The data processing method mainly comprises the steps of obtaining data to be processed; conducting statistics on numerical values in the data and drawing a histogram thereof; obtaining an empirical cumulative distribution function of the data; defining a fitting function and performing fitting; marking the RFI; selecting an optimal filter function; and weighting each data point and convolving the data to be processed with the filter function. According to the invention, the computational efficiency and speed are high and massive data can be quickly processed in real time. The method performs convolution processing on RFI data based on a two-dimensional Gaussian function, can perform multiple iteration operations on the data by using the two-dimensional Gaussian function to make the final image smoother and reduce background noise, and can effectively eliminate strong and weak RFI signals.

Description

technical field [0001] The invention relates to the field of radio interference reduction, in particular to a data processing method and device for radio frequency interference reduction in radio astronomy data. Background technique [0002] With the continuous update and development of radio astronomy equipment, its observation performance is getting higher and higher. However, due to the very limited spectrum resources allocated by radio astronomy, radio equipment is constantly affected by increasingly stronger non-astronomical signals, that is, Radio Frequency Interference (RFI), the main source of which is human daily activities and communication behaviors. The impact of radio frequency interference on observation results is very serious, which greatly reduces the quality of data and even makes data invalid. Therefore, effective technical means to eliminate radio frequency interference are needed. However, the existing data processing technology of radio frequency interf...

AI Technical Summary

Problems solved by technology

The impact of radio frequency interference on observation results is very serious, which greatly reduces the quality of data and even makes data invalid. Therefore, effective technical means to eliminate radio frequency interference are needed. However, the existing data processing technology of radio frequency interference has the following defects:

[0003] (1) Due to the update and upgrade of observation equipment, the bandwidth is constantly increasing, and the resolution is getting higher and higher, resulting in a very large amount of data, usually in the order of TB. However, the existing RFI elimination technology is very time-consuming and cannot be quickly Real-time processing of massive data

For example, in general, the data processing of conventional radio frequency interference adopts the mark threshold method, which is a local method, and the data is processed in the time domain or frequency domain, and the calculation amount of processing is N (number of frequency points)*M (number of time points)* L (baseline quantity) is not suitable for big data; its corresponding marking algorithm needs to generate an RFI statistical table first, and then delete it in subsequent operations. Identification and deletion of data are two independent operations, so it cannot be completed in real time

[0004] (2) The processing and recognition effect of weak RFI signals is poor, and the authenticity and sensitivity of detecting RFI signals are not enough. Often only strong RFI signals can be effectively identified, and the final data results have a large error with the real situation; for example, conventional The labeling algorithm in the paper is not effective in dealing with weak RFI signals, and can only identify the position of RFI in the data sequence. The potential consequence of this is that strong RFI signals will leave some residues in the noise, and these missed noises are related to Weak RFI signals constitute the noise floor together, which increases the difficulty of subsequent data processing, so it is difficult to further process the remaining weak RFI signals

[0005] (3) At present, most of the methods for reducing radio frequency interference use nonlinear processing methods, which cannot perform iterative calculations, which greatly limits the accuracy of RFI noise elimination

Images