NetCDF multi-source ionosphere data reconstruction method and device

Abstract

The invention discloses a NetCDF multi-source ionosphere data reconstruction method and device, and belongs to the technical field of GNSS satellite ionospheres. The method comprises the following steps: receiving multi-source ionosphere data and preprocessing the multi-source ionosphere data; creating a data structure body InfTR and an InfSP; judging the type of the preprocessed multi-source ionosphere data, and respectively storing the preprocessed multi-source ionosphere data to data structural bodies InfTR and InfSP according to the data type; matching satellite data with the same timestamps in the data structural bodies InfTR and InfSP through a hash algorithm to generate a unified data set; and recoding the unified data set according to a NetCDF format, defining global attributes and time dimensions, and storing variables in groups by using satellite numbers and frequency point numbers. According to the invention, efficient integration and output of ionosphere data are realized, the integrity of the ionosphere data is improved, and high-precision and high-efficiency data service is provided for the field of earth science.

Description

Technical Field

[0001] The present invention belongs to the technical field of GNSS satellite ionosphere technology, and in particular relates to a NetCDF multi-source ionosphere data reconstruction method and device. Background Art

[0002] With the rapid development of modern communications and navigation technologies, research on the impact of the ionosphere on them is becoming increasingly important. Parameters such as the total electron content (TEC), the rate of change of TEC index (ROTI), the ionospheric amplitude scintillation index (S4 index), and the phase scintillation index (SigmaPhi) play a key role in ensuring the performance of atmospheric electromagnetic wave propagation systems for satellite communications, navigation, and positioning.

[0003] However, due to the highly dynamic nature of the ionosphere and the limitations of measurement equipment, missing or incomplete data are common, making accurate ionospheric data acquisition a scientific challenge. Consequently, multi-source ionospheric data reconstruction techniques are gaining increasing attention. However, existing reconstruction models and preprocessing strategies often suffer from data alignment and timestamp synchronization issues, failing to meet the demands of high-speed, real-time, and high-precision ionospheric data processing.

[0004] Furthermore, the diversity of multi-source ionospheric data, particularly the inconsistencies in data storage formats, complicates data processing and analysis. The Network Common Data Format (NetCDF), a widely accepted scientific data format, improves the readability and processing efficiency of ionospheric data through its highly self-describing structured storage of dimensions, variables, attributes, and data. Despite this, how to better utilize the NetCDF format to process multi-source ionospheric data remains a pressing challenge for researchers. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a NetCDF multi-source ionospheric data reconstruction method and device, which realizes the efficient integration and output of ionospheric data through the preprocessing stage of time and space filtering, quality control, and time and space alignment, combined with cubic spline interpolation and hash algorithm.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A NetCDF multi-source ionospheric data reconstruction method, comprising:

[0008] Step 1: Receive multi-source ionospheric data and perform preprocessing;

[0009] Step 2: Create data structures InfTR and InfSP;

[0010] Step 3: Determine the type of multi-source ionospheric data after preprocessing, and store them into the data structures InfTR and InfSP respectively according to the data type;

[0011] Step 4: Use the hash algorithm to match the satellite data with the same timestamp in the data structures InfTR and InfSP to generate a unified data set;

[0012] Step 5: Recode the unified dataset in NetCDF format, define global attributes and time dimensions, and store variables in groups based on satellite numbers and frequency points.

[0013] In another aspect, the present invention provides a NetCDF multi-source ionospheric data reconstruction device, comprising:

[0014] A preprocessing module, used for receiving multi-source ionospheric data and performing preprocessing;

[0015] The structure creation module is used to create the data structures InfTR and InfSP;

[0016] The storage module is used to determine the type of multi-source ionospheric data after preprocessing and store it in the data structures InfTR and InfSP respectively according to the data type;

[0017] The matching module is used to match the satellite data with the same timestamp in the data structures InfTR and InfSP through a hash algorithm to generate a unified data set;

[0018] The reconstruction module is used to recode the unified dataset in NetCDF format, define global attributes and time dimensions, and store variables in groups based on satellite numbers and frequency points.

[0019] In a third aspect, the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned NetCDF multi-source ionospheric data reconstruction method.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned NetCDF multi-source ionospheric data reconstruction method.

[0021] The beneficial effects of the present invention are:

[0022] The present invention creatively applies the NetCDF data format to the ionosphere field, enhancing the readability and operability of the data. The cubic spline interpolation method is used to supplement the missing time ionosphere data to improve the data integrity rate. The hash algorithm is used to match satellite numbers and frequencies to enhance the data matching efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a NetCDF multi-source ionospheric data reconstruction method of the present invention;

[0024] Figure 2 It is the NetCDF ionospheric data file reconstructed based on the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described below with reference to the accompanying drawings and examples.

[0026] This paper provides a NetCDF multi-source ionospheric data reconstruction method, which uses the NetCDF standard to integrate and reconstruct multi-source ionospheric data. This method is applicable to multi-system satellite data, including Beidou, GPS, GLONASS, and Gailileo. Through a preprocessing phase involving temporal and spatial filtering, quality control, and temporal and spatial alignment, combined with cubic spline interpolation and a hashing algorithm, this method achieves efficient integration and output of ionospheric data. The NetCDF data format enhances data readability and operability, improves ionospheric data integrity, and provides high-precision and efficient data services for the Earth science field.

[0027] like Figure 1 As shown, a NetCDF multi-source ionospheric data reconstruction method of the present invention specifically includes the following steps:

[0028] Step 1: Receive multi-source ionospheric data and perform preprocessing, wherein the multi-source ionospheric data includes ionospheric TEC data, disturbance ROTI data, amplitude scintillation S4 data, and phase scintillation SigmaPhi data; receive the multi-source ionospheric data, and perform temporal and spatial filtering and quality control (such as outlier removal) on the ionospheric data;

[0029] Step 2: Create data structures InfTR and InfSP, where the data structure InfTR is used to cache TEC and ROTI related data, and the data structure InfSP is used to cache S4 and SigmaPhi related data;

[0030] Step 3: Determine the type of multi-source ionospheric data after preprocessing. If it is ionospheric TEC data and disturbance ROTI data, store the timestamp, satellite system, satellite number, and corresponding TEC and ROTI data in the InfTR structure. If it is amplitude scintillation S4 data and phase scintillation SigmaPhi data, store the timestamp, satellite system, satellite number, frequency point number, and corresponding S4 and SigmaPhi data in the InfSP structure.

[0031] Furthermore, for the data structure with missing timestamps, the cubic spline interpolation algorithm is used to supplement the TEC, ROTI or S4, SigmaPhi parameters; determine whether the timestamp of the InfTR structure storing TEC and ROTI data is missing. If it is missing, the cubic spline interpolation algorithm is used to supplement the TEC and ROTI data corresponding to each satellite at the missing time. If it is not missing, proceed to the next step; determine whether the timestamp of the InfSP structure storing S4 and SigmaPhi data is missing. If it is missing, the cubic spline interpolation method is used to supplement the S4 and SigmaPhi data corresponding to each frequency point of each satellite at the missing time. If it is not missing, proceed to the next step;

[0032] Step 4: Use a hash algorithm to match satellite data with the same timestamp in the InfTR and InfSP structures to generate a unified data set. Use a hash algorithm to calculate the hash value corresponding to the satellite number in the InfTR data structure, as well as the hash values ​​corresponding to the satellite number and frequency number in the InfSP data structure. Use the hash values ​​to match satellite data with the same timestamp in the InfTR and InfSP structures, extract the matched satellite data, and generate a unified data set.

[0033] The specific matching process includes:

[0034] (1) Establish a first-level hash index for the InfTR structure: using the satellite number as the key, calculate the hash value through the hash function to locate the SatEntry structure that stores the ionospheric TEC and ROTI data;

[0035] (2) Create a secondary hash index for the InfSP structure: first, use the satellite number as the key to locate the SatEntry in the primary hash table, and then use the frequency point number as the key to locate the FreqEntry structure storing the S4 and SigmaPhi data in the FreqHashMap;

[0036] (3) Timestamp synchronization matching: traverse each timestamp in InfTR and execute:

[0037] a) Obtain the satellite's TEC / ROTI data from the SatelliteHashMap of InfTR using the satellite ID hash value;

[0038] b) Locate the corresponding satellite’s FreqHashMap in InfSP using the hash value of the same satellite number;

[0039] c) Traverse all frequency points of the satellite and extract S4 / SigmaPhi data with the same timestamp;

[0040] (4) Data integration output: Merge the successfully matched TEC and ROTI data with the S4 and SigmaPhi data of all frequency points to generate a unified data record containing complete parameters;

[0041] (5) Conflict handling: When different satellite numbers generate the same hash value, a linked list method is used to store conflicting entries to ensure that all data can be retrieved.

[0042] Step 5: Recode the unified dataset using the NetCDF format, defining global attributes and a time dimension, and storing variables grouped by satellite number and frequency. The obtained unified dataset is recoded using the NetCDF data format. NetCDF describes and reconstructs ionospheric data using dimensions, variables, attributes, and data. Global attributes are defined as attributes, including common information such as station name, latitude and longitude, and equipment model. Dimensions are defined by time interval, and TEC and ROTI data and related data are grouped by satellite number. S4 and SigmaPhi data and related data are grouped by satellite number and frequency number.

[0043] Furthermore, the cubic spline interpolation algorithm in step 3 includes:

[0044] The time domain ionospheric parameter data is missing and needs to be interpolated based on the data of the previous and next time periods. The cubic spline interpolation algorithm has the advantages of smooth, continuous, accurate and flexible processing results. Compared with high-order polynomial interpolation, cubic spline interpolation is not prone to overfitting. The specific calculation is as follows:

[0045] Divide the ionospheric parameter data interval [a,b] into , a total of n+1 data points, among which, , , The cubic spline equation must meet the following requirements:

[0046] (1) ;

[0047] (2) Continuous and has second-order derivatives;

[0048] (3) In each segment superior, They are all cubic equations;

[0049] If the above conditions are met, then For function The cubic spline interpolation function of the node can be expressed as follows:

[0050] ,

[0051] Use fixed boundary conditions:

[0052] ,

[0053] In the above formula is the step length, i.e., the time interval, is the i moment, is the second differential value to be calculated at time i, is the ionospheric parameter data at time i, which is calculated and then brought into the cubic spline coefficient formula:

[0054] .

[0055] Furthermore, the hash algorithm designed in step 4 includes:

[0056] Design a hash table according to the satellite number and frequency. Each satellite number corresponds to a hash table, set as SatelliteHashMap. In SatelliteHashMap, the satellite number is the key, and the corresponding value is the ionospheric TEC and ROTI data. Another hash table is set as FreqHashMap. In FreqHashMap, the frequency is the key, and the corresponding value is the ionospheric S4 and SigmaPhi data.

[0057] For ionospheric TEC and ROTI data, a hash value is generated using the satellite number as the key to locate the TEC / ROTI data. The hash function takes the satellite number as input and generates a specific hash value. The hash value is used to quickly access the storage location of the TEC and ROTI data in the SatelliteHashMap.

[0058] For ionospheric S4 and SigmaPhi data, a hash value is generated using the satellite number and frequency as the key to locate the S4 / SigmaPhi data. The hash function uses the satellite number to locate the corresponding entry in SatelliteHashMap to obtain FreqHashMap, and then uses the frequency as input to generate a hash value for fast access to the storage location of S4 and SigmaPhi data in FreqHashMap. The specific design uses a two-layer hash table to achieve fast matching of satellite and frequency data. The first-level hash table (SatelliteHashMap) is designed as follows:

[0059] Key structure: 3-byte encoding of satellite identifier (e.g. "G01"):

[0060] struct SatKey {

[0061] char sys; / / system code (G / C / R / E, etc.)

[0062] uint8_t num; / / Satellite number (1-99)

[0063] };

[0064] Value structure:

[0065] struct SatEntry {

[0066] double timestamp; / / UTC timestamp (accuracy 1ms)

[0067] float TEC; / / Total Electron Content (TECU)

[0068] float ROTI; / / disturbance index

[0069] ; / / Secondary hash table pointer

[0070] };

[0071] The secondary hash table (FreqHashMap) is designed as follows:

[0072] Key structure: 2-byte encoding of the frequency point identifier (such as "L1"),

[0073] struct FreqKey {

[0074] char band; / / frequency band code (L / C, etc.)

[0075] uint8_t num; / / frequency point number (1-9)

[0076] };

[0077] Value structure:

[0078] struct FreqEntry {

[0079] float S4; / / Amplitude flicker index

[0080] float SigmaPhi; / / Phase flicker index (rad)

[0081] };

[0082] The hash function calculation uses the improved FNV-1a algorithm to achieve efficient hashing. The improved FNV-1a algorithm is a non-encrypted hash algorithm optimized for the characteristics of GNSS satellite data based on the standard FNV-1a hash algorithm. It achieves a nonlinear mapping between input data (satellite number / frequency) and hash value through the alternating effects of prime number multiplication and XOR operation, satisfying the following requirements:

[0083] ,

[0084] in: : Single-bit change of input data (any bit flip); : hash function; : Original input data (such as the binary code of satellite number "G01") : The value of the kth bit of the hash value; :Probability.

[0085] For input data s (satellite number or frequency), the hash value calculation process is:

[0086] ,

[0087] Where offset_basis is the initial hash value (FNV constant), s[i] is the i-th byte of the input string, and m is the size of the hash table.

[0088] The hash algorithm has the advantages of reducing system memory usage, fast interpolation, conflict resolution and data unification in satellite data reconstruction.

[0089] Furthermore, the method for re-encoding the ionospheric data in the NetCDF format in step 5 is:

[0090] Define global attributes and define common information such as station name, longitude and latitude, puncture point, equipment model, etc. as global attributes of the reconstructed ionosphere result file for easy use.

[0091] Determine the dimensions of your data using time intervals. Organizing data into time periods facilitates further analysis and visualization.

[0092] Define variables with satellite number as group, for example G01, define TEC and ROTI data as variables, and use satellite number GO1 as the group name.

[0093] Define variables with satellite number + frequency number as a group, for example, G01L1, define S4 and SigmaPhi data as variables, and use satellite number G01 + frequency number L1 as the group name. This ionospheric data NetCDF format encoding format can clearly display the ionospheric TEC, ROTI, S4, and SigmaPhi parameter data, making data access and processing more flexible, scalable, and reasonable in practical applications. Ultimately, it can generate ionospheric data files in the internationally accepted NetCDF data format, such as Figure 2 shown.

[0094] On the other hand, the present invention provides a NetCDF multi-source ionospheric data reconstruction device, which includes various modules capable of implementing various steps of the aforementioned method, specifically including:

[0095] A preprocessing module, used for receiving multi-source ionospheric data and performing preprocessing;

[0096] The structure creation module is used to create the data structures InfTR and InfSP;

[0097] The storage module is used to determine the type of multi-source ionospheric data after preprocessing and store it in the data structures InfTR and InfSP respectively according to the data type;

[0098] The matching module is used to match the satellite data with the same timestamp in the data structures InfTR and InfSP through a hash algorithm to generate a unified data set;

[0099] The reconstruction module is used to recode the unified dataset in NetCDF format, define global attributes and time dimensions, and store variables in groups based on satellite numbers and frequency points.

[0100] In a third aspect, the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned NetCDF multi-source ionospheric data reconstruction method.

[0101] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned NetCDF multi-source ionospheric data reconstruction method.

[0102] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A NetCDF multi-source ionospheric data reconstruction method, characterized in that: The method comprises: Step 1: Receive multi-source ionospheric data and perform preprocessing; Step 2: Create data structures InfTR and InfSP; Step 3: Determine the type of multi-source ionospheric data after preprocessing, and store them into the data structures InfTR and InfSP respectively according to the data type; Step 4: Use the hash algorithm to match the satellite data with the same timestamp in the data structures InfTR and InfSP to generate a unified data set; Step 5: Recode the unified dataset in NetCDF format, define global attributes and time dimensions, and store variables in groups based on satellite numbers and frequency points.

2. A NetCDF multi-source ionospheric data reconstruction method according to claim 1, characterized in that: In step 1, the multi-source ionospheric data includes ionospheric TEC data, disturbance ROTI data, amplitude scintillation S4 data, and phase scintillation SigmaPhi data, and the preprocessing includes time and space filtering and quality control.

3. A NetCDF multi-source ionospheric data reconstruction method according to claim 2, characterized in that: The step 3 includes, if the data type is ionospheric TEC data and disturbance ROTI data, storing the timestamp, satellite system, satellite number and corresponding ionospheric TEC data and disturbance ROTI data into the data structure InfTR; if it is amplitude scintillation S4 data and phase scintillation SigmaPhi data, storing the timestamp, satellite system, satellite number, frequency point number and the amplitude scintillation S4 data and phase scintillation SigmaPhi data corresponding to the frequency point into the data structure InfSP.

4. A NetCDF multi-source ionospheric data reconstruction method according to claim 3, characterized in that: The step 3 further includes using a cubic spline interpolation algorithm to supplement the ionospheric TEC data, disturbance ROTI data, amplitude scintillation S4 data, and phase scintillation SigmaPhi data parameters for the data structure with missing timestamps.

5. A NetCDF multi-source ionospheric data reconstruction method according to claim 4, characterized in that: The cubic spline interpolation algorithm in step 3 adopts fixed boundary conditions.

6. A NetCDF multi-source ionospheric data reconstruction method according to claim 1, characterized in that: The step 4 comprises: Generate hash values ​​using satellite numbers as keys to locate ionospheric TEC data and perturbation ROTI data; Generate hash values ​​using satellite number + frequency as keys to locate amplitude scintillation S4 data and phase scintillation SigmaPhi data; According to the hash value, the satellite data corresponding to the same timestamp in the InfTR structure and the InfSP structure are matched, and the matched satellite data are extracted to generate a unified data set.

7. A NetCDF multi-source ionospheric data reconstruction method according to claim 1, characterized in that: In step 5, the global attributes of the NetCDF format include the station name, longitude and latitude, and equipment model. The time dimension is divided into fixed intervals. The ionospheric TEC data and disturbance ROTI data are defined by satellite number as a group, and the amplitude scintillation S4 data and phase scintillation SigmaPhi data are defined by satellite number + frequency number as a group.

8. A NetCDF multi-source ionospheric data reconstruction device, characterized in that: include: A preprocessing module, used for receiving multi-source ionospheric data and performing preprocessing; The structure creation module is used to create the data structures InfTR and InfSP; The storage module is used to determine the type of multi-source ionospheric data after preprocessing and store it in the data structures InfTR and InfSP respectively according to the data type; The matching module is used to match the satellite data with the same timestamp in the data structures InfTR and InfSP through a hash algorithm to generate a unified data set; The reconstruction module is used to recode the unified dataset in NetCDF format, define global attributes and time dimensions, and store variables in groups based on satellite numbers and frequency points.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the NetCDF multi-source ionospheric data reconstruction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor can implement a NetCDF multi-source ionospheric data reconstruction method according to any one of claims 1 to 7.

Images