A radio spectrum map general purpose parallel computing method

Abstract

The application discloses a radio spectrum map general parallel computing method. In order to overcome the problem that the radio spectrum map data computing processing method cannot quickly analyze and compute massive radio monitoring road network data, the application comprises the following steps: S1, filtering and screening after parallel analysis of original radio monitoring data in a distributed processing unit; S2, performing channelization processing in the distributed computing unit according to radio services; S3, performing service channel computing on discrete radio monitoring data; and S4, synthesizing each channel map to form an entire spectrum map. The distributed memory computing technology is adopted, the required data for main computing is persisted in the memory RAM for processing, the data disk reading and writing are reduced, the IO time is reduced, the IO bottleneck of the traditional multithreading scheme is solved, the monitoring data processing and computing speed is greatly accelerated, and the fast computing requirement of large-scale monitoring data is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of radio electromagnetic environment spectrum, and particularly relates to a radio spectrum map general parallel computing method. BACKGROUND

[0002] The radio electromagnetic spectrum map is often used for airwave signal distribution, air frequency usage, unknown signal / interference positioning and troubleshooting, and fine frequency assignment, and is an important technical means for government radio spectrum resource management and large event radio support.

[0003] At present, there are generally two schemes for generating spectrum map data based on radio monitoring data:

[0004] A fixed station or fixed-point monitoring method is adopted, full-band monitoring data is collected at a limited number of geographical observation points, and a spectrum map is generated on the corresponding two-dimensional geographical information plane through an interpolation algorithm. However, the calculation accuracy of this method is limited. A mobile road test method is adopted, a large amount of monitoring data is collected, configuration parameters are input, and a spectrum map interpolation algorithm is completed through offline preprocessing to give the calculation result. However, this method cannot effectively and quickly calculate and process the spectrum map for a large amount of mobile monitoring data.

[0005] With the further increase in the accuracy requirement of spectrum map data, the construction of a spectrum map through a number of fixed monitoring stations cannot meet the actual engineering requirements. Therefore, mobile road test collection, walking collection, grid node collection, and even airborne collection and other monitoring data collection methods are applied to the construction of a spectrum map, which inevitably leads to a substantial increase in the amount of monitoring data involved in the calculation of the spectrum map. At the same time, the performance of the monitoring equipment is relatively high, the frequency scanning speed is fast, and the scanning frequency range is large, which results in a large amount of monitoring data, further increasing the calculation amount of the spectrum map construction data processing.

[0006] On the other hand, with the continuous emergence of new business demands such as continuous spectrum situation playback and full-band signal spectrum map, the speed of spectrum map construction is facing unprecedented challenges.

[0007] For example, a "spectrum map construction and distribution method and system based on deep reinforcement learning" disclosed in Chinese patent literature, its publication number CN114116209A, the method models the joint offloading and resource management problem in the mobile edge network into the computing and communication compromise model in the mobile edge network, and constructs a centralized training and distributed execution reinforcement learning framework, which includes an offline training module and an online execution module. The online execution module uses the learned offloading strategy to perform step-by-step construction of the spectrum map, and the offline training module dynamically updates the online execution module according to the offloading calculation results of the cognitive user. However, this scheme still cannot quickly analyze and calculate massive radio monitoring road network data. SUMMARY

[0008] The present application mainly solves the problem of the prior art radio spectrum map data calculation and processing method that cannot quickly analyze and calculate massive radio monitoring road network data, and provides a radio spectrum map general parallel calculation method.

[0009] The above technical problems of the present application are mainly solved by the following technical scheme:

[0010] A radio spectrum map general parallel calculation method, comprising the following steps:

[0011] S1: After the original radio monitoring data is parallel analyzed in the distributed processing unit, filtering and screening is performed;

[0012] S2: According to the radio service, channelization processing is performed in the distributed computing unit in parallel, the calculation result data of each channel is mapped to the geographic area grid in a gridding manner, and each channel grid data is obtained;

[0013] S3: According to the discrete channel grid data, a data set required by each type of interpolation algorithm is pre-constructed, and unknown grid channel data is calculated by using a service channel interpolation algorithm.

[0014] S4: Each channel map is synthesized to form an entire spectrum map.

[0015] The present application adopts distributed memory calculation technology, and persistently stores data required for main calculation to the memory RAM for processing, reduces data disk reading and writing, reduces IO time, solves the IO bottleneck of the traditional multi-threaded scheme, greatly speeds up the monitoring data processing and calculation speed, and can solve the fast calculation demand of large-scale monitoring data. The data set required by each type of interpolation algorithm is pre-constructed, and the appropriate interpolation algorithm is directly used for parallel calculation to generate the channel map during calculation, which greatly improves the calculation speed.

[0016] As a preferred, the step S1 specifically comprises the following process:

[0017] S101: The processing center scheduling unit analyzes and assigns the monitoring task files, and transmits the assigned data file path to the distributed computing unit;

[0018] S102: After the distributed processing unit decompresses the original radio monitoring data record file, it reads each data frame of the record, and generates a monitoring data index file according to the file name and data information;

[0019] S103: The distributed processing unit processes several monitoring task files in parallel, and performs data analysis on the original binary data based on the radio data protocol; the parsed monitoring original data and the index file are batch imported into the columnar storage database;

[0020] S104: The parsed data information is filtered according to the task timing and frequency, and useless and repeated data information is removed.

[0021] The monitoring data parallel processing module finally loads the radio monitoring data into the columnar storage database in chronological order, realizing fast processing of massive radio monitoring data analysis. Through distributed parallel computing method, multiple monitoring data files are processed simultaneously by parallel computing method, improving the computing speed.

[0022] As a preferred, the index file comprises a one-byte monitoring service code field, a four-byte timestamp field and a four-byte monitoring frequency field in sequence.

[0023] As a preferred, the file index generation process comprises a timestamp field generation step and a monitoring service code field generation step in sequence.

[0024] The timestamp field generation step comprises:

[0025] Reading the frame header of each frame of monitoring data, obtaining the recording time of each frame of data, and converting the recording time into milliseconds, and storing the milliseconds in the timestamp field;

[0026] The monitoring service code field generation step comprises:

[0027] Reading the monitoring task file monitoring device basic information and monitoring task basic information, obtaining the administrative code, monitoring facility identification, monitoring station type, monitoring device model, monitoring device ID, monitoring task frequency range and monitoring task serial number;

[0028] The read data information is filtered and spliced in sequence to generate a complete monitoring service code field.

[0029] As preferred, the distributed computing unit calculates the center value, maximum value and average value of the monitoring data in the service bandwidth according to the service channel bandwidth of the relevant frequency band; maps the calculation result data to the geographic area grid in a rasterization manner, obtains each channel grid data, and inserts the grid data into the grid table of the columnar storage database to form the monitoring grid data.

[0030] As preferred, the step S2 specifically includes the following processes:

[0031] S201: rasterize and segment all the geographic areas to be calculated according to the minimum defined grid;

[0032] S202: automatically aggregate each monitoring data to the corresponding geographic grid according to the longitude and latitude of the monitoring data;

[0033] S203: obtain the aggregated grid monitoring data according to the mean value and maximum value algorithms for all the channelized monitoring data in each grid;

[0034] S204: obtain each channel grid data, and insert the grid data into the grid table of the columnar storage database to form the monitoring grid data.

[0035] According to the radio frequency bands and service definitions, the center frequency and channel bandwidth are set, the mean value, effective value and other channelized calculations are performed on the original monitoring data of different collection modes, and all parallel processing is performed to realize the concurrent calculation of each service channelization.

[0036] As preferred, the step S3 specifically includes the following processes:

[0037] S301: query the discrete monitoring grid data of each channel, and pre-construct the data set required by each type of interpolation algorithm according to the discrete monitoring grid data;

[0038] S302: select the appropriate interpolation algorithm to calculate and generate all the channel maps in parallel through input parameter configuration.

[0039] The data set required by each type of interpolation algorithm is pre-constructed, and the appropriate interpolation algorithm is directly used for parallel calculation to generate the channel map, which greatly improves the calculation speed.

[0040] As preferred, the step S4 specifically includes the following processes:

[0041] S401: take the channel map data generated by the right, and synthesize each channel map according to the time stamp and channel value dimension to form all the frequency spectrum map data;

[0042] S402: save the synthesized frequency spectrum map data to the frequency spectrum map data storage module.

[0043] The reading, filtering and aggregating channel map data, the data aggregation unit aggregates data according to the task ID and the task time sequence, synthesizes the complete spectrum map by combining each channel map, and stores the aggregated complete spectrum map data into a columnar storage database for user query.

[0044] The beneficial effects of the present application are:

[0045] 1. The distributed memory computing technology is used to persist the data required for main calculation to the memory RAM for processing, reduces the data disk reading and writing, reduces the IO time, solves the IO bottleneck of the traditional multi-threaded scheme, greatly accelerates the monitoring data processing and calculation speed, and can solve the fast calculation demand of large-scale monitoring data.

[0046] 2. The data set required by various interpolation algorithms is constructed in advance, and the appropriate interpolation algorithm is directly used for parallel calculation to generate the channel map during calculation, so that the calculation speed is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 It is a general parallel computing system connection block diagram of a radio spectrum map.

[0048] Figure 2 It is a general parallel computing method flow chart of a radio spectrum map.

[0049] In the figure, 1. monitoring data parallel processing module, 1-1. processing center scheduling unit, 1-2. distributed processing unit, 2. spectrum map interpolation parallel computing module, 2-1. computing center scheduling unit, 2-2. distributed computing unit, 3. spectrum map data aggregation module, 3-1. data filtering unit, 3-2. data aggregation unit, 4. spectrum map data storage module. DETAILED DESCRIPTION

[0050] The technical solutions of the present application will be further specifically described below by examples, and in combination with the drawings.

[0051] Embodiment:

[0052] The system used by the general parallel computing method of a radio spectrum map of the present embodiment, as shown in the figure, includes a monitoring data parallel processing module 1, a spectrum map interpolation parallel computing module 2, a spectrum map data aggregation module 3 and a spectrum map data storage module 4 connected in sequence. Figure 1

[0053] The monitoring data parallel processing module 1 includes a processing center scheduling unit 1-1 and a plurality of distributed processing units 1-2.

[0054] ​The monitoring data parallel processing module 1 is used for collecting and analyzing original radio monitoring data files, the distributed processing unit 1-2 reads original radio monitoring task binary files, performs parallel data analysis according to a monitoring device protocol specification, generates a task data parquet serialized file with a task ID and a task time as indexes, and stores the analyzed monitoring task data file into a distributed database.

[0055] The spectrum map interpolation parallel computing module 2 comprises a computing center scheduling unit 2-1 and a plurality of distributed computing units 2-2.

[0056] According to the radio frequency bands and service definitions, a center frequency and a channel bandwidth are set, mean value, effective value and other channelization calculations are performed on the original monitoring data of different collection modes, and all parallel processing is performed to realize concurrent channelization calculation of each service.

[0057] The spectrum map data aggregation module 3 comprises a data filtering unit 3-1 and a data aggregation unit 3-2.

[0058] The spectrum map data aggregation module 3 is used for reading, filtering and aggregating each channel map data, the data aggregation unit 3-2 aggregates data according to a task ID and a task time sequence, synthesizes each channel map to form a complete spectrum map, and stores the aggregated complete spectrum map data into a columnar storage database for user query.

[0059] A radio spectrum map general parallel computing method of the embodiment, as shown in Figure 2 S1: original radio monitoring data processing and analysis.

[0060] S101: The processing center scheduling unit 1-1 of the monitoring data parallel processing module 1 analyzes and distributes monitoring task files, and transmits the distributed data file path to the distributed computing unit 2-2.

[0061] S102: The distributed processing unit 1-2 of the monitoring data parallel processing module 1 decompresses the original radio monitoring data record file, reads each data frame of the record, and generates a monitoring data index file according to the file name and data information.

[0062] In the embodiment, the index file sequentially comprises a one-byte monitoring service code field, a four-byte timestamp field and a four-byte monitoring frequency field.

[0063] The generation process of the file index comprises a timestamp field generation step and a monitoring service code field generation step.

[0064] The timestamp field generation step comprises:

[0065] Read the frame header of each frame of monitoring data, get the recording time of each frame of data, and convert the recording time into milliseconds, and store the milliseconds into the timestamp field.

[0066] S103: The distributed processing unit 1-2 processes a plurality of monitoring task files in parallel, and performs data analysis on the original binary data based on the radio data protocol.

[0067] The data aggregation unit of the monitoring data parallel processing module 1 aggregates the parsed monitoring raw data and the index file in chronological order and batch imports them into the columnar storage database. The massive radio monitoring data analysis processing is realized quickly.

[0068] S104: The parsed data information is filtered according to the task timing and frequency, and useless data information and repeated data information are removed.

[0069] The monitoring service coding field generation step includes:

[0070] Read the monitoring task file monitoring equipment basic information and monitoring task basic information, get the administrative code, monitoring facility identification, monitoring station type, monitoring equipment model, monitoring equipment ID, monitoring task frequency range, and monitoring task serial number;

[0071] S105: The read data information is filtered and spliced in sequence to generate a complete monitoring service coding field.

[0072] S2: Channelization processing is performed according to the radio service.

[0073] The distributed computing unit 2-2 of the spectrum map interpolation parallel computing module 2 calculates the center value, maximum value and average value of the monitoring data in the service bandwidth according to the related frequency band service channel bandwidth definition, maps the calculation result data to the geographic area grid in a rasterization manner, obtains each channel grid data, and inserts the grid data into the columnar storage database grid table to form monitoring grid data.

[0074] S201: According to the minimum defined grid (generally requires a square with a side length less than 100 meters), the entire geographic area to be calculated is rasterized and segmented.

[0075] S202: According to the latitude and longitude of each monitoring data, it is automatically aggregated to the corresponding geographic grid.

[0076] S203: For all channelized monitoring data in each grid, the mean value / maximum value algorithm is used to obtain the aggregated grid monitoring data.

[0077] S204: Obtain each channel grid data, and insert the grid data into the columnar storage database grid table to form monitoring grid data.

[0078] S3: Service channel calculation on discrete radio monitoring data.

[0079] S301: The spectrum map interpolation parallel computing module 2 queries the discrete monitoring grid data of each channel, and according to the discrete monitoring grid data, a data set required by each type of interpolation algorithm is constructed in advance, such as a Delaunay triangular mesh data set and a Voronoi diagram data set.

[0080] S302: The spectrum map interpolation parallel computing module 2 generates all channel spectrum map data estimates in parallel by selecting input different algorithms and corresponding parameter configurations, and the system calculates and generates all channel spectrum map data estimates according to the corresponding interpolation algorithm. The interpolation algorithm includes but is not limited to Delaunay triangular mesh interpolation, natural neighborhood interpolation, IDW inverse distance weighted interpolation, and Kriging interpolation.

[0081] S4: Synthesizing each channel map to form a whole spectrum map.

[0082] The data filtering unit 3-1 of the spectrum map data aggregation module 3 corresponds to generate channel spectrum map data, filter invalid data and duplicate data, and the data set constitutes a complete channel spectrum map data.

[0083] The data aggregation unit 3-2 of the spectrum map data aggregation module 3 reads each channel spectrum map data, synthesizes each channel spectrum map according to the time stamp and channel value dimension, sorts and aggregates the data of the same time stamp according to the channel in the 30Khz-6000Mhz interval service channel, generates each time channel spectrum map, and then sorts according to the time sequence to form the spectrum map result data of the whole frequency band.

[0084] After the data aggregation unit 3-2 of the spectrum map data aggregation module 3 generates the frequency band spectrum map data, the frequency band spectrum map data is inserted into the spectrum map table of the spectrum map data storage module 4.

[0085] As shown in Table 1, the time used for radio spectrum map calculation based on different resource configurations is compared between the traditional scheme and the technical scheme of the present application.

[0086] Table 1. Comparison table of radio spectrum map calculation time

[0087]

[0088]

[0089] From the above table of experimental data, it can be seen that the method of the present application can greatly improve the calculation speed, and the speed is improved by more than ten times.

[0090] It should be understood that the examples are only used to illustrate the present application but not to limit the scope of the present application. Furthermore, it should be understood that after reading the present application, those skilled in the art can make various modifications or changes to the application, and these equivalent forms should also fall within the scope of the appended claims of the present application.

Claims

1. A method for radio spectrum map general purpose parallel computing, the method comprising: It comprises the following steps: S1: original radio monitoring data processing and analysis filtering; S2: according to the radio service, channelization processing is carried out, the calculation results of each channel are mapped to the geographical area grid in a rasterization manner to obtain each channel grid data; It comprises: according to the minimum defined grid, the geographical area to be calculated is segmented by rasterization; According to the longitude and latitude of each monitoring data, it is automatically aggregated to the corresponding geographical grid; For all channelized monitoring data in each grid, the aggregated grid monitoring data is obtained according to the mean and maximum value algorithms; Each channel grid data is obtained, and the grid data is inserted into the column storage database grid table to form monitoring grid data. S3: According to the discrete channel grid data, the data set required by each type of interpolation algorithm is constructed in advance, and the unknown grid channel data is calculated by using the business channel interpolation algorithm. S4: Each channel map is synthesized to form a whole spectrum map.

2. A general parallel computing method for radio spectrum map according to claim 1, characterized in that, The step S1 specifically It comprises the following processes: S101: The processing center scheduling unit analyzes and distributes the monitoring task file, and transmits the distributed data file path to the distributed computing unit; S102: The distributed processing unit decompresses the original radio monitoring data record file, reads each data frame of the record, and generates a monitoring data index file according to the file name and data information; S103: The distributed processing unit processes a plurality of monitoring task files in parallel, performs data analysis on the original binary data based on the radio data protocol; the parsed monitoring original data and the index file are batch imported into the column storage database; S104: The parsed data information is filtered according to the task time sequence and frequency, and useless and repeated data information is removed.

3. A method for general parallel computation of radio spectrum maps according to claim 2, characterized in that, The index file comprises a one-byte monitoring service code field, a four-byte timestamp field and a four-byte monitoring frequency field in sequence.

4. The method of claim 3, wherein, The generation process of the file index comprises a timestamp field generation step and a monitoring service code field generation step in sequence; The timestamp field generation step comprises: Reading the frame header of each monitoring data frame, obtaining the recording time of each data frame, and converting the recording time into milliseconds, and storing the milliseconds in the timestamp field; The monitoring service code field generation step comprises: Reading the monitoring task file monitoring equipment basic information and monitoring task basic information to obtain the administrative code, monitoring facility identification, monitoring station type, monitoring equipment model, monitoring equipment ID, monitoring task frequency range and monitoring task serial number; The read data information is filtered and spliced in sequence to generate a complete monitoring service code field.

5. The method of claim 1, wherein, The distributed computing unit calculates the center value, maximum value and average value of the monitoring data in the business bandwidth according to the related frequency band business channel bandwidth definition; the calculation result data is mapped to the geographical area grid in a rasterization manner to obtain each channel grid data, and the grid data is inserted into the column storage database grid table to form monitoring grid data.

6. The method of claim 1 or 5, wherein, The step S3 specifically It comprises the following processes: S301: Query the discrete monitoring grid data of each channel, and pre-construct the data set required by each type of interpolation algorithm according to the discrete monitoring grid data; S302: Calculate and generate the channel map in parallel through input parameter configuration and selection of the interpolation algorithm.

7. A general parallel computing method for radio spectrum map according to claim 6, wherein, The step S4 specifically includes the following processes: S401: Take the channel map data generated by the right, and synthesize each channel map according to the time stamp and channel value dimension to form the full spectrum map data; S402: Save the synthesized frequency spectrum map data to the spectrum map data storage module.

Images