Method, system, device, processor and storage medium for realizing grid signal coverage rate analysis and processing based on historical big data

Abstract

The present invention relates to a method for realizing grid signal coverage rate analysis and processing based on historical big data, comprising the following steps: selecting frequency band parameters and data filtering parameters to be analyzed; submitting analysis preparation and starting the analysis; screening data IDs, and submitting frequency band parameters and data filtering parameters; performing parallel analysis and calculation according to the frequency band; and saving the analysis results. The present invention also relates to a system, a device, a processor and a storage medium thereof for realizing grid signal coverage rate analysis and processing based on historical big data. By adopting the method, the system, the device, the processor and the storage medium thereof for realizing grid signal coverage rate analysis and processing based on historical big data, a large amount of historical data is utilized, repeated collection is avoided, the analysis speed is fast, the calculation logic structure is simple, and subsequent function expansion is facilitated. The analysis results are diversified, the coverage rates of different frequency bands and different signals can be analyzed at one time, the decoupling property is strong, the complex distributed calculation logic is simplified, and it is easy to implement and apply.

Description

Technical Field

[0001] The present invention relates to the field of radio technology, and particularly to the field of radio signal analysis. Specifically, it refers to a method, system, device, processor and its storage medium for realizing grid signal coverage analysis and processing based on historical big data. Background Art

[0002] With the vigorous development of radio technology and radio services, mastering the signal coverage rate of a national region is of great significance for signal spectrum planning and electromagnetic order maintenance. At the same time, as the systems of each station across the country are successively connected to the integrated platform, the radio monitoring platform has accumulated a large amount of data, providing a massive data basis for mastering the signal coverage rate of relevant regions.

[0003] Making full use of the advantages of the radio platform and aggregating massive data requires relevant signal coverage rates for convenient later overall planning. In view of the current requirements, the present invention provides a system and method for grid signal coverage analysis based on radio historical big data, which solves the problems of large data volume, wide geographical scope, and the ability to only analyze a single signal. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned disadvantages of the prior art and provide a method, system, device, processor and its storage medium for realizing grid signal coverage analysis and processing based on historical big data, which has strong decoupling, simple calculation logic, and diverse results.

[0005] In order to achieve the above purpose, the method, system, device, processor and its storage medium for realizing grid signal coverage analysis and processing based on historical big data of the present invention are as follows:

[0006] The method for realizing grid signal coverage analysis and processing based on historical big data is mainly characterized in that the method includes the following steps:

[0007] (1) Upload spectrum data to the background server, generate data associations and upload them to the Hadoop file system;

[0008] (2) Select the frequency band parameters and data filtering parameters to be analyzed;

[0009] (3) Submit the analysis preparation and start the analysis;

[0010] (4) The data screening module obtains the data id and submits the frequency band parameters and data filtering parameters to the Spark big data analysis cluster;

[0011] (5) The Spark computing cluster performs parallel analysis and calculation according to the frequency band;

[0012] (6) Save the analysis results to the Hadoop file system;

[0013] (7) After the data download module monitors the completion of the analysis, it automatically downloads the analysis results from the Hadoop file system;

[0014] (8) According to the intermediate frequency and bandwidth of the input signal, extract the analysis results of the corresponding signal from the analysis results.

[0015] Preferably, the step (5) specifically includes the following steps:

[0016] (5.1) Read the spectrum data file in parallel from the Hadoop distributed file system according to the data ID;

[0017] (5.2) Parse the frame data and filter the data according to the selected analysis range;

[0018] (5.3) Calculate the grid position of the frame data according to the data longitude and latitude;

[0019] (5.4) Merge and process the data according to the start frequency, cut-off frequency and bandwidth of the analysis frequency band;

[0020] (5.5) Calculate the noise floor threshold and the grid center position;

[0021] (5.6) Aggregate the data according to the grid center to count the signal coverage rate and level value;

[0022] (5.7) Convert the aggregated spark distributed data set into a signal frequency point data set;

[0023] (5.8) End the parallel loop calculation and save the analysis results to the Hadoop file system.

[0024] Preferably, in the step (5.3), calculating the grid position of the frame data includes calculating the longitude and latitude of the grid center of the frame data respectively, specifically:

[0025] Calculate the longitude of the grid center of the frame data according to the following formula:

[0026] xLon = (iXDelta_LON × rasterSize - rasterSize / 2) × M_LON + CONTROL_LON;

[0027] Among them, CONTROL_LON is the longitude of the grid reference point, rasterSize is the grid size of the set analysis frequency band, M_LON is the longitude value of 1 meter, and iXDelta_LON is the number of grids of the current point in the longitude;

[0028] Calculate the latitude of the grid center of the frame data according to the following formula:

[0029] xLat = (iXDelta_Lat × rasterSize - rasterSize / 2) × M_Lat + CONTROL_Lat;

[0030] Among them, CONTROL_Lat is the latitude of the grid reference point, rasterSize is the grid size for setting the analysis frequency band, M_Lat is the latitude value per meter, and iXDelta_Lat is the number of grids of the current point in the latitude direction.

[0031] Preferably, step (5.6) specifically includes the following steps:

[0032] (5.6.1) Save the grid center and frame data as an aggregated Spark distributed dataset;

[0033] (5.6.2) Form a grid matrix based on the frequency point values, number of data frames, level values of frequency points, and noise floor values of frequency points within the analysis frequency band;

[0034] (5.6.3) Calculate the signal occupancy rate and frequency band occupancy, and extract the signal coverage rate, maximum level, and average level.

[0035] Preferably, in step (5.6.3), calculating the signal occupancy rate is specifically as follows:

[0036] Calculate the signal occupancy rate according to the following formula:

[0037] OCC = Kmax / n;

[0038] Among them, OCC is the signal occupancy, Kmax is the number of data in the grid where the signal is greater than 1, and n is the number of data frames.

[0039] Preferably, in step (5.6.3), calculating the frequency band occupancy is specifically as follows:

[0040] Calculate the frequency band occupancy according to the following formula:

[0041] FredBandOcc = Omax / m;

[0042] Among them, FredBandOcc is the frequency band occupancy, Omax is the number of signals in the frequency band where the signal occupancy is greater than 1, and m is the total number of signals in the frequency band.

[0043] The system for realizing grid-based signal coverage rate analysis and processing based on historical big data is mainly characterized in that the system includes:

[0044] A data management layer for importing original spectrum files, generating data IDs, and uploading and downloading data files to and from the data file layer;

[0045] The analysis control layer, connected to the data management layer, is used to filter the data IDs that need to be analyzed, submit analysis parameters to the data file layer, and monitor and supervise the analysis process;

[0046] The parallel computing layer, connected to the analysis control layer, is used to receive analysis tasks and read and calculate data in parallel;

[0047] The data file layer, connected to the data management layer and the parallel computing layer, is used to store the original spectrum file and the analysis result file.

[0048] Preferably, the data management layer includes:

[0049] The data import module, connected to the data file layer and the analysis control layer, is used to import the original spectrum file in the spectrum evaluation format, generate data IDs, and upload them to the data file layer;

[0050] The data download module, connected to the data file layer and the parallel computing layer, is used to download the analysis results from the data file layer.

[0051] Preferably, the analysis control layer includes:

[0052] The data screening module, connected to the data management layer, is used to query all the data IDs that need to be analyzed according to conditions during the preparation process;

[0053] The analysis task control module, connected to the parallel computing layer, is used to control the analysis task and stop the analysis task during the analysis process;

[0054] The analysis task monitoring module, connected to the analysis task control module and the parallel computing layer, is used to monitor the analysis process;

[0055] The signal result extraction module, connected to the data screening module and the data file layer, is used to submit analysis parameters to the data file layer.

[0056] Preferably, the parallel computing layer includes:

[0057] The Spark cluster connection module, connected to the analysis control layer, is used to receive analysis tasks and start the Spark computing and analysis module;

[0058] The Spark computing and analysis module, connected to the Spark cluster connection module and the data file layer, is used to perform parallel analysis and calculation by frequency band.

[0059] Preferably, the data file layer includes the Hadoop file system, connected to the data management layer and the parallel computing layer, and is used to store the original spectrum file and the analysis result file.

[0060] The device for realizing grid signal coverage rate analysis and processing based on historical big data is characterized in that the device comprises:

[0061] A processor configured to execute computer-executable instructions;

[0062] A memory storing one or more computer-executable instructions, and when the computer-executable instructions are executed by the processor, each step of the method for realizing grid signal coverage rate analysis and processing based on historical big data as described above is realized.

[0063] The processor for realizing grid signal coverage rate analysis and processing based on historical big data is characterized in that the processor is configured to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, each step of the method for realizing grid signal coverage rate analysis and processing based on historical big data as described above is realized.

[0064] The computer-readable storage medium is characterized in that a computer program is stored thereon, and the computer program can be executed by a processor to realize each step of the method for realizing grid signal coverage rate analysis and processing based on historical big data as described above.

[0065] By adopting the method, system, device, processor and storage medium for realizing grid signal coverage rate analysis and processing based on historical big data of the present invention, a large amount of historical data is utilized, repeated collection is avoided, and by using the spark big data analysis platform, the analysis speed is fast, the calculation logic structure is simple, and it is convenient for later function expansion. The analysis results are diversified, the coverage rates of different frequency bands and different signals can be analyzed at one time, the decoupling is strong, the service system and the data processing system are completely separated, the mature spark big data calculation platform is used, the complex distributed calculation logic is simplified, and the focus is on the business logic, which is easy to implement and apply. Since the present invention is convenient and simple to implement, it is convenient for later business expansion and modification. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic diagram of the modules of the system for realizing grid signal coverage rate analysis and processing based on historical big data of the present invention.

[0067] Figure 2 It is a flowchart of the method for realizing grid signal coverage rate analysis and processing based on historical big data of the present invention.

[0068] Figure 3 It is a schematic diagram of the intermediate result of the analysis data of the method for realizing grid signal coverage rate analysis and processing based on historical big data of the present invention.

[0069] Figure 4Flowchart of the Spark analysis operator for the method of grid signal coverage rate analysis and processing based on historical big data of the present invention.

[0070] Figure 5 Schematic diagram of the device for grid signal coverage rate analysis and processing based on historical big data of the present invention.

[0071] Figure 6 Application schematic diagram of the embodiment of the method for grid signal coverage rate analysis and processing based on historical big data of the present invention. Detailed implementation manners

[0072] In order to more clearly describe the technical content of the present invention, the following will be further described in combination with specific embodiments.

[0073] The method for grid signal coverage rate analysis and processing based on historical big data of the present invention includes the following steps:

[0074] (1) Upload the spectrum data to the background server, generate data association and upload it to the Hadoop file system;

[0075] (2) Select the frequency band parameters and data filtering parameters to be analyzed;

[0076] (3) Submit the analysis preparation and start the analysis;

[0077] (4) The data screening module obtains the data id and submits the frequency band parameters and data filtering parameters to the Spark big data analysis cluster;

[0078] (5) The Spark computing cluster performs parallel analysis and calculation according to the frequency band;

[0079] (6) Save the analysis result to the Hadoop file system;

[0080] (7) After the data download module monitors the completion of the analysis, it automatically downloads the analysis result from the Hadoop file system;

[0081] (8) Extract the analysis result of the corresponding signal from the analysis result according to the intermediate frequency and bandwidth of the input signal.

[0082] As a preferred implementation manner of the present invention, step (5) specifically includes the following steps:

[0083] (5.1) Read the spectrum data file in parallel from the Hadoop distributed file system according to the data ID;

[0084] (5.2) Analyze the frame data and filter the data according to the selected analysis range;

[0085] (5.3) Calculate the grid position of this frame of data according to the data longitude and latitude;

[0086] (5.4) Merge and process the data according to the start frequency, cut-off frequency and bandwidth of the analysis frequency band;

[0087] (5.5) Calculate the background noise threshold and the grid center position;

[0088] (5.6) Aggregate the data according to the grid center to count the signal coverage rate and level value;

[0089] (5.7) Convert the aggregated spark distributed data set into a signal frequency point data set;

[0090] (5.8) End the parallel loop calculation and save the analysis result to the Hadoop file system.

[0091] As a preferred embodiment of the present invention, in the step (5.3) of calculating the grid position of this frame of data, it includes calculating the longitude and latitude of the grid center of this frame of data respectively, specifically:

[0092] Calculate the longitude of the grid center of this frame of data according to the following formula:

[0093] xLon = (iXDelta_LON × rasterSize - rasterSize / 2) × M_LON + CONTROL_LON;

[0094] Wherein, CONTROL_LON is the longitude of the grid reference point, rasterSize is the grid size of the set analysis frequency band, M_LON is the longitude value of 1 meter, and iXDelta_LON is the number of grids of the current point in the longitude;

[0095] Calculate the latitude of the grid center of this frame of data according to the following formula:

[0096] xLat = (iXDelta_Lat × rasterSize - rasterSize / 2) × M_Lat + CONTROL_Lat;

[0097] Wherein, CONTROL_Lat is the latitude of the grid reference point, rasterSize is the grid size of the set analysis frequency band, M_Lat is the latitude value of 1 meter, and iXDelta_Lat is the number of grids of the current point in the latitude.

[0098] As a preferred embodiment of the present invention, the step (5.6) specifically includes the following steps:

[0099] (5.6.1) Save the data according to the grid center and the frame data as an aggregated spark distributed data set;

[0100] (5.6.2) A grid matrix is formed according to the frequency point values, the number of data frames, the level values of the frequency points, and the noise floor values of the frequency points within the analysis frequency band;

[0101] (5.6.3) Calculate the signal occupancy rate and the frequency band occupancy rate, and extract the signal coverage rate, the maximum level, and the average level.

[0102] As a preferred embodiment of the present invention, in the step (5.6.3), calculating the signal occupancy rate specifically includes:

[0103] Calculate the signal occupancy rate according to the following formula:

[0104] OCC = Kmax / n;

[0105] Wherein, OCC is the signal occupancy, Kmax is the number of data in the grid where the signal is greater than 1, and n is the number of data frames.

[0106] As a preferred embodiment of the present invention, in the step (5.6.3), calculating the frequency band occupancy rate specifically includes:

[0107] Calculate the frequency band occupancy rate according to the following formula:

[0108] FredBandOcc = Omax / m;

[0109] Wherein, FredBandOcc is the frequency band occupancy, Omax is the number of signals in the frequency band where the signal occupancy is greater than 1, and m is the total number of signals in the frequency band.

[0110] The system for realizing grid - based signal coverage rate analysis and processing based on historical big data of the present invention includes:

[0111] A data management layer, used for importing the original spectrum file, generating a data ID, and uploading and downloading data files to and from the data file layer;

[0112] An analysis control layer, connected to the data management layer, used for screening the data IDs to be analyzed, submitting analysis parameters to the data file layer, and monitoring and detecting the analysis process;

[0113] A parallel computing layer, connected to the analysis control layer, used for receiving analysis tasks and reading and calculating data in parallel;

[0114] A data file layer, connected to the data management layer and the parallel computing layer, used for storing the original spectrum file and the analysis result file.

[0115] As a preferred embodiment of the present invention, the data management layer includes:

[0116] A data import module, connected to the data file layer and the analysis control layer, for importing the original spectrum file in the spectrum evaluation format, generating a data ID, and uploading it to the data file layer;

[0117] A data download module, connected to the data file layer and the parallel computing layer, for downloading the analysis results from the data file layer.

[0118] As a preferred embodiment of the present invention, the analysis control layer includes:

[0119] A data screening module, connected to the data management layer, for querying all data IDs that need to be analyzed according to conditions during the preparation process;

[0120] An analysis task control module, connected to the parallel computing layer, for controlling the analysis task and stopping the analysis task during the analysis process;

[0121] An analysis task monitoring module, connected to the analysis task control module and the parallel computing layer, for monitoring the analysis process;

[0122] A signal result extraction module, connected to the data screening module and the data file layer, for submitting the analysis parameters to the data file layer.

[0123] As a preferred embodiment of the present invention, the parallel computing layer includes:

[0124] A Spark cluster connection module, connected to the analysis control layer, for receiving the analysis task and starting the Spark computing analysis module;

[0125] A Spark computing analysis module, connected to the Spark cluster connection module and the data file layer, for parallel analysis and calculation according to frequency bands.

[0126] As a preferred embodiment of the present invention, the data file layer includes a Hadoop file system, connected to the data management layer and the parallel computing layer, for storing the spectrum original file and the analysis result file.

[0127] The device for realizing grid signal coverage analysis and processing based on historical big data of the present invention, wherein the device includes:

[0128] A processor, configured to execute computer-executable instructions;

[0129] A memory, storing one or more computer-executable instructions, and when the computer-executable instructions are executed by the processor, each step of the method for realizing grid signal coverage analysis and processing based on historical big data as described above is implemented.

[0130] The processor for grid signal coverage rate analysis and processing based on historical big data according to the present invention, wherein the processor is configured to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, each step of the method for grid signal coverage rate analysis and processing based on historical big data as described above is implemented.

[0131] The computer-readable storage medium of the present invention, on which a computer program is stored, and the computer program can be executed by a processor to implement each step of the method for grid signal coverage rate analysis and processing based on historical big data as described above.

[0132] In the specific implementation manner of the present invention, the system is divided into four layers, including:

[0133] The data management layer is responsible for importing the original spectrum file in the spectrum evaluation format into the service, generating a data ID, and then uploading it to the Hadoop file system (HDFS). After the analysis is completed, the analysis result is downloaded from the Hadoop file system.

[0134] The analysis control layer is responsible for querying all the data IDs to be analyzed according to conditions such as time and data name when preparing to start a task; and submitting the analysis parameters to Spark, and monitoring the analysis process, and can control the stop of the analysis.

[0135] The parallel computing layer is responsible for starting Spark and calculating data in parallel.

[0136] The data file layer is responsible for storing the original spectrum file and the analysis result file.

[0137] The method for grid signal coverage rate analysis and processing based on historical big data according to the present invention, including the following steps:

[0138] 1. Upload the spectrum data to the background server through the data management layer, generate data association and upload it to the HDFS platform, and save it according to the task ID and time granularity (the data format is the spectrum evaluation format);

[0139] 2. Select the analysis frequency band parameters (starting frequency, cut-off frequency, bandwidth, automatic threshold, grid accuracy), and multiple frequency bands can be selected, as shown in Table 1;

[0140] Table 1

[0141]

[0142] 3. Select the analysis data filtering parameters (data name, time);

[0143] 4. The analysis task control module is responsible for starting the analysis and stopping the analysis task in the cluster during the analysis process;

[0144] 5. The data screening module submits the required analysis parameters and data parameters to the Spark big data analysis cluster;

[0145] 6. The Spark computing cluster receives the submitted analysis tasks and performs parallel analysis and calculation by frequency band according to the preset calculation module;

[0146] 7. After the Spark parallel analysis and calculation is completed, the analysis results are saved to the Hadoop cluster file system;

[0147] 8. After the data control module detects the completion of the analysis, it automatically downloads the analysis results from the HDFS file system;

[0148] 9. The central service finds the corresponding signal from the analysis results according to the intermediate frequency and bandwidth of the signal input on the web side and displays it.

[0149] Among them, the calculation steps of the Spark analysis operator are as follows:

[0150] (1) Parallelly read the road test spectrum data file from the Hadoop distributed file system (HDFS) according to the data ID;

[0151] (2) Parse the frame data according to the spectrum evaluation format uniformly specified by the country and filter the data according to the selected analysis range;

[0152] (3) Calculate the grid position of the frame data according to the data longitude and latitude. The method is as follows:

[0153] Set the grid reference point (CONTROL_LON, CONTROL_Lat), calculate the distance xLonDis between the point (DataLon, CONTROL_Lat) and the point (CONTROL_LON, CONTROL_Lat), then the number of grids iXDelta_LON of the current point in the longitude direction is as follows:

[0154] iXDelta_LON = xLonDis / rasterSize

[0155] Then the longitude xLon of the center of the grid of the frame data is as follows:

[0156] xLon = (iXDelta_LON × rasterSize - rasterSize / 2) × M_LON + CONTROL_LON

[0157] Among them, CONTROL_LON and CONTROL_Lat are the longitude and latitude of the grid reference point, DataLon is the longitude of the data, rasterSize is the grid size of the set analysis frequency band, and M_LON is the longitude value of 1 meter.

[0158] The latitude xLat at the center of the frame data grid can be calculated using the same method.

[0159] (4) Process the data through a decimation algorithm according to the start frequency, stop frequency, and bandwidth of the analysis frequency band (the bandwidth formula for analysis is a multiple of the data scan);

[0160] (5) Calculate the background noise of this frame of data according to the threshold of the analysis frequency band;

[0161] (6) Save the distributed dataset with the grid center as the key and the data T of this frame as the value:

[0162] JavaPairRDD<String, T> freqbandDatas = data.mapToPair(...);

[0163] The storage structure of the dataset is as Figure 3 shown.

[0164] (7) Extract the information of the frequency band and signal according to one grid:

[0165] Based on the aggregated spark distributed dataset freqbandDatas, a matrix can be formed with signal frequency points as columns, number of data frames as rows, and level values as values, as shown in Table 2.

[0166] Table 2

[0167] F1 F2 ... Fm 1 D11 D12 ... D1m 2 D21 D22 ... D2m ... ... ... ... ... n Dn1 Dn2 ... Dnm

[0168] F1 F2 ... Fm 1 T11 T12 ... T1m 2 T21 T22 ... T2m ... ... ... ... ... n Tn1 Tn2 ... Tnm

[0169] Among them, Fm is the frequency point value within the analysis frequency band, n is the number of data frames, Dnm is the level value of the original data signal frequency point, Tnm is the background noise value of the corresponding original data signal frequency point. If Dnm > Tnm, the value at the i-th row and j-th column of the matrix is 1, otherwise it is 0. Finally, the matrix can be obtained, as shown in Table 3.

[0170] Table 3

[0171] F1 F2 ... Fm 1 0 1 ... 1 2 0 1 ... 1 ... ... ... ... ... n 1 0 ... 1

[0172] According to the above table, the signal coverage rate, maximum level, and average level of the signal in one grid can be extracted.

[0173] Among them, the formula for calculating the signal occupancy rate is:

[0174] OCC = Kmax / n;

[0175] Among them, OCC is the signal occupancy, Kmax is the number of times the signal is greater than 1 in all data in one grid, and n is the number of data frames.

[0176] The calculation rule for the frequency band occupancy is as follows:

[0177] FredBandOcc = Omax / m;

[0178] Wherein, FredBandOcc is the frequency band occupancy, Omax is the number of signals with a signal occupancy greater than 1 in the frequency band, and m is the total number of signals in the frequency band.

[0179] The analysis result data of a signal in a grid includes the following information:

[0180] <Intermediate frequency of the signal, signal bandwidth, level value, coverage rate, longitude of the grid point, latitude of the grid point>

[0181] Wherein, the intermediate frequency of the signal is the frequency point of the frequency band, the signal bandwidth is the bandwidth of the analyzed frequency band, the level is the maximum level, and the coverage rate is the frequency point occupancy.

[0182] (8) Save the analysis results according to the signal frequency points:

[0183] Convert the aggregated spark distributed data set freqbandDatas into new data:

[0184] JavaPairRDD<K, M> meshDataResult = freqbandDatas.flatMapToPair(...);

[0185] Wherein, K is the intermediate frequency of the signal, and M is the analysis result data structure of the signal in a grid, including the following information:

[0186] <Intermediate frequency of the signal, signal bandwidth, level value, coverage rate, longitude of the grid point, latitude of the grid point>

[0187] Wherein, for the frequency band coverage rate, the frequency point is denoted as 0.

[0188] (9) Save the data result to hdfs.

[0189] For the specific implementation solution of this embodiment, reference can be made to the relevant descriptions in the above embodiments, which will not be elaborated here.

[0190] It can be understood that the same or similar parts in the above embodiments can be referred to each other, and the content not described in detail in some embodiments can be referred to the same or similar content in other embodiments.

[0191] It should be noted that in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "a plurality of" refers to at least two.

[0192] Any process or method description depicted in the flowchart or described otherwise herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present invention includes additional implementations where functions may be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0193] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0194] Those of ordinary skill in the art of the present technology can understand that all or part of the steps carried by the method of the above embodiments can be completed by instructing relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0195] In addition, in each embodiment of the present invention, the functional units can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0196] The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disk, etc.

[0197] In the description of this specification, descriptions with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0198] The method, system, device, processor and storage medium for realizing grid signal coverage analysis and processing based on historical big data of the present invention are adopted. By utilizing a large amount of historical data, repeated collection is avoided. With the spark big data analysis platform, the analysis speed is fast, the calculation logic structure is simple, and it is convenient for later function expansion. The analysis results are diversified, and the coverage rates of different frequency bands and different signals can be analyzed at one time. The decoupling is strong, and the service system and the data processing system are completely separated. The mature spark big data calculation platform is used, which simplifies the complex distributed calculation logic required and focuses on the business logic, making it easy to implement and apply. Since the present invention is convenient and simple to implement, it is convenient for later business expansion and modification.

[0199] In this specification, the present invention has been described with reference to its specific embodiments. However, it is obvious that various modifications and transformations can still be made without departing from the spirit and scope of the present invention. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method for realizing grid signal coverage rate analysis and processing based on historical big data, characterized in that, the method includes the following steps: (1) Upload spectrum data to the background server, generate data association and upload it to the Hadoop file system; (2) Select the frequency band parameters and data filtering parameters to be analyzed; (3) Submit analysis preparation and start analysis; (4) The data screening module obtains the data id and submits the frequency band parameters and data filtering parameters to the Spark big data analysis cluster; (5) The Spark computing cluster performs parallel analysis and calculation according to the frequency band; (6) Save the analysis result to the Hadoop file system; (7) After the data download module detects that the analysis is completed, it automatically downloads the analysis result from the Hadoop file system; (8) Extract the analysis result of the corresponding signal from the analysis result according to the intermediate frequency and bandwidth of the input signal; The specific steps of the step (5) include the following steps: (5.1) Parallelly read the spectrum data file from the Hadoop distributed file system according to the data ID; (5.2) Parse the frame data and filter the data according to the selected analysis range; (5.3) Calculate the grid position of the frame data according to the data longitude and latitude; (5.4) Merge and process the data according to the start frequency, cut-off frequency and bandwidth of the analysis frequency band; (5.5) Calculate the noise floor threshold and the grid center position; (5.6) Aggregate the data according to the grid center to count the signal coverage rate and level value; (5.7) Convert the aggregated spark distributed data set into a signal frequency point data set; (5.8) End the parallel loop calculation and save the analysis result to the Hadoop file system.

2. The method for realizing grid signal coverage rate analysis and processing based on historical big data according to claim 1, characterized in that, in the step (5.3), calculating the grid position of the frame data includes calculating the longitude and latitude of the grid center of the frame data respectively, specifically: Calculate the longitude of the grid center of the frame data according to the following formula: xLon = (iXDelta_LON × rasterSize - rasterSize / 2) × M_LON + CONTROL_LON; where, CONTROL_LON is the longitude of the grid reference point, rasterSize is the grid size of the set analysis frequency band, M_LON is the longitude value of 1 meter, and iXDelta_LON is the grid number of the current point in the longitude; Calculate the latitude of the grid center of the frame data according to the following formula: xLat = (iXDelta_Lat × rasterSize - rasterSize / 2) × M_Lat + CONTROL_Lat; where, CONTROL_Lat is the latitude of the grid reference point, rasterSize is the grid size of the set analysis frequency band, M_Lat is the latitude value of 1 meter, and iXDelta_Lat is the grid number of the current point in the latitude.

3. The method for realizing grid signal coverage rate analysis and processing based on historical big data according to claim 1, characterized in that, The specific steps of step (5.6) include the following steps: (5.6.1) Save the grid center and frame data as an aggregated spark distributed dataset; (5.6.2) Compose a grid matrix according to the frequency point values, number of data frames, level values of frequency points, and noise floor values within the analysis frequency band; (5.6.3) Calculate the signal occupancy rate and frequency band occupancy, and extract the signal coverage rate, maximum level, and average level.

4. The method for realizing grid-based signal coverage rate analysis and processing based on historical big data according to claim 3, characterized in that, In step (5.6.3), calculating the signal occupancy rate specifically is: Calculate the signal occupancy rate according to the following formula: OCC = Kmax / n; where OCC is the signal occupancy, Kmax is the number of data in the grid where the signal is greater than 1, and n is the number of data frames.

5. The method for realizing grid-based signal coverage rate analysis and processing based on historical big data according to claim 3, characterized in that, In step (5.6.3), calculating the frequency band occupancy specifically is: Calculate the frequency band occupancy according to the following formula: FredBandOcc = Omax / m; where FredBandOcc is the frequency band occupancy, Omax is the number of signals in the frequency band where the signal occupancy is greater than 1, and m is the total number of signals in the frequency band.

6. A system for realizing grid-based signal coverage rate analysis and processing based on historical big data, characterized in that, The system includes: A data management layer, used to import the original spectrum file, generate a data ID, and perform data upload and download on the data file layer; An analysis control layer, connected to the data management layer, used to screen the data IDs that need to be analyzed, submit the analysis parameters to the data file layer, and monitor and supervise the analysis process; A parallel computing layer, connected to the analysis control layer, used to receive analysis tasks and perform parallel data reading and calculation; A data file layer, connected to the data management layer and the parallel computing layer, used to store the original spectrum file and the analysis result file; The parallel computing layer includes: A Spark cluster connection module, connected to the analysis control layer, used to receive analysis tasks and start the Spark calculation and analysis module; A Spark calculation and analysis module, connected to the Spark cluster connection module and the data file layer, used to perform parallel analysis and calculation by frequency band; The parallel analysis and calculation by frequency band of the Spark calculation and analysis module specifically includes the following steps: (5.1) Parallelly read the spectrum data file from the Hadoop distributed file system according to the data ID; (5.2) Parse the frame data and filter the data according to the selected analysis range; (5.3) Calculate the grid position of the frame data according to the data longitude and latitude; (5.4) Merge and process the data according to the start frequency, stop frequency, and bandwidth of the analysis frequency band; (5.5) Calculate the noise floor threshold and the grid center position; (5.6) Aggregate the data by the grid center to statistically calculate the signal coverage rate and level value; (5.7) Convert the aggregated spark distributed dataset into a signal frequency point dataset; (5.8) End the parallel loop calculation and save the analysis results to the Hadoop file system.

7. The system for realizing grid signal coverage rate analysis and processing based on historical big data according to claim 6, wherein, the data management layer includes: a data import module, connected to the data file layer and the analysis control layer, for importing the original spectrum file in the spectrum evaluation format, generating a data ID, and uploading it to the data file layer; a data download module, connected to the data file layer and the parallel computing layer, for downloading the analysis results from the data file layer.

8. The system for realizing grid signal coverage rate analysis and processing based on historical big data according to claim 6, wherein, the analysis control layer includes: a data screening module, connected to the data management layer, for querying all data IDs to be analyzed according to conditions during the preparation process; an analysis task control module, connected to the parallel computing layer, for controlling the analysis task and stopping the analysis task during the analysis process; an analysis task monitoring module, connected to the analysis task control module and the parallel computing layer, for monitoring the analysis process; a signal result extraction module, connected to the data screening module and the data file layer, for submitting the analysis parameters to the data file layer.

9. The system for realizing grid signal coverage rate analysis and processing based on historical big data according to claim 6, wherein, the data file layer includes the Hadoop file system, connected to the data management layer and the parallel computing layer, for storing the spectrum original file and the analysis result file.

10. An apparatus for realizing grid signal coverage rate analysis and processing based on historical big data, wherein, the apparatus includes: a processor configured to execute computer-executable instructions; a memory storing one or more computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method for realizing grid signal coverage rate analysis and processing based on historical big data according to any one of claims 1 to 5 are implemented.

11. A processor for realizing grid signal coverage rate analysis and processing based on historical big data, wherein, the processor is configured to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method for realizing grid signal coverage rate analysis and processing based on historical big data according to any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium, wherein, a computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method for realizing grid signal coverage rate analysis and processing based on historical big data according to any one of claims 1 to 5.

Images