Ultra-short wave radio signal monitoring and data analysis method and system

Abstract

The invention provides an ultrashort wave radio signal monitoring data analysis system, and aims to improve the monitoring data processing efficiency and flexibility. The system comprises a data preprocessing module, a monitoring data warehouse and a data analysis module. The data preprocessing module obtains data from a monitoring original database, processes the data through a data driving layer, a data cleaning layer and a data integration layer, and stores the data in a monitoring data warehouse according to a unified format. The monitoring data warehouse comprises a unified format database, a monitoring sample library and a monitoring analysis result library which are respectively used for storing formatted data, spectrum characteristic information and historical analysis results. And the data analysis module provides a service component and an application component, supports a user-defined analysis request and generates a visual result. Through multi-dimensional classified storage and abnormal data discrimination, the system improves data statistics and analysis efficiency, data accuracy and reliability are improved by adopting a data compression and automatic ground noise calculation method, and efficient technical support is provided for monitoring and management of ultrashort wave radio signals.

Description

Technical Field

[0001] This invention relates to a method and system for monitoring and analyzing ultra-shortwave radio signals, particularly a technology for real-time monitoring, analysis, and data processing of ultra-shortwave radio signals, which is widely used in fields such as communications, power, military, and transportation. Background Technology

[0002] Currently, the efficient management and rational use of radio spectrum are crucial for ensuring the stability of communication networks and the order of radio waves in the air. To ensure the scientific utilization of radio resources, existing radio monitoring systems typically rely on technologies from different manufacturers, and these systems suffer from various problems in data acquisition, storage, and analysis. For example, there are issues with inconsistent data formats, non-standardized storage methods, and low data processing efficiency. This makes it difficult to flexibly extract, merge, and analyze multi-dimensional data after collecting large amounts of data, thus affecting system performance and analytical efficiency. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes an improved VHF / UHF radio signal monitoring data processing system. This system optimizes data processing efficiency and enhances the system's analytical capabilities and flexibility through preprocessing, compressed storage, and multi-dimensional analysis of signal data. The technical solution of this invention is as follows:

[0004] The ultra-shortwave radio signal monitoring data processing system provided by this invention includes the following modules:

[0005] Data preprocessing module: Retrieves data from the original monitoring database, processes and compresses the data, and then stores it in the monitoring data warehouse. Monitoring data warehouse: Used to store preprocessed data. The data is stored in a unified format and can be redundantly stored according to different dimensions (such as frequency domain, time domain, etc.).

[0006] Data analysis module: Analyzes and processes the stored data and returns corresponding statistical and analytical results based on user requests.

[0007] The data preprocessing module includes the following sub-modules:

[0008] Data acquisition layer: Used to collect different types of monitoring data from multiple heterogeneous data sources and perform preliminary format conversion and standardization processing.

[0009] Data cleaning layer: Uses data screening algorithms to remove invalid data (such as outliers and noise) to ensure the accuracy of subsequent analysis data. Data integration layer: Compresses and transforms data in multiple dimensions to ensure optimized storage according to analytical needs.

[0010] The monitoring data warehouse consists of the following three parts:

[0011] Uniformly formatted database: Stores pre-processed data in a uniformly formatted way for subsequent analysis.

[0012] Monitoring Sample Library: Stores data samples of different times, frequencies, and frequency bands, providing them to the real-time monitoring system for comparison and analysis.

[0013] Monitoring and Analysis Results Repository: Used to store analysis results and provide support for trend analysis and historical data statistics.

[0014] The data analysis module includes:

[0015] Service layer: Responsible for statistical analysis and summarization of results, which are then available for use by the application layer.

[0016] Application layer: Responds to user requests and displays data analysis results.

[0017] Data cleaning and outlier removal algorithms

[0018] This invention proposes an algorithm for outlier detection and removal in radio monitoring data, the calculation steps of which are as follows:

[0019] Set a continuity threshold (Lc) and a proportion threshold (Lp) for outlier data.

[0020] The dataset is sorted and divided into multiple subgroups based on thresholds.

[0021] Calculate the data percentage of each subgroup. If the percentage of a certain group is lower than the threshold, it is judged as abnormal data and removed.

[0022] Data compression and multi-dimensional storage

[0023] During data integration, the system employs multi-dimensional data compression technology to compress the raw data and categorize and store it according to analytical needs. Data is stored with integer precision, and redundant storage strategies are used for each dimension (such as frequency domain, time domain, and spatial domain) to improve the efficiency of subsequent statistical analysis.

[0024] advantage

[0025] The storage and processing methods following data preprocessing improve data access efficiency, especially in the analysis and processing of large-scale data, significantly reducing computational burden. The system supports flexible multi-dimensional data extraction and statistical analysis, enabling data extraction, segmentation, and merging operations according to different business needs. An efficient outlier removal algorithm ensures the accuracy of analysis results, avoiding errors caused by outliers. Data compression technology significantly reduces storage requirements while ensuring data scalability. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall architecture of the present invention.

[0027] Figure 2 This is a schematic diagram of the monitoring data warehouse of the present invention.

[0028] Figure 3 This is a schematic diagram of the noise data distribution.

[0029] Figure 4 This is a schematic diagram of the distribution of abnormal signal data.

[0030] Figure 5 This is a diagram illustrating the results of data processing. Detailed Implementation

[0031] For ease of understanding, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] This invention proposes a system for monitoring and analyzing ultra-shortwave radio signals. Please refer to [link / reference]. Figure 1 The application software of this monitoring system is used for the collection of raw data, and all data obtained from the monitoring is stored in the raw monitoring database. The system includes a data preprocessing module, a monitoring data warehouse, and a data analysis module.

[0033] 1. Data Preprocessing Module

[0034] The data preprocessing module obtains data from the raw monitoring database and stores it in the monitoring data warehouse after processing and compression. This module adopts a client / server architecture to meet high-speed data processing requirements and uses specific algorithms to process and integrate the data. The data preprocessing process can be divided into the following three levels:

[0035] 1) Data-driven layer: Responsible for acquiring data from radio monitoring systems of different manufacturers and formats, and converting it into a standardized format. Since different data formats exist in the market, the data-driven layer designs multiple driver modules based on different data source formats to ensure lossless data acquisition and formatted output.

[0036] 2) Data Cleaning Layer: This layer removes outliers from the acquired formatted data. By employing anomaly detection algorithms, invalid or abnormal data is removed to prevent interference with subsequent analysis. The data cleaning layer effectively removes noisy, inconsistent, and null data, ensuring the accuracy and reliability of the data.

[0037] 3) Data Integration Layer: This layer is responsible for compressing data and performing multi-dimensional transformations before finally storing it in the monitoring data warehouse. During data compression, pre-statistical sampling techniques are used to reduce data volume while ensuring data integrity and efficient query capabilities, based on business needs and statistical requirements. Furthermore, through multi-dimensional transformations (such as frequency domain, time domain, and spatial domain), data is redundantly stored according to different dimensions to support subsequent flexible data extraction and statistical analysis.

[0038] 4) The data preprocessing module is designed in a layered and modular manner, which allows the system performance to be optimized in the future by replacing or upgrading individual layer modules without the need for large-scale system modifications.

[0039] 2. Monitor the data warehouse

[0040] The monitoring data warehouse stores data processed by the data preprocessing module, and all data is stored in a uniform format. It is a subject-oriented, integrated, dynamically updated, non-volatile collection of data. Unlike traditional transactional databases, the monitoring data warehouse focuses on data analysis and decision support, organizing data using a multidimensional data cube model.

[0041] Data in a data warehouse is divided into three main storage categories based on different analytical needs:

[0042] 1) Uniformly formatted database: This type of database stores raw data that has been compressed by the data preprocessing module and stored in a uniform format. This type of data occupies most of the warehouse's storage space.

[0043] 2) Monitoring Sample Library: Data from various time periods and frequency ranges are extracted from the raw monitoring data to establish a frequency usage database for comparison and analysis of the real-time monitoring system.

[0044] 3) Monitoring and Analysis Results Database: Stores the results of various statistical analyses, forming a historical trend analysis database to support future data analysis.

[0045] 3. Data Analysis Module

[0046] The data analysis module is responsible for retrieving and analyzing data from the data warehouse. This module adopts a B / S architecture, allowing users to operate it via a web interface, facilitating access for different users. To improve system scalability, the data analysis module is divided into a two-layer architecture: service components and application components.

[0047] 1) Service component: responsible for summarizing all analysis results and providing them to application components for use.

[0048] 2) Application components: Respond to user analysis requests, obtain and display analysis results.

[0049] The data analysis module, as the core of the system, operates in a component-based manner and adopts a flexible configuration approach, providing customized data analysis services according to different needs. Depending on the user's requirements, the system can select and configure different statistical analysis components, thereby enhancing the system's flexibility and scalability.

[0050] 4. Handling of Abnormal Data

[0051] During data acquisition, sudden electromagnetic interference such as electrical sparks and lightning may occur, causing anomalies in the acquired signal data. These anomalies may affect the accuracy of the entire spectrum analysis and even trigger a cumulative effect, resulting in distorted final statistical results. Therefore, the cleaning layer in the data preprocessing module employs anomaly detection and removal algorithms to ensure data quality.

[0052] The basic principle of outlier detection is to group the data according to the principle of continuity and calculate the proportion of each group in the total data. When the proportion is less than a preset outlier probability threshold, it is considered outlier and removed. Specific steps include:

[0053] 1) Set a continuity threshold (e.g., 2dB) to divide the data into groups.

[0054] 2) Sort and group all the data.

[0055] 3) Calculate the percentage of each group of data. If the percentage is lower than the preset threshold, the group is considered abnormal data and is removed.

[0056] 5. Data compression and organization

[0057] At the data integration layer, data processing techniques are used to compress the cleaned data. The compression algorithm is based on the data's distribution characteristics, reducing the data volume through fine-grained statistics and summarization without affecting the accuracy of subsequent statistical analyses. For example, in signal strength data for a certain channel, when the signal is interference-free, the data follows a normal distribution; however, when there is a stable signal, the data tends to concentrate within a certain range. Based on this characteristic, the system organizes and statistically processes the signal strength data, thereby compressing it to an acceptable data volume.

[0058] 6. Automatic noise floor calculation

[0059] In data analysis, noise floor is an important indicator of signal quality. Automatic noise floor calculation technology automatically calculates the noise floor value of a frequency band based on the signal strength distribution characteristics, and adjusts the threshold accordingly to optimize the signal detection process.

[0060] The calculation steps include:

[0061] 1) Organize and organize the collected signal strength data.

[0062] 2) Calculate the cumulative probability of signal strength.

[0063] 3) By performing layered calculations, the noise floor value and threshold are determined, thereby adapting to different types of frequency bands.

[0064] Summarize

[0065] This invention utilizes multi-level data preprocessing, compression, and analysis techniques to efficiently process and store VHF / UHF radio signal data, ensuring the accuracy and efficiency of data analysis. Through modular design, the system possesses strong flexibility and scalability, capable of meeting the real-time data analysis and trend prediction needs of diverse business applications.

[0066] The above embodiments are merely examples of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A system for monitoring and analyzing ultra-shortwave radio signals, characterized in that, The system includes: a data preprocessing module for acquiring radio signal monitoring data from the original monitoring database and preprocessing the data, the data preprocessing including data-driven layer, data cleaning layer, and data integration layer processing; a monitoring data warehouse for storing the processed data, the monitoring data warehouse including a uniformly formatted database, a monitoring sample database, and a monitoring analysis result database, the uniformly formatted database for storing formatted data after data cleaning, the monitoring sample database for storing monitoring sample data, and the monitoring analysis result database for storing historical analysis results; and a data analysis module for performing statistical analysis on the data stored in the monitoring data warehouse, the data analysis module including service components and application components, the service components for performing flexible statistical analysis on the data and supporting user-defined analysis requests, and the application components for generating visualized statistical results.

2. The ultra-shortwave radio signal monitoring and data analysis system according to claim 1, characterized in that, The data preprocessing module processes the raw data using anomaly detection and removal algorithms, data compression technology, and automatic noise floor calculation methods. The UHF radio signal monitoring data analysis system according to claim 1 is characterized in that the data analysis module supports the classification, storage and analysis of data based on multiple dimensions (frequency domain, time domain, spatial domain, etc.).

3. The ultra-shortwave radio signal monitoring and data analysis system according to claim 1, characterized in that, The unified format database in the monitoring data warehouse can store data from different sources in a unified format to facilitate subsequent processing and analysis.

4. The ultra-shortwave radio signal monitoring and data analysis system according to claim 1, characterized in that, The data analysis module also includes the function of performing regression analysis on historical data in order to predict future signal characteristics.

5. The ultra-shortwave radio signal monitoring and data analysis system according to claim 1, characterized in that, The system also includes a data visualization module, which converts data analysis results into a graphical interface for users to view and operate.

6. The ultra-shortwave radio signal monitoring and data analysis system according to claim 1, characterized in that, The data preprocessing module automatically filters noise data according to preset standards and corrects abnormal signal strength values.

7. The ultra-shortwave radio signal monitoring and data analysis system according to claim 1, characterized in that, The data analysis module can be connected to external signal monitoring equipment to acquire data in real time for processing and analysis.

8. The UHF radio signal monitoring and data analysis system according to claim 1, characterized in that, The monitoring data warehouse also includes a data backup module for regularly backing up the stored data to prevent data loss or damage.

9. The ultra-shortwave radio signal monitoring and data analysis system according to claim 1, characterized in that, The system also includes a data security module for encrypting stored and transmitted data to ensure data security.

Images