Open source radio wave environment data acquisition method based on web crawler and computer readable storage medium

Abstract

The invention discloses an open source radio wave environment data acquisition method based on a web crawler and a computer readable storage medium. The method comprises the following steps: step 1, investigation and survey of open source data and webpage feature analysis; step 2, web crawler design; and step 3, automatic acquisition of system architecture and integrated application. According to the method disclosed by the invention, various radio wave environment associated data sources can be investigated and screened, information such as data types, acquisition means, acquisition time and updating frequency of the data sources can be deeply analyzed, and the data sources with high association degree with the radio wave environment and high reliability are incorporated into objects of radio wave environment web crawler data acquisition.

Description

technical field [0001] The invention belongs to the technical field of military information calculation and processing, and in particular relates to a network crawler-based open-source radio wave environment data collection method and a computer-readable storage medium in the field. Background technique [0002] With the increasing sharing of global data resources, the Internet has become an important source of information. The NOAA website of the United States, the Ionospheric Prediction Service (IPS) website of Australia, and the public service websites of domestic meteorological departments (cma, t7online) all have massive multi-source heterogeneous radio wave environment support data available for acquisition, but customized data collection for different websites The labor cost of the program is very high, and it is urgent to design and implement an open source radio wave environment data collection method based on web crawlers. By using web crawler tools, multiple radio...

AI Technical Summary

Problems solved by technology

The NOAA website of the United States, the Ionospheric Prediction Service (IPS) website of Australia, and the public service websites of domestic meteorological departments (cma, t7online) all have massive multi-source heterogeneous radio wave environment support data available for acquisition, but customized data collection for different websites The labor cost of the program is very high, and it is urgent to design and implement an open source radio wave environment data collection method based on web crawlers. By using web crawler tools, multiple radio wave environment data sources are integrated, and the website structure and web page characteristics of each data source are analyzed. Information, customize and configure corresponding data collection rules, realize the automatic collection of massive radio wave environment open source data, provide a convenient and low-cost means for the acquisition of open source radio wave environment data resources, and meet the later development of radio wave environment event detection and radio wave environment situation analysis data support needs, laying the foundation for the improvement of radio wave environmental information support capabilities in the era of big data

Images