Distributed quick query, distribution and acquisition system for geographic space data and implementation method

Abstract

The invention discloses a distributed quick query, distribution and acquisition system for geographic space data and an implementation method. The distributed quick query, distribution and acquisition system comprises a specification module and three core modules: a geographic space data list directory specification module, a data list directory construction and node synchronization module, a data query extraction list generation module and a distributed data distribution and acquisition module. The modules work cooperatively, and efficient management, query and rapid distribution and extraction of geographic space data are achieved jointly.

Description

Technical Field

[0001] The present invention belongs to the field of computer information technology, and in particular relates to a distributed fast query distribution acquisition system for geographic space data and an implementation method thereof. Background Art

[0002] With the development of science and technology, such as the increase in the number of image acquisition satellites, the significant improvement in acquisition resolution, and the rapid update of real-time data sent back by different types of sensors throughout the country, the amount of geographic information data generated has increased explosively, and the scale of data is becoming increasingly large. These data are often concentrated in different receiving and production institutions and distributed in unit computer rooms in different locations. In reality, various geospatial application systems are deployed in their respective units, and the geospatial data they rely on comes from different data receiving and production units. Traditional data acquisition methods require manual application, approval, copying, and storage to obtain the required data. Today's application systems have very high requirements for data polymorphism and timeliness, and frequently need to update data. The original application process consumes a lot of time, manpower and material resources. With the popularization of dedicated lines and dedicated networks, various units communicate through dedicated lines and dedicated networks, and data transmission through the network has become a new channel for data distribution, reducing the manpower and material resources of the traditional manual application process. However, for the extraction and distribution of geospatial data with huge data volumes (TB / PB level) over long distances, whether it is the traditional manual copying method or the network transmission method, it is often limited by the physical distance, transmission medium, and transmission bandwidth, and cannot achieve the purpose of fast and effective extraction and distribution. This makes it difficult for data users to quickly obtain the required data, affecting the timeliness and update of data in geospatial applications.

[0003] In summary, the prior art lacks a system and implementation scheme that can efficiently integrate multiple data storage nodes (units) and provide fast query extraction and fast distribution and acquisition of geospatial data. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a distributed fast query distribution acquisition system and implementation method for geospatial data, including a specification and three core modules: geospatial data list directory specification, data list directory construction and node synchronization module, data query extraction form generation module, and distributed data distribution acquisition module. These modules work together to achieve efficient management, query, and fast distribution and extraction of geospatial data.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0006] A distributed fast query distribution acquisition system for geospatial data, comprising: geospatial data list directory specification, data list directory construction and node synchronization module, data query extraction form generation module and distributed data distribution acquisition module;

[0007] The geospatial data inventory directory specification defines the basic information, data type, data storage method, node storage information and data inventory directory information of geospatial data; the geospatial data inventory directory storage method is stored in a relational database and supports exporting SQLite database format files for distribution;

[0008] The data list directory construction module and the node synchronization module include a master node and a sub-node; monitor the changes of the locally stored geospatial data in real time, form the data list directory of the node according to the geospatial data list directory specification, synchronize with the master node in the form of network or manual delivery at regular intervals, merge and update by the master node to form a data list directory, and distribute it to the sub-nodes in the form of network or manual delivery at regular intervals;

[0009] The data query algorithm in the data query extraction form generation module supports data type, data size, data name, data space range, data production time, data source, production unit, data version, and data storage location query conditions; through the data list general directory synchronized from the master node, the required data list is obtained to form a data query extraction form; the data query extraction form includes the queried data list, and the data items in the list include basic information of the data and the storage location, network path, and data transmission method of each node where the data is located;

[0010] The distributed data distribution and acquisition module automatically calculates and forms the optimal distribution and acquisition plan based on the file size, network path, network bandwidth, and data distribution method of each data item by analyzing the data query and extraction content. The plan includes the nearest node and distribution process information of each data item.

[0011] The distributed data distribution and acquisition module supports submitting data distribution and acquisition plans to the master node, which will issue application approval to the responsible unit of the node where the data is located. After approval, the master node will automatically issue data synchronization tasks to the nodes where the data is located in the plan. The nodes will quickly distribute data according to the distribution method in the distribution and acquisition plan. Multi-node network transmission of the same data supports HTTP protocol, FTP protocol, and P2P protocol for distribution. After data distribution is completed, the module will perform data verification to ensure data consistency.

[0012] Preferably, the basic information, data type, data storage method, node storage information and data list directory information are specifically as follows:

[0013] Basic information: including data name, data type, data format, data size, data source, production time, production unit, data version, claiming unit, and claiming time;

[0014] Data types: including digital elevation model DEM, digital surface model DSM, digital orthophoto DOM, true orthophoto TDOM, oblique photography 3D model, laser point cloud, target entity, place name point of interest, administrative division, road network data;

[0015] Data storage methods: including database storage, object storage, and file storage;

[0016] Node storage information: including basic information of existing nodes, storage methods, specific data paths, network paths, and data distribution methods;

[0017] Data list directory information: including version number, generation time and historical operation log records.

[0018] Preferably, the changes to the locally stored geographic spatial data include data addition, update, modification, and deletion operations that trigger real-time inventory actions.

[0019] Preferably, the distributed data distribution and acquisition module can be exported into a distribution and acquisition list file to facilitate manual application when the network is not connected.

[0020] A distributed fast query distribution acquisition implementation method for geographic spatial data includes the following steps:

[0021] Step 1: Build geospatial data inventory catalog specifications;

[0022] Step 2: Data list directory construction and node synchronization module implementation;

[0023] Step 2-1: When adding, deleting, or updating data in the system, the module automatically triggers the monitoring mechanism and updates the changed data information, including basic information, data type, data storage method, and node storage information, to the local node data list directory table;

[0024] Step 2-2: Enter the system synchronization setting interface, set the sub-node upload synchronization frequency, which can be selected every hour, every day, or at a fixed interval, and activate the synchronization function; when the set synchronization time is reached, the system automatically sends the local data list directory to the master node; the master node can set the synchronization frequency, which can be selected every hour, every day, or at a fixed interval, and activate the synchronization function; when the set synchronization time is reached, the system automatically sends the local data list directory to the sub-node;

[0025] Step 2-3: The synchronization module on the master node receives the data list directory sent by the child node, and merges and updates it according to the data change status; if it is a new state, a new record is added to the general directory; if it is an update, the change is made according to the data unique identifier matching the record in the general directory; if it is a deletion, the deletion is made according to the data unique identifier matching the record in the general directory; the above operations will form an operation log and be recorded in the log of the general directory;

[0026] Step 2-4: The master node sends the total directory of the synchronization data list to the registered child nodes according to the scheduled synchronization time;

[0027] Step 2-5: The user can: click the data list export button on the interface to export the local data list directory or the data list directory as a SQLite database for manual delivery and distribution;

[0028] Step 3: Data query and extraction form generation module implementation;

[0029] Step 3-1: Click the Data Query button to open the query panel. Through the parameter selection panel of the interface, you can set the data name, data type, data size, data production time range, data source, production unit, data version, and data storage location. Click the Spatial Range button, and the map panel that pops up allows you to manually draw points, lines, and surfaces, or directly select the area to obtain the spatial range parameters, and finally submit the query.

[0030] Step 3-2: The query function reads the local data list directory file, reads all the contents into the memory, uses the high-speed reading characteristics of the memory and the grid index, GeoHash algorithm, Hibert curve and Google S2 spatial index algorithm to retrieve the matching results and display the results as a list;

[0031] Step 3-3: Expand the list item to view the basic information of the data and the storage location of each node where the data is located, network path, data transmission method, etc. The user can read and retain or delete the non-compliant items. Click the Save button to store it as a data query and extraction form record. Click Export to save the record as a data query and extraction form file in Excel format according to the template.

[0032] Step 4: Implementation of distributed data distribution and acquisition module;

[0033] Step 4-1: The user opens the data distribution acquisition panel, selects a data query and extraction record, and clicks the analysis button. The system analysis algorithm analyzes and calculates the file size, network path, network bandwidth, and data distribution method of each data item in the record. First, it sorts by distance, then sorts by network path to determine whether the network is connected to this node, and then obtains the optimal distribution method based on the path bandwidth and the size of the data to be obtained; if there are multiple nodes available and the network is connected, multiple nodes can be designed to be distributed simultaneously based on the bandwidth; if the network link is not accessible, manual delivery is required for transmission, and the distance is determined to give priority to sorting; at the same time, analysis parameters can be set. If offline distribution is selected, the module automatically matches the node that can copy the disk closest by setting the size and number of the distribution hard disk;

[0034] Step 4-2: After the distribution acquisition list is generated, click Submit Distribution to submit it to the master node. After the master node receives the distribution acquisition list submitted by the child node, it parses the node corresponding to each data item, forms a node distribution approval form by node grouping, and sends it to the node. The node operation and maintenance personnel review the distribution approval form and prepare the data:;

[0035] Step 4-3: After the distribution node is approved, the program automatically transmits data to the target node according to the distribution method set in the distribution table. The transmission supports HTTP protocol, FTP protocol, and P2P protocol for distribution. P2P protocol is enabled first to cooperate with other nodes to speed up the transmission;

[0036] Step 4-4: After obtaining a single piece of data, the distribution acquisition module of the application node performs MD5 hash value verification on the data to ensure the integrity of the data;

[0037] Step 4-5: For nodes that cannot be distributed through the network, you need to carry the distribution acquisition list to the data node for manual copying. Click the Distribution Acquisition List Export button on the interface to export the Distribution Acquisition List file in Excel format.

[0038] A computer program enables a computer to execute the above-mentioned distributed fast query distribution acquisition implementation method.

[0039] An electronic device comprises: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned distributed fast query distribution acquisition implementation method.

[0040] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned distributed fast query distribution acquisition implementation method.

[0041] A chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the above-mentioned distributed fast query distribution acquisition implementation method.

[0042] A computer program product includes a computer storage medium storing a computer program, wherein the computer program includes instructions executable by at least one processor, and when the instructions are executed by the at least one processor, the above-mentioned distributed fast query distribution acquisition implementation method is implemented.

[0043] The beneficial effects of the present invention are as follows:

[0044] 1. Efficient data management and query: Through the construction of geospatial data catalog list and node synchronization capabilities, each node can have the latest data catalog, making data management more efficient and more timely. The data query algorithm of the data query extraction form generation module supports multiple query conditions, and can use the data catalog of the node or the main node to conveniently, quickly and accurately locate the data required by the user, greatly improving the efficiency and accuracy of data query.

[0045] 2. Optimized data distribution and acquisition methods: The distributed data distribution and acquisition module provides user-friendly software interaction and flexible distribution and acquisition options, supporting automated application approval, task distribution, P2P (peer-to-peer) transmission, and manual application. In addition, the module also has a data verification function to ensure the integrity and controllability of the data.

[0046] 3. Improve data application value: By providing efficient and flexible data query, extraction and distribution capabilities, the present invention enables geospatial data to be applied to various geographic applications more quickly and accurately, thereby improving the application value and utilization of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a structural block diagram of the system of the present invention;

[0048] Figure 2 A schematic diagram of the catalog specification of the geospatial data list of the present invention;

[0049] Figure 3 A topology diagram is obtained for distributed data distribution according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0051] The present invention provides a construction system and implementation scheme based on a master-slave node mode, including: a geospatial data list directory specification, a data list directory construction and node synchronization module, a data query extraction form generation module, and a distributed data distribution acquisition module.

[0052] The geospatial data inventory directory specification formulates a set of multi-type and multi-form geospatial data inventory directory specifications, which clarifies the basic information, data types, data storage methods and node storage information of geospatial data. The geospatial data inventory directory storage method is stored in a relational database and supports exporting SQLite database format files for distribution.

[0053] The data list directory construction module and node synchronization module are deployed in each node by setting up the master node and sub-node mode. The system monitors the changes of the locally stored geospatial data in real time (data addition, update, modification, and deletion operations trigger real-time inventory actions), forms the data list directory of this node according to the geospatial data list directory specification, and synchronizes it to the master node regularly in the form of network and manual delivery. The master node merges and updates to form a total data list directory, and distributes it to the sub-nodes regularly in the form of network and manual delivery, realizing "data in multiple places, unified list directory", and providing basic conditions for the generation of data query and extraction orders.

[0054] The data query extraction form generation module uses a special algorithm to support query conditions such as data type, data size, data name, data space range, data production time, data source, production unit, data version, data storage location, etc. The required data list is obtained through the data list general directory synchronized from the master node to form a data query extraction form. The data query extraction form includes the queried data list, and the data items in the list contain the basic information of the data and the storage location of each node where the data is located, the network path, the data transmission method, etc.

[0055] The distributed data distribution and acquisition module analyzes the content of the data query and extraction, and automatically calculates the optimal distribution and acquisition plan based on multiple dimensions such as the file size, network path, network bandwidth, and data distribution method of each data item. The plan content includes the nearest node and distribution process information of each data item.

[0056] This module supports submitting data distribution and acquisition plans to the master node, which will issue application approval to the responsible unit of the node where the data is located. After approval, the master node will automatically issue data synchronization tasks to the nodes where the data is located in the plan. The nodes will quickly distribute data according to the distribution method in the distribution acquisition plan. The same data can be distributed through multiple nodes through HTTP protocol, FTP protocol, and P2P (peer-to-peer) protocol. After data distribution is completed, the module will perform data verification to ensure data consistency.

[0057] At the same time, this module can be exported as a distribution acquisition list file, which is convenient for manual claiming when the network is not connected.

[0058] Example:

[0059] like Figures 1 to 3 As shown, the construction system of the present invention mainly includes a specification and three core modules: geospatial data list directory specification, data list directory construction and node synchronization module, data query and extraction form generation module, and distributed data distribution and acquisition module. These modules work together to achieve efficient management, query, and rapid distribution and extraction of geospatial data.

[0060] 1. Specifications for the inventory catalog of geospatial data;

[0061] Specification requirements: This specification clarifies the basic information, data types, data storage methods, node storage information and data list directory information specifications of multi-type and multi-form geographic spatial data.

[0062] 1. Basic information: including but not limited to data name, data type, data format, data size, data source, production time, production unit, data version, claiming unit, claiming time and other information.

[0063] 2. Data types: including but not limited to digital elevation model (DEM), digital surface model (DSM), digital orthophoto (DOM), true orthophoto (TDOM), oblique photography 3D model, laser point cloud, target entity, place name point of interest, administrative division, road network data, etc.

[0064] 3. Data storage methods: including but not limited to database storage, object storage, file storage and other methods.

[0065] 4. Node storage information includes basic information of existing nodes, storage methods, specific data paths, network accesses (links, bandwidth), data distribution methods (network, CD, hard disk), etc.

[0066] 5. Data list directory information: includes version number, generation time and historical operation log records to ensure data traceability.

[0067] 2. Data directory list construction and node synchronization module;

[0068] 1. Functional description:

[0069] The data list directory construction and node synchronization module is divided into two functions: data list directory construction and node synchronization.

[0070] 1) The data list directory construction function monitors the change operations of the geospatial data of this node and updates the changed data information to the data list directory of this node.

[0071] 2) The node synchronization function supports the synchronization of data directory lists to the master node in the form of network or manual delivery at regular intervals. The master node merges and updates the total directory of the data list and distributes it to the child nodes in the form of network or manual delivery at regular intervals.

[0072] 2. Implementation steps:

[0073] 1) When adding, deleting, or updating data in the system, the module automatically triggers the monitoring mechanism and updates the changed data information, including basic information, data type, data storage method, and node storage information, to the local node data list directory table.

[0074] 2) Enter the system synchronization setting interface, set the sub-node upload synchronization frequency, which can be selected every hour, every day, or at a fixed interval, and activate the synchronization function. When the set synchronization time is reached, the system automatically sends the local data list directory to the master node. The master node can set the synchronization frequency, which can be selected every hour, every day, or at a fixed interval, and activate the synchronization function. When the set synchronization time is reached, the system automatically sends the local data list directory to the sub-node.

[0075] 3) The synchronization module on the master node receives the data list directory sent by the child node and merges and updates it according to the data change status. If it is a new state, a new record is added to the general directory; if it is an update, the change is made according to the data unique identifier matching the record in the general directory; if it is a deletion, the deletion is made according to the data unique identifier matching the record in the general directory. The above operations will form an operation log and be recorded in the general directory log.

[0076] 4) The master node sends the synchronization data list directory to the registered child nodes according to the scheduled synchronization time.

[0077] 5) Users can click the data list export button on the interface to export the local data list directory or the data list directory as a SQLite database for manual delivery and distribution.

[0078] 3. Data query and extraction form generation module;

[0079] 1. Function description:

[0080] Data query uses a special algorithm to support query conditions such as data type, data size, data name, data space range, data production time, data source, production unit, data version, data storage location, etc. The required data list is obtained by querying the data list directory to form a data query extraction form. The data query extraction form includes the queried data list, and the data items in the list include the basic information of the data and the storage location of each node where the data is located, network path, data transmission method, etc.

[0081] 2. Implementation steps:

[0082] 1) Click the data query button to open the query panel. Through the parameter selection panel of the interface, you can set the data name, data type, data size, data production time range, data source, production unit, data version, and data storage location. Click the spatial range button, and the map panel that pops up can be used to manually draw points, lines, and surfaces, or directly select the area to obtain the spatial range parameters, and finally submit the query.

[0083] 2) The query function reads the local data list directory file and reads all the contents into the memory. By using the high-speed reading characteristics of the memory and the grid index, GeoHash algorithm, Hibert curve and Google S2 spatial index algorithm, the matching results can be quickly retrieved and displayed as a list.

[0084] 3) The list items can be expanded to view the basic information of the data and the storage location of each node where the data is located, the network path, the data transmission method, etc. Through user interpretation, the non-compliant items can be retained or deleted. Click the Save button to store it as a data query extraction form record. Click Export to save the record as a data query extraction form file in Excel format according to the template.

[0085] 4. Distributed data distribution and acquisition module;

[0086] 1. Functional overview;

[0087] Supports single content analysis for data query and extraction, and automatically calculates the optimal distribution and acquisition plan.

[0088] It supports submitting data distribution and acquisition plans to the master node, which will complete the application approval. After approval, the master node will automatically issue data synchronization tasks for rapid distribution. Data distribution supports distributed network distribution technology for distribution.

[0089] Support data verification to ensure data consistency.

[0090] Supports exporting into distribution list files for traditional manual claiming.

[0091] 2. Implementation steps

[0092] 1) The user opens the data distribution acquisition panel, selects a data query and extraction record, and clicks the analysis button. The system analysis algorithm analyzes and calculates the file size, network path, network bandwidth, and data distribution method of each data item in the record. First, it is sorted by distance, and then the network path is used to determine whether the network is connected to this node for secondary sorting. Then, the optimal distribution method is obtained based on the path bandwidth and the size of the data to be obtained. If multiple nodes are available and the network is connected, multiple nodes can be designed to be distributed simultaneously based on the bandwidth. If the network link is blocked, manual delivery is required for transmission, and the distance is determined to give priority. At the same time, analysis parameters can be set. If offline distribution is selected, the module automatically matches the node that can copy the disk closest to the node by setting the size and number of the distribution hard disk.

[0093] 2) After the distribution acquisition list is generated, click Submit Distribution to submit it to the master node. After the master node receives the distribution acquisition list submitted by the child node, it parses the node corresponding to each data item, forms a node distribution approval form by node grouping, and sends it to the node. The node operation and maintenance personnel review the distribution approval form and prepare the data.

[0094] 3) After the distribution node is approved, the program automatically transmits data to the target node according to the distribution method set in the distribution table. The transmission supports HTTP protocol, FTP protocol, and P2P (peer-to-peer) protocol for distribution. P2P protocol is enabled first to coordinate with other nodes to speed up transmission.

[0095] 4) After obtaining a single piece of data, the distribution acquisition module of the application node performs MD5 hash value verification on the data to ensure the integrity of the data.

[0096] 5) For nodes that cannot be distributed through the network, you need to carry the distribution acquisition list to the data node for manual copying. Click the Distribution Acquisition List Export button on the interface to export the Distribution Acquisition List file in Excel format.

Claims

1. A distributed fast query, distribution and acquisition system for geographic spatial data, characterized in that: include: Geospatial data inventory directory specification, data inventory directory construction and node synchronization module, data query extraction form generation module and distributed data distribution acquisition module; The geospatial data inventory directory specification defines the basic information, data type, data storage method, node storage information and data inventory directory information of geospatial data; The geospatial data list directory storage method is stored in a relational database, and supports exporting SQLite database format files for distribution; The data list directory construction module and the node synchronization module include a master node and a sub-node; Monitor the changes of locally stored geospatial data in real time, form the data list directory of this node according to the geospatial data list directory specification, synchronize it to the master node regularly through the network or manual delivery, and the master node merges and updates it to form a total data list directory, which is regularly distributed to child nodes through the network or manual delivery; The data query algorithm used in the data query extraction form generation module supports query conditions of data type, data size, data name, data space range, data production time, data source, production unit, data version, and data storage location; By synchronizing the data list from the master node, the required data list is obtained to form a data query extraction form; the data query extraction form includes the queried data list, and the data items in the list include the basic information of the data and the storage location, network path, and data transmission method of each node where the data is located; The distributed data distribution and acquisition module automatically calculates and forms the optimal distribution and acquisition plan based on the file size, network path, network bandwidth, and data distribution method of each data item by analyzing the data query and extraction content. The plan includes the nearest node and distribution process information of each data item. The distributed data distribution and acquisition module supports submitting data distribution and acquisition plans to the master node, which will issue application approval to the responsible unit of the node where the data is located. After approval, the master node will automatically issue data synchronization tasks to the nodes where the data is located in the plan. The nodes will quickly distribute data according to the distribution method in the distribution and acquisition plan. Multi-node network transmission of the same data supports HTTP protocol, FTP protocol, and P2P protocol for distribution. After data distribution is completed, the module will perform data verification to ensure data consistency.

2. A distributed fast query, distribution and acquisition system for geographic spatial data according to claim 1, characterized in that: The basic information, data type, data storage method, node storage information and data list directory information are as follows: Basic information: including data name, data type, data format, data size, data source, production time, production unit, data version, claiming unit, and claiming time; Data types: including digital elevation model DEM, digital surface model DSM, digital orthophoto DOM, true orthophoto TDOM, oblique photography 3D model, laser point cloud, target entity, place name point of interest, administrative division, road network data; Data storage methods: including database storage, object storage, and file storage; Node storage information: including basic information of existing nodes, storage methods, specific data paths, network paths, and data distribution methods; Data list directory information: including version number, generation time and historical operation log records.

3. A distributed fast query, distribution and acquisition system for geographic spatial data according to claim 1, characterized in that: Changes to the locally stored geographic spatial data include data addition, update, modification, and deletion operations that trigger real-time inventory actions.

4. A distributed fast query, distribution and acquisition system for geographic spatial data according to claim 1, characterized in that: The distributed data distribution and acquisition module can be exported into a distribution and acquisition list file, which is convenient for manual application when the network is not connected.

5. A method for implementing the distributed fast query distribution acquisition system of geographic spatial data as claimed in claim 1, characterized in that: The steps include: Step 1: Build geospatial data inventory catalog specifications; Step 2: Data list directory construction and node synchronization module implementation; Step 2-1: When adding, deleting, or updating data in the system, the module automatically triggers the monitoring mechanism and updates the changed data information, including basic information, data type, data storage method, and node storage information, to the local node data list directory table; Step 2-2: Enter the system synchronization setting interface, set the sub-node upload synchronization frequency, which can be selected every hour, every day, or at a fixed interval, and activate the synchronization function; when the set synchronization time is reached, the system automatically sends the local data list directory to the master node; the master node can set the synchronization frequency, which can be selected every hour, every day, or at a fixed interval, and activate the synchronization function; when the set synchronization time is reached, the system automatically sends the local data list directory to the sub-node; Step 2-3: The synchronization module on the master node receives the data list directory sent by the child node and merges and updates it according to the data change status; If it is a new status, add a new record to the general directory; If it is an update, the record in the general catalog is changed according to the data unique identifier; If it is a deletion, the data is deleted by matching the record in the general directory according to the data unique identifier; the above operations will form an operation log and be recorded in the log of the general directory; Step 2-4: The master node sends the total directory of the synchronization data list to the registered child nodes according to the scheduled synchronization time; Step 2-5: The user can: click the data list export button on the interface to export the local data list directory or the data list directory as a SQLite database for manual delivery and distribution; Step 3: Data query and extraction form generation module implementation; Step 3-1: Click the data query button to open the query panel. Through the parameter selection panel of the interface, you can set the data name, data type, data size, data production time range, data source, production unit, data version, and data storage location. Click the spatial range button, and the map panel that pops up can be used to manually draw points, lines, and surfaces, or directly select the area to obtain the spatial range parameters, and finally submit the query; Step 3-2: The query function reads the local data list directory file, reads all the contents into the memory, uses the high-speed reading characteristics of the memory and the grid index, GeoHash algorithm, Hibert curve and Google S2 spatial index algorithm to retrieve the matching results and display the results as a list; Step 3-3: Expand the list item to view the basic information of the data and the storage location of each node where the data is located, network path, data transmission method, etc. The user can read and retain or delete the non-compliant items. Click the Save button to store it as a data query and extraction form record. Click Export to save the record as a data query and extraction form file in Excel format according to the template. Step 4: Implementation of distributed data distribution and acquisition module; Step 4-1: The user opens the data distribution acquisition panel, selects a data query and extraction record, and clicks the analysis button. The system analysis algorithm analyzes and calculates the file size, network path, network bandwidth, and data distribution method of each data item in the record. First, it sorts by distance, then sorts by network path to determine whether the network is connected to this node, and then obtains the optimal distribution method based on the path bandwidth and the size of the data to be obtained; if there are multiple nodes available and the network is connected, multiple nodes can be designed to be distributed simultaneously based on the bandwidth; if the network link is not accessible, manual delivery is required for transmission, and the distance is determined to give priority to sorting; at the same time, analysis parameters can be set. If offline distribution is selected, the module automatically matches the node that can copy the disk closest by setting the size and number of the distribution hard disk; Step 4-2: After the distribution acquisition list is generated, click Submit Distribution to submit it to the master node. After the master node receives the distribution acquisition list submitted by the child node, it parses the node corresponding to each data item, forms a node distribution approval form by node grouping, and sends it to the node. The node operation and maintenance personnel review the distribution approval form and prepare the data:; Step 4-3: After the distribution node is approved, the program automatically transmits data to the target node according to the distribution method set in the distribution table. The transmission supports HTTP protocol, FTP protocol, and P2P protocol for distribution. P2P protocol is enabled first to cooperate with other nodes to speed up the transmission; Step 4-4: After obtaining a single piece of data, the distribution acquisition module of the application node performs MD5 hash value verification on the data to ensure the integrity of the data; Step 4-5: For nodes that cannot be distributed through the network, you need to carry the distribution acquisition list to the data node for manual copying. Click the Distribution Acquisition List Export button on the interface to export the Distribution Acquisition List file in Excel format.

6. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as claimed in any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A chip, characterized in that: include: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product comprises a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and when the instructions are executed by the at least one processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer program, characterized in that The computer program enables a computer to execute the method according to any one of claims 1 to 5.

Images