Distributed fast query distribution and acquisition system for geospatial data and implementation method

Abstract

The application discloses a distributed fast query distribution and acquisition system and implementation method of geospatial data, comprising one specification and three core modules: a geospatial data list directory specification, a data list directory construction and node synchronization module, a data query extraction single generation module and a distributed data distribution and acquisition module. The modules work cooperatively to realize efficient management, query and fast distribution and extraction of geospatial data.

Description

Technical Field

[0001] This invention belongs to the field of computer information technology, specifically relating to a distributed and fast query, distribution and acquisition system for geospatial data and its implementation method. Background Technology

[0002] With the development of science and technology, such as the increase in the number of image acquisition satellites and the significant improvement in acquisition resolution, the rapid updates of real-time data transmitted by various types of sensors across different locations have led to an explosive increase in the amount of geographic information data generated. This data is increasingly massive, often concentrated in different receiving and production institutions and distributed across computer rooms in various locations. In reality, various geospatial application systems are deployed within their respective organizations, relying on geospatial data from different data receiving and production units. Traditional data acquisition methods require manual application, approval, copying, and warehousing processes to obtain the required data. Today, application systems have high requirements for data polymorphism and timeliness, frequently requiring data updates; going through the original application process consumes a significant amount of time, manpower, and resources. With the widespread adoption of dedicated lines and networks, communication between organizations via dedicated lines and networks has become a new channel for data distribution, reducing the manpower and resources required by traditional manual application processes. However, for the extraction and distribution of geospatial data involving long distances and massive amounts of data (TB / PB level), both traditional manual copying methods and network transmission methods are often limited by physical distance, transmission media, and transmission bandwidth, making it impossible to achieve the goal of fast and effective extraction and distribution. This makes it difficult for data users to quickly obtain the data they need, affecting the timeliness of data updates in geospatial applications.

[0003] In summary, the existing technology lacks a system and implementation plan that can efficiently integrate multiple data storage nodes (units) and provide rapid query, extraction, and distribution of geospatial data. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention provides a distributed, rapid query, distribution, and retrieval system and implementation method for geospatial data, comprising a specification and three core modules: a geospatial data list directory specification, a data list directory construction and node synchronization module, a data query and extraction form generation module, and a distributed data distribution and retrieval module. These modules work together to achieve efficient management, querying, and rapid distribution and retrieval of geospatial data.

[0005] The technical solution adopted by this invention to solve its technical problem is as follows:

[0006] A distributed and fast query, distribution and acquisition system for geospatial data includes: a geospatial data list catalog specification, a data list catalog construction and node synchronization module, a data query and extraction form generation module and a distributed data distribution and acquisition module;

[0007] The geospatial data inventory directory specification clarifies the basic information, data types, data storage methods, node storage information, and data inventory directory information of geospatial data; the geospatial data inventory directory storage method uses a relational database and supports exporting SQLite database format files for distribution;

[0008] The data list directory construction module and node synchronization module include a master node and child nodes; it monitors changes in locally stored geospatial data in real time, forms a data list directory for this node according to the geospatial data list directory specification, and synchronizes it to the master node periodically via network and manual delivery. The master node then merges and updates the data list to form a master data list directory, which is then periodically distributed to child nodes via network and manual delivery.

[0009] The data query and extraction form generation module uses an algorithm to support query conditions such as data type, data size, data name, data space range, data production time, data source, production unit, data version, and data storage location. It obtains the required data list from the master directory synchronized from the master node, forming a data query and extraction form. This form includes the queried data list, where each data item contains basic information about the data, its storage location on each node, network path, and data transmission method.

[0010] The distributed data distribution and acquisition module analyzes the single content extracted from the data query and automatically calculates the optimal distribution and acquisition scheme based on multiple dimensions such as the file size, network path, network bandwidth, and data distribution method of each data item. The scheme includes the nearest node and distribution process information for each data item.

[0011] The distributed data distribution and acquisition module supports submitting data distribution and acquisition plans to the master node. The master node then issues an application for approval to the responsible unit of the data node. After approval, the master node automatically issues data synchronization tasks to each data node in the plan. The nodes quickly distribute the data according to the distribution method in the distribution and acquisition plan. The same data can be distributed across multiple nodes via HTTP, FTP, and P2P protocols. After the data distribution is completed, this module performs data verification to ensure data consistency.

[0012] Preferably, the basic information, data type, data storage method, node storage information, and data list directory information are as follows:

[0013] Basic information includes: data name, data type, data format, data size, data source, production time, production unit, data version, requesting unit, and request time;

[0014] Data types include: Digital Elevation Model (DEM), Digital Surface Model (DSM), Digital Orthophoto (DOM), True Orthophoto (TDOM), Oblique Photogrammetry 3D Model, Laser Point Cloud, Target Entities, Points of Interest (POIs), Administrative Divisions, and Road Network Data;

[0015] Data storage methods include database storage, object storage, and file storage.

[0016] Node storage information: including basic information about the existing nodes, storage method, specific data path, network path, and data distribution method;

[0017] Data list directory information: including version number, generation time, and historical operation log records.

[0018] Preferably, changes to the locally stored geospatial data include data addition, update, modification, and deletion operations that trigger real-time inventory actions.

[0019] Preferably, the distributed data distribution and acquisition module can be exported as a distribution and acquisition list file, which facilitates manual application when the network is not connected.

[0020] A method for distributed, fast querying, distribution, and retrieval of geospatial data includes the following steps:

[0021] Step 1: Construct a geospatial data inventory catalog specification;

[0022] Step 2: Implementation of the data list directory construction and node synchronization module;

[0023] Step 2-1: When data is added, deleted, or updated in the system, the module automatically triggers the monitoring mechanism to update the information of the changed data, including basic information, data type, data storage method, and node storage information, to the local node data list directory table.

[0024] Step 2-2: Enter the system synchronization settings interface, set the upload synchronization frequency of the child nodes, which can be selected as hourly, daily, or fixed interval time, and activate the synchronization function; when the set synchronization time is reached, the system will automatically send the local data list directory to the master node; the master node can set the distribution synchronization frequency, which can be selected as hourly, daily, or fixed interval time, and activate the synchronization function; when the set synchronization time is reached, the system will automatically send the local data list master directory to the child nodes.

[0025] Steps 2-3: The synchronization module on the master node receives the data list directory sent by the child node and merges and updates it according to the data change status. If it is a new record, the new record is added to the main directory. If it is an update, the record in the main directory is matched and modified according to the unique data identifier. If it is a deletion, the record in the main directory is matched and deleted according to the unique data identifier. All of the above operations will generate operation logs and be recorded in the log of the main directory.

[0026] Steps 2-4: The master node distributes the master directory of the synchronized data list to the registered child nodes according to the scheduled synchronization time.

[0027] Steps 2-5: Users can click the "Export Data List" button on the interface to export the local data list directory or the master data list directory as an SQLite database for manual delivery and distribution.

[0028] Step 3: Implementation of the data query and extraction form generation module;

[0029] Step 3-1: Click the data query button to open the query panel. Through the parameter selection panel on the interface, you can optionally set the data name, data type, data size, data production time range, data source, production unit, data version, and data storage location. Click the spatial range button to open the map panel, where you can manually draw points, lines, and polygons, or directly select the area to obtain the spatial range parameters. Finally, submit the query.

[0030] Step 3-2: The query function reads the local data list directory file, loads all the content into memory, and uses the high-speed reading characteristics of memory, grid index, GeoHash algorithm, Hibert curve and Google S2 spatial index algorithm to retrieve matching results and display the results as a list.

[0031] Step 3-3: Expand the list items to view the basic information of the data, including the storage location, network path, and data transmission method of each node. Based on user judgment, retain or delete items that do not meet the requirements. Click the save button to save it as a data query and extraction record. Click export to save the record as an Excel data query and extraction file according to the template.

[0032] Step 4: Implementation of the distributed data distribution and acquisition module;

[0033] Step 4-1: The user opens the data distribution and acquisition panel, selects a data query and extraction record, and clicks the analysis button. The system's analysis algorithm analyzes and calculates the file size, network path, network bandwidth, and data distribution method of each data item in the record. First, it sorts the data by distance, then sorts it again by network path to determine if the network is connected to the current node. Finally, it determines the optimal distribution method based on the bandwidth and the size of the data to be acquired. If multiple nodes are available and the network is connected, multiple nodes can be designed to distribute packets simultaneously based on bandwidth. If the network link is not available, manual delivery is required, prioritizing based on distance. Analysis parameters can also be set. If offline distribution is selected, the module automatically matches the nearest node that can copy the data to the disk by setting the size and number of hard drives.

[0034] Step 4-2: After the distribution acquisition list is generated, click "Distribution Submission" to submit it to the master node. After receiving the distribution acquisition list submitted by the child node, the master node parses the node corresponding to each data item, groups them by node to form a node distribution approval table, and distributes it to the nodes. The node operation and maintenance personnel then review the distribution approval table and prepare the data.

[0035] Step 4-3: After the distribution node is approved, the program will automatically transmit data to the target node according to the distribution method set in the distribution table. The transmission supports HTTP protocol, FTP protocol and P2P protocol for distribution. P2P protocol will be enabled first to coordinate with other nodes to speed up the transmission.

[0036] Step 4-4: After obtaining a single piece of data, the distribution module of the application node performs MD5 hash value verification on the data to ensure data integrity.

[0037] Steps 4-5: For nodes that cannot be distributed via the network, the distribution acquisition list needs to be manually copied to the data node. Clicking the "Export Distribution Acquisition List" button on the interface will export the distribution acquisition list file in Excel format.

[0038] A computer program that causes a computer to execute the above-described distributed fast query and distribution retrieval method.

[0039] An electronic device includes a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the above-described distributed fast query and distribution acquisition method.

[0040] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described distributed fast query and distribution retrieval method.

[0041] A chip includes a processor for retrieving and running a computer program from memory, causing a device equipped with the chip to execute the aforementioned distributed fast query and distribution acquisition method.

[0042] A computer program product includes a computer storage medium storing a computer program, the computer program including instructions executable by at least one processor, which, when executed by the at least one processor, implement the above-described distributed fast query and distribution acquisition method.

[0043] The beneficial effects of this invention are as follows:

[0044] 1. Efficient Data Management and Query: Through the construction of geospatial data catalogs and node synchronization capabilities, each node can have the latest master data catalog, making data management more efficient and timely. The data query extraction module's data query algorithm supports multiple query conditions and can utilize the master data catalog of the local node or the master node to quickly and accurately locate the data needed by the user, greatly improving the efficiency and accuracy of data queries.

[0045] 2. Optimized Data Distribution and Acquisition Methods: The distributed data distribution and acquisition module provides user-friendly software interaction and flexible distribution and acquisition options, supporting automated application and approval, task assignment, P2P (peer-to-peer) transmission, and manual application. Furthermore, this module also features data verification capabilities, ensuring data integrity and controllability.

[0046] 3. Enhancing the value of data applications: By providing efficient and flexible data query, extraction, and distribution capabilities, this invention enables geospatial data to be applied more quickly and accurately to various geographic applications, thereby enhancing the application value and utilization rate of the data. Attached Figure Description

[0047] Figure 1 This is a structural block diagram of the system of the present invention;

[0048] Figure 2 This is a schematic diagram of the geospatial data list catalog specification of the present invention;

[0049] Figure 3 This invention provides a topology diagram for distributed data distribution in an embodiment of the invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] This invention provides a construction system and implementation scheme based on a master-slave node mode, including: a geospatial data list catalog specification, a data list catalog construction and node synchronization module, a data query and extraction form generation module, and a distributed data distribution and acquisition module.

[0052] The Geospatial Data Inventory Directory Specification establishes a set of specifications for multi-type and multi-format geospatial data inventories. This specification clarifies the basic information, data types, data storage methods, and node storage information of geospatial data. The geospatial data inventory directory uses a relational database for storage and supports exporting SQLite database format files for distribution.

[0053] The data inventory directory construction module and node synchronization module are implemented using a master node and child node model, with each node deploying this system. The system monitors changes to locally stored geospatial data in real time (data addition, update, modification, and deletion operations trigger real-time inventory actions), forms the node's data inventory directory according to geospatial data inventory directory specifications, and periodically synchronizes it to the master node via network and manual delivery. The master node then merges and updates the data to form a master data inventory directory, which is periodically distributed to child nodes via network and manual delivery, achieving "data in multiple locations, unified inventory directory," providing the foundation for data query and extraction form generation.

[0054] The data query and extraction form generation module uses a specialized algorithm to support query conditions such as data type, data size, data name, data spatial range, data production time, data source, production unit, data version, and data storage location. It retrieves the required data list from the master data list synchronized from the master node, forming a data query and extraction form. This form includes the retrieved data list, with each data item containing basic data information, its storage location on each node, network path, and data transmission method.

[0055] The distributed data distribution and acquisition module analyzes the extracted data and automatically calculates the optimal distribution and acquisition scheme based on multiple dimensions such as file size, network path, network bandwidth, and data distribution method for each data item. The scheme includes the nearest node and distribution process information for each data item.

[0056] This module supports submitting data distribution and acquisition plans to the master node. The master node then issues an application for approval to the responsible unit of the data node. After approval, the master node automatically issues data synchronization tasks to each data node in the plan. The nodes quickly distribute the data according to the distribution method in the plan. Multi-node network transmission of the same data supports HTTP, FTP, and P2P (peer-to-peer) protocols. After data distribution is completed, this module performs data verification to ensure data consistency.

[0057] This module can also be exported as a distribution acquisition list file, which facilitates manual application when the network is not connected.

[0058] Example:

[0059] like Figures 1 to 3 As shown, the system constructed in this invention mainly includes a specification and three core modules: a geospatial data inventory directory specification, a data inventory directory construction and node synchronization module, a data query and extraction form generation module, and a distributed data distribution and acquisition module. These modules work together to achieve efficient management, querying, and rapid distribution and extraction of geospatial data.

[0060] I. Geospatial Data Inventory Catalog Standards;

[0061] The specification requires that the basic information, data types, data storage methods, node storage information, and data list catalog information of various types and forms of geospatial data be clearly defined.

[0062] 1. Basic Information: This includes, but is not limited to, data name, data type, data format, data size, data source, production time, production unit, data version, requesting unit, and request time.

[0063] 2. Data types: including but not limited to digital elevation models (DEM), digital surface models (DSM), digital orthophotos (DOM), true orthophotos (TDOM), oblique photogrammetry 3D models, laser point clouds, target entities, points of interest, administrative divisions, road network data, etc.

[0064] 3. Data storage methods: including but not limited to database storage, object storage, file storage, and other methods.

[0065] 4. Node storage information includes basic information about the existing nodes, storage method, specific data path, network path (link, bandwidth), data distribution method (network, optical disc, hard disk), etc.

[0066] 5. Data list directory information: includes version number, generation time, and historical operation log records to ensure data traceability.

[0067] II. Data catalog list construction and node synchronization module;

[0068] 1. Function Description:

[0069] The data list directory construction and node synchronization module is divided into two functions: data list directory construction and node synchronization.

[0070] 1) The data list directory construction function monitors changes to the geospatial data of this node and updates the changed data information to the data list directory of this node.

[0071] 2) The node synchronization function supports the scheduled delivery of the data catalog list to the master node via network or manual delivery. The master node then merges and updates the overall data catalog list and distributes it to the child nodes via network or manual delivery on a scheduled basis.

[0072] 2. Implementation steps:

[0073] 1) When data is added, deleted, or updated in the system, the module automatically triggers the monitoring mechanism to update the information of the changed data, including basic information, data type, data storage method, and node storage information, to the local node data list directory table.

[0074] 2) Enter the system synchronization settings interface, set the upload synchronization frequency of the child nodes (options: hourly, daily, or fixed interval), and activate the synchronization function. When the set synchronization time is reached, the system automatically sends the local data list directory to the master node. The master node can set the distribution synchronization frequency (options: hourly, daily, or fixed interval) and activate the synchronization function. When the set synchronization time is reached, the system automatically sends the local data list master directory to the child nodes.

[0075] 3) The synchronization module on the master node receives the data list directory sent by the child nodes and merges and updates it according to the data change status. If it is a new entry, the new record is added to the main directory; if it is an update, the record in the main directory is matched and modified according to the data's unique identifier; if it is a deletion, the record in the main directory is matched and deleted according to the data's unique identifier. All of the above operations will generate operation logs and be recorded in the logs of the main directory.

[0076] 4) The master node distributes the master directory of the synchronized data list to the registered child nodes according to the scheduled synchronization time.

[0077] 5) Users can click the data list export button on the interface to export the local data list directory or the data list master directory to an SQLite database for manual delivery and distribution.

[0078] III. Data Query and Extraction Form Generation Module;

[0079] 1. Function Description:

[0080] The data query employs a specialized algorithm to support query conditions such as data type, data size, data name, data spatial range, data production time, data source, production unit, data version, and data storage location. The required data list is obtained by querying the master data list, forming a data query extraction form. This form includes the retrieved data list, with each data item containing basic data information, including the storage location, network path, and data transmission method of each node.

[0081] 2. Implementation steps:

[0082] 1) Click the data query button to open the query panel. Through the parameter selection panel on the interface, you can optionally set the data name, data type, data size, data production time range, data source, production unit, data version, and data storage location. Click the spatial range button to open the map panel, where you can manually draw points, lines, and polygons, or directly select the area to obtain the spatial range parameters. Finally, submit the query.

[0083] 2) The query function reads the local data list directory file and loads all the content into memory. Utilizing the high-speed reading characteristics of memory and the grid index, GeoHash algorithm, Hibert curve and Google S2 spatial index algorithm, it can quickly retrieve matching results and display the results as a list.

[0084] 3) List items can be expanded to view basic data information, including the storage location, network path, and data transmission method of each node. Users can judge and retain or delete items that do not meet the requirements. Clicking the save button will save the data as a data query and extraction record. Clicking export will save the record as an Excel data query and extraction file according to the template.

[0085] IV. Distributed data distribution and acquisition module;

[0086] 1. Functional Overview;

[0087] It supports data query and extraction analysis of single content, and automatically calculates the optimal distribution and acquisition plan.

[0088] It supports submitting data distribution acquisition plans to the master node, which completes the application and approval process. Once approved, the master node automatically issues data synchronization tasks for rapid distribution. Data distribution supports distributed network distribution technology.

[0089] Supports data validation to ensure data consistency.

[0090] It supports exporting to a distribution acquisition list file for use in traditional manual application.

[0091] 2. Implementation Steps

[0092] 1) Users open the data distribution and acquisition panel, select a data query and extraction record, and click the analysis button. The system's analysis algorithm analyzes and calculates the file size, network path, network bandwidth, and data distribution method for each data item in the record. First, it sorts the data by distance, then by network path to determine network connectivity to the current node for a second sorting. Finally, based on the bandwidth and the size of the data to be acquired, it determines the optimal distribution method. If multiple nodes are available and the network is connected, multiple nodes can be designed to distribute packets simultaneously based on bandwidth. If the network link is unavailable, manual delivery is required, prioritizing based on distance. Analysis parameters can also be set. If offline distribution is selected, the module automatically matches the nearest node that can copy the data to the disk by setting the size and number of hard drives.

[0093] 2) After the distribution acquisition list is generated, click "Distribution Submission" to submit it to the master node. After receiving the distribution acquisition list submitted by the child node, the master node parses the node corresponding to each data item, groups them by node to form a node distribution approval table, and distributes it to the nodes. The node operation and maintenance personnel then review the distribution approval table and prepare the data.

[0094] 3) After the distribution node is approved, the program will automatically transmit data to the target node according to the distribution method set in the distribution table. The transmission supports HTTP protocol, FTP protocol and P2P (peer-to-peer) protocol for distribution. P2P protocol will be enabled first to coordinate with other nodes to accelerate the transmission.

[0095] 4) After obtaining a single piece of data, the distribution and acquisition module of the application node performs MD5 hash value verification on the data to ensure data integrity.

[0096] 5) For nodes that cannot be distributed via the network, the distribution acquisition list needs to be manually copied to the data node. Clicking the "Export Distribution Acquisition List" button on the interface will export the distribution acquisition list file in Excel format.

Claims

1. A distributed fast query distribution and acquisition system for geospatial data, characterized in that, The application relates to a geospatial data list directory specification, a data list directory construction and node synchronization module, a data query extraction sheet generation module and a distributed data distribution acquisition module. The geospatial data list directory specification clearly defines basic information, data types, data storage modes, node storage information and data list directory information of geospatial data. The geospatial data list directory storage mode is stored in a relational database and supports export of an SQLite database format file for distribution. The data list directory construction module and the node synchronization module comprise a master node and a slave node. Real-time monitoring of changes in locally stored geospatial data is performed, a node data list directory is formed according to the geospatial data list directory specification, and the node data list directory is synchronized to the master node in a network or manual delivery mode at a fixed time; the master node performs merging and updating to form a total data list directory, and the total data list directory is distributed to the slave nodes in a network or manual delivery mode at a fixed time. In the data query extraction sheet generation module, data query adopts an algorithm to support data type, data size, data name, data space range, data production time, data source, production unit, data version and data storage location query conditions. A required data list is acquired from the total data list directory synchronized from the master node, and a data query extraction sheet is formed; the data query extraction sheet content comprises the queried data list, and the data items in the list comprise basic information of the data and storage locations, network paths and data transmission modes of the data in the nodes. The distributed data distribution acquisition module automatically calculates an optimal distribution acquisition scheme according to multiple dimensions of file size, network path, network bandwidth and data distribution mode of each data item by analyzing the content of the data query extraction sheet; the scheme content comprises the nearest node of each data item and distribution process information. The distributed data distribution acquisition module supports submission of the data distribution acquisition scheme to the master node, and the master node issues a requisition approval to the responsible unit of the data node; after the approval is passed, the master node automatically issues a data synchronization task to each data node in the scheme, and the nodes perform rapid distribution of the data according to the distribution mode in the distribution acquisition scheme; the same data in multiple nodes is distributed by supporting HTTP protocol, FTP protocol and P2P protocol; and the module performs data checking after the data distribution is completed to ensure data consistency. The basic information, data type, data storage mode, node storage information and data list directory information are as follows:

2. The distributed fast query distribution and acquisition system for geospatial data according to claim 1, wherein, Basic information: data name, data type, data format, data size, data source, production time, production unit, data version, requisition unit and requisition time; Data type: digital elevation model DEM, digital surface model DSM, digital orthographic image DOM, true orthographic image TDOM, oblique photogrammetry three-dimensional model, laser point cloud, target entity, place of interest, administrative division and road network data; Data storage mode: database storage, object storage and file storage; Node storage information: existing node basic information, storage mode, specific data path, network path and data distribution mode. ​ Data inventory directory information: including version number, generation time and historical operation log records.

3. The distributed fast query distribution and acquisition system for geospatial data according to claim 1, wherein, The change of the locally stored geospatial data includes data addition, update, modification, and deletion operation triggering real-time inventory action.

4. The distributed fast query distribution and acquisition system for geospatial data according to claim 1, wherein, The distributed data distribution acquisition module can export a distribution acquisition list file, facilitating manual application in the case of network disconnection.

5. The implementation method of the distributed fast query distribution and acquisition system of geospatial data according to claim 1, characterized in that, Comprising the following steps: Step 1: Constructing geospatial data inventory directory specification; Step 2: Data inventory directory construction and node synchronization module implementation; Step 2-1: When data is added, deleted, or updated in the system, the module automatically triggers the monitoring mechanism, and updates the information of the changed data, including basic information, data type, data storage method, and node storage information, to the local node data inventory directory table; Step 2-2: Enter the system synchronization setting interface and set the synchronization frequency of the subnode, which can be selected as every hour, every day, or fixed interval time, and activate the synchronization function; when the set synchronization time is reached, the system automatically sends the local data inventory directory to the master node; the master node can set the synchronization frequency, which can be selected as every hour, every day, or fixed interval time, and activate the synchronization function; when the set synchronization time is reached, the system automatically sends the local data inventory directory to the subnode; Step 2-3: The synchronization module on the master node receives the data inventory directory sent by the subnode, and updates it according to the change status of the data; If it is a new state, add a new record to the total directory; If it is updated, change the record in the total directory according to the unique identification of the data; If it is deleted, delete the record in the total directory according to the unique identification of the data; the above operations will form operation logs and be recorded in the log of the total directory; Step 2-4: The master node sends the synchronization data inventory total directory to the registered subnode according to the set synchronization time; Step 2-5: The user clicks the data inventory export button on the interface to export the local data inventory directory or data inventory total directory as a SQLite database for manual delivery of distribution; Step 3: Data query extraction module implementation; Step 3-1: Click the data query button to open the query panel, and select the data name, data type, data size, data production time range, data source, production unit, data version, and data storage location through the parameter selection panel on the interface, click the spatial range button, and draw the spatial range parameters manually or select the area directly on the pop-up map panel, and finally submit the query; Step 3-2: The query function reads the local data inventory total directory file, reads the content into the memory, uses the high-speed reading characteristics of the memory, and uses grid index, GeoHash algorithm, Hibert curve, and Google S2 spatial index algorithm to retrieve the matching results, and displays the results as a list; Step 3-3: The basic information of the data and the storage location, network path, and data transmission mode of each node where the data is located are displayed when the list item is expanded. The user can judge and retain or delete items that do not meet the requirements. Click the save button to store the data query extraction sheet as a record. Click the export button to save the record as an Excel-formatted data query extraction sheet file according to the template. Step 4: The distributed data distribution acquisition module is implemented. Step 4-1: The user opens the data distribution acquisition panel and selects a data query extraction record. Click the analysis button. The system analyzes the algorithm based on the file size, network path, network bandwidth, and data distribution method of each data item in the record. First, sort by distance, then judge whether the network is connected to the node for secondary sorting, and then obtain the optimal distribution method based on the path bandwidth and the size of the data to be acquired. If multiple nodes are available and the network is connected, multiple nodes can be designed for simultaneous distribution based on bandwidth. If the network link is not connected, the data needs to be transmitted manually. The distance is prioritized. Analysis parameters can be set. If offline distribution is selected, the module automatically matches the nearest available node for disk copying based on the distribution hard disk size and quantity. Step 4-2: After generating the distribution acquisition list, click the distribution submission button to submit it to the master node. After receiving the distribution acquisition list submitted by the sub-node, the master node parses the corresponding node for each data item, groups the nodes, and forms a node distribution approval table. The table is sent to the node for review and data preparation by the node operation and maintenance personnel. Step 4-3: After the distribution node approval is passed, the data is automatically transmitted to the target node according to the distribution method set in the distribution table. Transmission supports HTTP protocol, FTP protocol, and P2P protocol for distribution. P2P protocol is preferred to accelerate transmission with other nodes. Step 4-4: The distribution acquisition module of the application node performs MD5 hash value verification on the data after acquiring a single data to ensure data integrity. Step 4-5: For nodes that cannot be distributed through the network, the distribution acquisition list export button on the interface can export an Excel-formatted distribution acquisition list file.

6. An electronic device, comprising: Comprising: A processor and a memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to enable the electronic device to perform the method of claim 5.

7. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the method of claim 5.

8. A chip, characterized by Comprising: a processor for calling and running a computer program from a memory, so that the device installed with the chip performs the method of claim 5.

9. A computer program product, characterised in that, The computer program product comprises a computer storage medium storing a computer program, and the computer program comprises instructions executable by at least one processor, which implement the method of claim 5 when executed by the at least one processor.

Images