Vector space data transmission processing method and system

Abstract

The invention discloses a vector space data transmission processing method and system, and relates to the technical field of data processing, the method comprises the following steps: a client performs format conversion on vector space data to be uploaded to obtain preprocessed vector space data in a table format; the client carries out roll-by-roll compression on the preprocessed vector space data according to a preset roll-by-roll size to obtain a plurality of pieces of roll-by-roll data; the client performs parallel transmission on the roll-dividing data so as to transmit the roll-dividing data to a server; and the server decompresses the received roll-dividing data, imports the roll-dividing data into a candidate data table of a target database, and performs format conversion on the candidate data table to obtain a target data table in a spatial layer format. The data transmission speed can be improved, so that the transmission efficiency of the vector space data can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a vector space data transmission processing method and system. Background Art

[0002] With the advancement of science and technology, the application of various data types is becoming increasingly widespread. For example, vector spatial data is increasingly being used. Vector spatial data is a data format that describes the location and shape of geographic entities by recording coordinates. Vector data generally uses coordinates to accurately represent the spatial location of geographic entities.

[0003] In actual use, vector space data needs to be transmitted and processed. In existing technologies, vector space data is usually transmitted directly. However, the problem with existing technologies is that the vector space data format is complex, and direct transmission of vector space data is slow, which is not conducive to improving the transmission efficiency of vector space data.

[0004] Therefore, relevant technologies still need to be improved and developed. Summary of the Invention

[0005] The main purpose of this application is to provide a vector space data transmission and processing method and system, aiming to solve the technical problem in the related art that when directly transmitting vector space data, the transmission speed is slow, which is not conducive to improving the transmission efficiency of vector space data.

[0006] In order to achieve the above-mentioned object, the first aspect of the present application provides a vector space data transmission processing method, wherein the vector space data transmission processing method includes:

[0007] The client converts the format of the vector space data to be uploaded to obtain pre-processed vector space data in a table format;

[0008] The client compresses the pre-processed vector space data in sub-volumes according to a preset sub-volume size to obtain a plurality of sub-volume data;

[0009] The client transmits the volume data in parallel to transmit the volume data to the server;

[0010] The server decompresses the received volume data and imports it into the candidate data table of the target database, and performs format conversion on the candidate data table to obtain the target data table in the spatial layer format.

[0011] Optionally, the client performs format conversion on the vector space data to be uploaded to obtain pre-processed vector space data in a table format, including:

[0012] The client converts the format of the vector space data to be uploaded by using a preset data conversion tool to obtain pre-processed vector space data in a table format;

[0013] The table format is a comma-separated value format, and the pre-processed vector spatial data stores coordinate values ​​of spatial vector elements in the vector spatial data to be uploaded and attribute values ​​corresponding to the spatial vector elements.

[0014] Optionally, the client performs volume compression on the pre-processed vector space data according to a preset volume size to obtain a plurality of volume data, including:

[0015] The client obtains a preset volume size and a target compressed file format;

[0016] The client performs volume compression on the pre-processed vector space data according to the preset volume size and the target compressed file format to obtain a plurality of volume data.

[0017] Optionally, the client transmits the sub-volume data in parallel to transmit the sub-volume data to the server, including:

[0018] The client transmits the volume data in parallel through a multi-file parallel transmission tool to transmit the volume data to the server.

[0019] Optionally, the server decompresses the received volume data and imports the decompressed data into a candidate data table of a target database, and converts the format of the candidate data table to obtain a target data table in a spatial layer format, including:

[0020] The server decompresses the received volume data according to the target compressed file format to obtain decompressed data;

[0021] The server imports the decompressed data into a candidate data table of a target database;

[0022] The server performs format conversion on the candidate data table after importing the decompressed data to obtain a target data table in a spatial layer format.

[0023] Optionally, the server performs format conversion on the candidate data table after importing the decompressed data to obtain a target data table in a spatial layer format, including:

[0024] The server splits the candidate data table after importing the decompressed data to obtain multiple data sub-tables;

[0025] The server performs format conversion on the data sub-table to obtain a target data table in a spatial layer format.

[0026] Optionally, the server performs format conversion on the data sub-table to obtain a target data table in a spatial layer format, including:

[0027] The server converts the coordinate string into a spatial feature object format for the data sub-tables, and converts each of the data sub-tables into a spatial layer format to obtain a target data table corresponding to each of the data sub-tables.

[0028] A second aspect of the present application provides a vector space data transmission and processing system, wherein the vector space data transmission and processing system includes:

[0029] The client and the server include a format conversion module, a volume compression module and a data transmission module;

[0030] The format conversion module is used to convert the format of the vector space data to be uploaded to obtain pre-processed vector space data in a table format;

[0031] The volume compression module is used to perform volume compression on the pre-processed vector space data according to a preset volume size to obtain a plurality of volume data;

[0032] The data transmission module is used to transmit the volume data in parallel to transmit the volume data to the server;

[0033] The server is used to decompress the received volume data and import it into the candidate data table of the target database, and perform format conversion on the candidate data table to obtain the target data table in the spatial layer format.

[0034] Optionally, the format conversion module is specifically configured to:

[0035] Convert the vector space data to be uploaded into a format using a preset data conversion tool to obtain pre-processed vector space data in a table format;

[0036] The table format is a comma-separated value format, and the pre-processed vector spatial data stores coordinate values ​​of spatial vector elements in the vector spatial data to be uploaded and attribute values ​​corresponding to the spatial vector elements.

[0037] Optionally, the volume compression module is specifically used to:

[0038] Get the preset volume size and target compressed file format;

[0039] According to the preset sub-volume size and the target compressed file format, the pre-processed vector space data is subjected to sub-volume compression to obtain a plurality of sub-volume data.

[0040] As can be seen from the above, in the present application scheme, the client performs format conversion on the vector spatial data to be uploaded to obtain pre-processed vector spatial data in a tabular format; the client performs volume compression on the pre-processed vector spatial data according to a preset volume size to obtain multiple volume data; the client transmits the volume data in parallel to transmit the volume data to the server; the server decompresses the received volume data and imports it into the candidate data table of the target database, and performs format conversion on the candidate data table to obtain the target data table in the spatial layer format.

[0041] Compared to existing technologies, the vector space data transmission and processing method provided in this application corresponds to a solution in which the client first converts the format of the uploaded vector space data to obtain pre-processed vector space data in a tabular format. The client then performs volume compression to obtain the volume data, which is then transmitted in parallel. The server then decompresses the received volume data and imports it into a candidate data table in the target database. The server then performs format conversion to restore the target data table. This allows for transmission using compressed volume data and enables parallel data transmission, which improves data transmission speed and, consequently, vector space data transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a flow chart of a vector space data transmission processing method provided by an embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of a specific process for uploading a data file to a server provided in an embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of a specific process for importing data into a database table provided by an embodiment of the present application;

[0046] Figure 4 This is a schematic diagram of a specific process for converting database table data into vector space layer data provided by an embodiment of the present application;

[0047] Figure 5 This is a schematic diagram of the components of a vector space data transmission and processing system provided by an embodiment of the present application;

[0048] Figure 6 This is a block diagram of the internal structure principle of a smart terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0050] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0051] It should also be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0052] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0053] As used in this specification and the appended claims, the term "if" can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to being classified into," depending on the context. Similarly, the phrase "if it is determined" or "if it is classified into [described condition or event]" can be interpreted as meaning "upon determination" or "in response to determining" or "upon classification into [described condition or event]" or "in response to being classified into [described condition or event]," depending on the context.

[0054] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0055] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0056] Currently, the applications of vector spatial data are increasing, and the demand for its transmission is also increasing. For example, in some application scenarios, large quantities of vector spatial data need to be quickly transmitted and stored. Typically, vector spatial data is transmitted directly. However, existing technologies suffer from the complex format of vector spatial data, resulting in slow transmission speeds and hindering efficiency.

[0057] Specifically, existing vector spatial data formats, such as shp and filegdb, suffer from complex formats, low file compression ratios, slow network transmission of individual files, and excessively long file transfer times. When importing existing vector files into the SDE spatial database, data can only be imported one by one, not batch-copying, resulting in a lengthy data import process.

[0058] In order to solve at least one of the above-mentioned technical problems, in the solution of the present application, the client performs format conversion on the vector space data to be uploaded to obtain pre-processed vector space data in a tabular format; the client performs volume compression on the pre-processed vector space data according to a preset volume size to obtain multiple volume data; the client transmits the volume data in parallel to transmit the volume data to the server; the server decompresses the received volume data and imports it into the candidate data table of the target database, and performs format conversion on the candidate data table to obtain the target data table in a spatial layer format.

[0059] Compared to existing technologies, the vector space data transmission and processing method provided in this application corresponds to a solution in which the client first converts the format of the uploaded vector space data to obtain pre-processed vector space data in a tabular format. The client then performs volume compression to obtain the volume data, which is then transmitted in parallel. The server then decompresses the received volume data and imports it into a candidate data table in the target database. The server then performs format conversion to restore the target data table. This allows for transmission using compressed volume data and enables parallel data transmission, which improves data transmission speed and, consequently, vector space data transmission efficiency.

[0060] like Figure 1 As shown, the embodiment of the present application provides a vector space data transmission processing method, specifically, the method includes the following steps:

[0061] Step S100: The client converts the format of the vector space data to be uploaded to obtain pre-processed vector space data in a table format.

[0062] Step S200: the client compresses the pre-processed vector space data into sub-volumes according to a preset sub-volume size to obtain a plurality of sub-volume data;

[0063] Step S300: the client transmits the volume data in parallel to transmit the volume data to the server;

[0064] In step S400, the server decompresses the received volume data and imports the decompressed data into a candidate data table of a target database, and converts the format of the candidate data table to obtain a target data table in a spatial layer format.

[0065] The volume size is used to limit the size of each volume data during compression. The volume size can be set and adjusted according to actual needs and is not specifically limited here.

[0066] It should be noted that, when performing parallel transmission, the number of data transmitted in parallel at one time can also be set and adjusted according to actual needs, and is not specifically limited here.

[0067] In this way, the client first converts the format of the uploaded vector space data to obtain pre-processed vector space data in a tabular format. It then compresses the data into separate volumes, which are then transmitted in parallel. The server then decompresses the received data into a candidate data table in the target database, converting it to restore the target data table. This allows for transmission using compressed separate volume data and enables parallel data transmission, which improves data transmission speed and, consequently, vector space data transmission efficiency.

[0068] Specifically, the client converts the format of the vector space data to be uploaded to obtain pre-processed vector space data in a table format, including:

[0069] The client converts the format of the vector space data to be uploaded by using a preset data conversion tool to obtain pre-processed vector space data in a table format;

[0070] The table format is a comma-separated value format, and the pre-processed vector spatial data stores coordinate values ​​of spatial vector elements in the vector spatial data to be uploaded and attribute values ​​corresponding to the spatial vector elements.

[0071] In the embodiments of the present application, the comma-separated values ​​(CSV) format is used. In actual application, other specific formats can also be used, which is not specifically limited here.

[0072] Furthermore, the client performs volume compression on the pre-processed vector space data according to a preset volume size to obtain a plurality of volume data, including:

[0073] The client obtains a preset volume size and a target compressed file format;

[0074] The client performs volume compression on the pre-processed vector space data according to the preset volume size and the target compressed file format to obtain a plurality of volume data.

[0075] The volume size and target compressed file format may also be set and adjusted according to actual needs and are not specifically limited here.

[0076] In the embodiment of the present application, the client transmits the sub-volume data in parallel to transmit the sub-volume data to the server, including:

[0077] The client transmits the volume data in parallel through a multi-file parallel transmission tool to transmit the volume data to the server.

[0078] In this way, the client transmits the compressed volume data in parallel to the cloud server, which can effectively improve the data transmission efficiency.

[0079] Specifically, the server decompresses the received volume data and imports it into the candidate data table of the target database, and converts the format of the candidate data table to obtain the target data table in the spatial layer format, including:

[0080] The server decompresses the received volume data according to the target compressed file format to obtain decompressed data;

[0081] The server imports the decompressed data into a candidate data table of a target database;

[0082] The server performs format conversion on the candidate data table after importing the decompressed data to obtain a target data table in a spatial layer format.

[0083] Specifically, the server performs corresponding decompression processing according to the target compressed file format to restore the vector space data.

[0084] The server converts the format of the candidate data table after importing the decompressed data to obtain a target data table in a spatial layer format, including:

[0085] The server splits the candidate data table after importing the decompressed data to obtain multiple data sub-tables;

[0086] The server performs format conversion on the data sub-table to obtain a target data table in a spatial layer format.

[0087] Furthermore, the server performs format conversion on the data sub-table to obtain a target data table in a spatial layer format, including:

[0088] The server converts the coordinate string into a spatial feature object format for the data sub-tables, and converts each of the data sub-tables into a spatial layer format to obtain a target data table corresponding to each of the data sub-tables.

[0089] In one specific application scenario, a large volume of vector spatial data, exceeding one million in size, is converted to a common tabular CSV format on the client side. This CSV file stores the coordinate values ​​of spatial vector elements and their other attribute values. The CSV data files are compressed into separate volumes of 200-300MB each to minimize file size and support concurrent upload of multiple files. Using a multi-file parallel transfer tool, the compressed data files are uploaded to the database server's file directory.

[0090] The compressed files are then decompressed on the server side to restore the data to CSV format. The CSV file COPY method supported by the Spatial Database Engine (SDE) is used to import the CSV format data into the database table. The database table splitting method is used to split the stored data into multiple sub-tables. Within the sub-tables, the coordinate string is converted to a spatial feature object format to convert the database table into a spatial layer format, completing the storage of the vector spatial data. This solves the traditional problem of uploading a single, large-volume vector spatial data file to a cloud-based spatial database. This problem is caused by the large amount of data in a single, one-time vector format file, resulting in long network transmission and data storage times, prone to interruptions, and storage anomalies. This can increase the data transmission and storage speed by 5-10 times. This facilitates the rapid transmission and storage of large quantities of vector spatial data.

[0091] In one application scenario, the vector space data transmission and processing method is divided into three stages. In the first stage, the data file is quickly uploaded to the cloud server. Figure 2 This is a specific flow chart of uploading a data file to a server provided by an embodiment of the present application, such as Figure 2As shown, the data conversion tool is used on the client to convert the spatial data files to be stored into CSV format; and the files are compressed into ZIP compression format according to the volume size of 200-300M to control the size of a single data file; a multi-file parallel transmission tool is used to upload the volume-compressed data files to the file directory where the database server is located, and the volume-compressed files are decompressed on the server to restore the CSV format of the data. Through file compression and parallel transmission, the data files are quickly uploaded to the database server in the cloud.

[0092] The second stage is to quickly import data files into the database. Figure 3 This is a specific flow chart of importing data into a database table provided by an embodiment of the present application, such as Figure 3 As shown, the CSV file COPY method supported by the database is used to import CSV format data into the data table of the database.

[0093] The third stage is to convert the database table data into a vector space layer. Figure 4 This is a specific flow chart of converting database table data into vector space layer data provided by an embodiment of the present application. Figure 4 As shown, the database table splitting method is used to split the stored data into multiple sub-tables. Within the sub-tables, the coordinate string is converted into a spatial feature object format to convert the database table into a spatial layer format, completing the storage of vector spatial data. The converted vector data from the sub-table is then stored in the layer, forming the stored spatial feature layer format, completing the storage of vector spatial data. Specifically, a multi-threaded parallel upload tool developed with the J2EE spring + VUE architecture is used, along with a data copy method and vector feature conversion tool written in SQL. A spatial database formed by the PostgreSQL database and the postGIS spatial engine is used as the storage medium.

[0094] Based on the method of this application, the problems of traditional methods such as uploading a single large batch of vector spatial data files to the cloud spatial database are solved, such as the large amount of data in a single vector format file at one time, long network transmission and data storage time, easy interruption and storage anomalies, etc., which can increase the data transmission and storage speed by 5-10 times.

[0095] Specifically, by combining and applying multi-threaded parallel data processing technologies such as data format preprocessing, volume compression, multi-threaded upload, database copying, and parallel conversion of spatial elements, we achieve rapid transmission and storage of spatial data elements. The specific process includes format conversion, compression, storage and transmission, decompression, and storage. This enables volume compression and parallel transmission of vector spatial data in CSV format, and rapid storage of copied data based on the CSV format and the (PostgresSQL + postGIS) paradigm.

[0096] In a specific application scenario, the rapid warehousing process is divided into multiple stages. A data conversion tool is used on the client to convert the spatial data files that need to be stored into CSV format; and the files are compressed into ZIP compression format according to the volume size of 200-300M to control the size of a single data file; a multi-file parallel transmission tool is used to upload the volume-compressed data files to the file directory where the database server is located, and the volume-compressed files are decompressed on the server side to restore the data to CSV format. Through file compression and parallel transmission, the data files are quickly uploaded to the database server in the cloud.

[0097] On the client side, large quantities of vector spatial data exceeding 1 million are converted into a common table CSV format. The CSV format file stores the coordinate values ​​of the spatial vector elements and other attribute values ​​of the elements, and the CSV format data files are compressed in volumes with a single volume size of 200-300M to control the data file size as much as possible and support multi-file parallel upload operations; a multi-file parallel transmission tool is used to upload the volume-compressed data files to the file directory where the database server is located, and the volume-compressed files are decompressed on the server side to restore the data to the CSV format; the CSV file COPY method supported by the database is used to import the CSV format data into the database data table; the database table splitting method is used to split the stored data into multiple sub-tables, and the coordinate string is converted to the spatial feature object format method in the sub-table to convert the database table into the spatial layer format to complete the storage operation of the vector spatial data. The present invention solves the problems of traditional methods of uploading single large-scale vector spatial data files to cloud spatial databases, such as the large amount of data in a single vector format file at a time, long network transmission and data storage time, easy interruption and storage anomalies, etc. This method can increase the data transmission and storage speed by 5-10 times.

[0098] To ensure timely data ingestion in large quantities, tools such as QGIS, ArcGIS, and the SDE library can be used to preprocess the data to create CSV files containing attribute and coordinate string data (preprocessing time: 2 minutes). The CSV files are then compressed using ZIP files, converting the file into a group of compressed files, significantly reducing the uploaded file size while controlling the size of each compressed file. The files can be split into 50-100MB files (compression time: 1 minute). Multi-threaded parallel upload technology is used to upload the files to a folder on the database server (upload time: 1 minute). The group of files is then decompressed to restore the CSV file format (decompression time: 1 minute). PostgresSQL's batch copy technology is used to quickly and fully copy the CSV files into an intermediate attribute table (the table structure of the intermediate table must remain consistent with the CSV table structure during import) (time: 0.5 minutes). Once all data has been ingested, the attribute table is converted into a spatial feature layer in the database based on the imported coordinate string fields. To improve data conversion efficiency, create a new set of spatial feature intermediate layers. Create an intermediate table with 50,000 to 100,000 records for processing (2-3 minutes). Once all records have been converted to spatial feature layer records, import all intermediate layer data into the target layer (1-2 minutes).

[0099] By combining and applying multi-threaded parallel data processing technologies such as data format preprocessing, volume compression, multi-threaded uploading, database copying, and parallel conversion of spatial elements, the rapid storage and processing of spatial data elements can be achieved.

[0100] like Figure 5 As shown in , corresponding to the vector space data transmission processing method, an embodiment of the present application further provides a vector space data transmission processing system, the vector space data transmission processing system comprising:

[0101] Client 1 and server 2, wherein client 1 includes a format conversion module 110, a volume compression module 120 and a data transmission module 130;

[0102] The format conversion module 110 is used to convert the format of the vector space data to be uploaded to obtain pre-processed vector space data in a table format;

[0103] The volume compression module 120 is used to perform volume compression on the pre-processed vector space data according to a preset volume size to obtain a plurality of volume data;

[0104] The data transmission module 130 is used to transmit the volume data in parallel to transmit the volume data to the server;

[0105] The server 2 is used to decompress the received volume data and import it into the candidate data table of the target database, and perform format conversion on the candidate data table to obtain the target data table in the spatial layer format.

[0106] The format conversion module 110 is specifically configured to:

[0107] Convert the vector space data to be uploaded into a format using a preset data conversion tool to obtain pre-processed vector space data in a table format;

[0108] The table format is a comma-separated value format, and the pre-processed vector spatial data stores coordinate values ​​of spatial vector elements in the vector spatial data to be uploaded and attribute values ​​corresponding to the spatial vector elements.

[0109] The volume compression module 120 is specifically used for:

[0110] Get the preset volume size and target compressed file format;

[0111] According to the preset sub-volume size and the target compressed file format, the pre-processed vector space data is subjected to sub-volume compression to obtain a plurality of sub-volume data.

[0112] Thus, in the solution corresponding to the vector space data transmission and processing system provided by this application, the client first converts the format of the uploaded vector space data to obtain pre-processed vector space data in a tabular format, then performs volume compression to obtain the volume data, and then transmits the volume data in parallel. After receiving the volume data, the server decompresses it and imports it into the candidate data table of the target database. It then performs format conversion to restore the target data table. In this way, the compressed volume data is used for transmission, and parallel data transmission can be achieved, which is beneficial to improving data transmission speed and thus the transmission efficiency of vector space data.

[0113] It should be noted that the specific structure and implementation of the vector space data transmission and processing system and its various modules or units can refer to the corresponding description in the method embodiment, and will not be repeated here.

[0114] It should be noted that the division method of the modules of the vector space data transmission and processing system is not unique and is not specifically limited here.

[0115] Based on the above embodiment, the present application also provides a smart terminal, whose principle block diagram can be as follows: Figure 6As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected via a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps of any one of the above-mentioned vector space data transmission and processing methods are implemented. The display screen of the intelligent terminal can be a liquid crystal display or an electronic ink display.

[0116] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the smart terminal to which the solution of the present application is applied. The specific smart terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0117] In one embodiment, a smart terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of any one of the vector space data transmission processing methods provided in the embodiments of the present application are implemented.

[0118] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any one of the vector space data transmission processing methods provided in the embodiments of the present application are implemented.

[0119] It should be understood that the serial numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0121] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0122] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0123] In the embodiments provided herein, it should be understood that the disclosed systems / terminal devices and methods can be implemented in other ways. For example, the system / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units described above is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or omitting or not implementing certain features.

[0124] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, electric signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0125] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A vector space data transmission processing method, characterized in that: The method comprises: The client converts the format of the vector space data to be uploaded to obtain pre-processed vector space data in a table format; The client compresses the pre-processed vector space data in sub-volumes according to a preset sub-volume size to obtain a plurality of sub-volume data; The client transmits the volume data in parallel to transmit the volume data to the server; The server decompresses the received volume data and imports it into the candidate data table of the target database, and performs format conversion on the candidate data table to obtain the target data table in the spatial layer format.

2. The vector space data transmission processing method according to claim 1, characterized in that: The client performs format conversion on the vector space data to be uploaded to obtain pre-processed vector space data in a table format, including: The client converts the format of the vector space data to be uploaded by using a preset data conversion tool to obtain pre-processed vector space data in a table format; The table format is a comma-separated value format, and the pre-processed vector spatial data stores coordinate values ​​of spatial vector elements in the vector spatial data to be uploaded and attribute values ​​corresponding to the spatial vector elements.

3. The vector space data transmission processing method according to claim 1, characterized in that: The client compresses the pre-processed vector space data into sub-volumes according to a preset sub-volume size to obtain a plurality of sub-volume data, including: The client obtains a preset volume size and a target compressed file format; The client performs volume compression on the pre-processed vector space data according to the preset volume size and the target compressed file format to obtain a plurality of volume data.

4. The vector space data transmission processing method according to claim 1, characterized in that: The client transmits the volume data in parallel to transmit the volume data to the server, including: The client transmits the volume data in parallel through a multi-file parallel transmission tool to transmit the volume data to the server.

5. The vector space data transmission processing method according to claim 3, characterized in that: The server decompresses the received volume data and imports it into the candidate data table of the target database, and converts the format of the candidate data table to obtain the target data table in the spatial layer format, including: The server decompresses the received volume data according to the target compressed file format to obtain decompressed data; The server imports the decompressed data into a candidate data table of a target database; The server performs format conversion on the candidate data table after importing the decompressed data to obtain a target data table in a spatial layer format.

6. The vector space data transmission processing method according to claim 5, characterized in that: The server performs format conversion on the candidate data table after importing the decompressed data to obtain a target data table in a spatial layer format, including: The server splits the candidate data table after importing the decompressed data to obtain multiple data sub-tables; The server performs format conversion on the data sub-table to obtain a target data table in a spatial layer format.

7. The vector space data transmission processing method according to claim 6, characterized in that: The server performs format conversion on the data sub-table to obtain a target data table in a spatial layer format, including: The server converts the coordinate string into a spatial feature object format for the data sub-tables, and converts each of the data sub-tables into a spatial layer format to obtain a target data table corresponding to each of the data sub-tables.

8. A vector space data transmission and processing system, characterized in that: The system comprises: The client and the server include a format conversion module, a volume compression module and a data transmission module; The format conversion module is used to convert the format of the vector space data to be uploaded to obtain pre-processed vector space data in a table format; The volume compression module is used to perform volume compression on the pre-processed vector space data according to a preset volume size to obtain a plurality of volume data; The data transmission module is used to transmit the volume data in parallel to transmit the volume data to the server; The server is used to decompress the received volume data and import it into the candidate data table of the target database, and perform format conversion on the candidate data table to obtain the target data table in the spatial layer format.

9. The vector space data transmission and processing system according to claim 8, characterized in that: The format conversion module is specifically used for: Convert the vector space data to be uploaded into a format using a preset data conversion tool to obtain pre-processed vector space data in a table format; The table format is a comma-separated value format, and the pre-processed vector spatial data stores coordinate values ​​of spatial vector elements in the vector spatial data to be uploaded and attribute values ​​corresponding to the spatial vector elements.

10. The vector space data transmission and processing system according to claim 8, characterized in that: The volume compression module is specifically used for: Get the preset volume size and target compressed file format; According to the preset sub-volume size and the target compressed file format, the pre-processed vector space data is subjected to sub-volume compression to obtain a plurality of sub-volume data.

Images