A point data management method, system and electronic device

Abstract

The application relates to a point data management method, system and electronic equipment, wherein the method comprises the following steps: acquiring point data and at least two data types of the point data, processing the point data according to a preset data structure and the data types to obtain substructure data and an index offset of each point data in the corresponding substructure data; determining a data part according to the substructure data, and obtaining an index part according to the data type of each point data and the corresponding index offset; determining a target data structure according to the index part and the data part, and storing the point data in the target data structure.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of Internet of Things data management, and in particular to a point data management method and system and an electronic device. BACKGROUND

[0002] In the field of Internet of Things, a "point" refers to a data source formed by various sensors and actuators on an Internet of Things device for collection, monitoring and control, and is one of the most important basic data units in an Internet of Things application scenario. In an application scenario in the field of Internet of Things, a device often has multiple points, and these points often have different data types and different sampling frequencies. Different types of point data are uploaded separately, thereby increasing the difficulty of data modeling and data application.

[0003] If the existing wide table modeling is used to store point data, many empty table cells will be stored, causing a serious data sparsity problem and wasting storage space. The more points and devices, the more obvious the data sparsity problem. The narrow table modeling scheme includes maintaining a data table for each different data type of point and uniformly converting all point types into a string type for storage. Maintaining a data table for each different data type of point requires the use of multiple independent data tables for management and storage when there are multiple data types of points. Although this solves the data sparsity problem, the number of data tables to be maintained will increase with the increase in point types, which brings inconvenience to data management, maintenance and use. If all point types are uniformly converted into a string type for storage, users only need to access one table object when performing write and query operations, greatly improving usability, but converting data into a string not only increases storage costs but also reduces query and calculation efficiency.

[0004] The existing point data management method has high storage cost and low storage efficiency, and an efficient point data management method is needed to improve the management capability of devices. SUMMARY

[0005] Embodiments of the present application provide a point data management method, system and electronic device to at least solve the problem of high storage cost and low storage efficiency of point data management methods in related technologies.

[0006] In a first aspect, embodiments of the present application provide a point data management method, comprising:

[0007] obtaining point data and at least two data types of the point data, processing the point data according to a preset data structure and the data types to obtain substructure data, and an index offset of each point data in the corresponding substructure data;

[0008] determine a data portion according to the substructure data, and obtain an index portion according to a data type and a corresponding index offset of each point data;

[0009] determine a target data structure according to the index portion and the data portion, and store the point data in the target data structure.

[0010] In an embodiment, the processing of the point data according to the preset data structure and the data type to obtain the substructure data and the index offset of each point data in the corresponding substructure data comprises:

[0011] storing the point data of each data type according to the preset data structure to obtain the substructure data corresponding to the data type and the index offset of each point data in the substructure data corresponding to the data type of the point data.

[0012] In an embodiment, after obtaining the substructure data corresponding to the data type, the method further comprises:

[0013] sorting and grouping the substructure data according to a preset sorting condition to obtain secondary grouping structure data, and sequentially compressively storing the secondary grouping structure data according to the sorting result.

[0014] In an embodiment, the storing of the point data in the target data structure comprises:

[0015] storing the point data according to a preset storage order, and determining an index of each data in the data portion in the target data structure according to the storage order.

[0016] In an embodiment, the method further comprises querying a target data, and the querying of the target data comprises:

[0017] obtaining a target index of the target data in the target data structure, and determining a target data type and a target index offset according to the target index;

[0018] determining the target data from the substructure data according to the target data type and the target index offset.

[0019] In an embodiment, the querying of the target data comprises:

[0020] in response to the data type of the target data being the same, returning the target data in the preset data structure;

[0021] in response to the data type of the target data being different, returning the target data in the target data structure.

[0022] In a second aspect, an embodiment of the present application provides a point data management system, comprising:

[0023] The acquisition module is configured to acquire point data and at least two data types of the point data, process the point data according to a preset data structure and the data types to obtain substructure data, and an index offset of each point data in the corresponding substructure data;

[0024] The determination module is configured to determine a data part according to the substructure data, and determine an index part according to a data type of each point data and the corresponding index offset;

[0025] The storage module is configured to determine a target data structure according to the index part and the data part, and store the point data in the target data structure.

[0026] In an embodiment, the acquisition module comprises:

[0027] The acquisition module is configured to store the point data of each data type in the corresponding substructure data according to the preset data structure, and obtain an index offset of each point data in the corresponding substructure data.

[0028] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the point data management method according to the first aspect when executing the computer program.

[0029] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executable on a processor to implement the point data management method according to the first aspect.

[0030] The point data management method, system and electronic device provided by the embodiments of the present application have at least the following technical effects.

[0031] The present application uniformly manages different types of point data through a target data structure, and stores different types of point data in the same data structure in the Internet of Things. Compared with the traditional scheme of simulating the storage of point data by using a string, the present application not only improves the storage density, but also effectively reduces the memory usage. The point data is stored after being grouped according to the type, which ensures the consistency of the data type in each group, thereby improving the storage compression rate of the point data and effectively reducing the storage cost.

[0032] Details of one or more embodiments of the present application are described in the following drawings and description to make other features, objects and advantages of the present application more clear and easy to understand. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0034] Figure 1 is a flow chart of a point data management method according to an embodiment of the application;

[0035] Figure 2 is a structural schematic diagram of an IOTANY Vector according to an exemplary embodiment;

[0036] Figure 3 is a schematic diagram of a point data grouping process according to an exemplary embodiment;

[0037] Figure 4 is a flow chart of point data persistence according to an exemplary embodiment;

[0038] Figure 5 is a structural block diagram of a point data management system according to an embodiment of the application;

[0039] Figure 6 is a structural schematic diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0040] In order to make the objects, technical solutions and advantages of the application clearer, the application is described and explained below in connection with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and should not be used to limit the application. Based on the embodiments provided by the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the application.

[0041] Obviously, the drawings in the following description are only some examples or embodiments of the application, and for those of ordinary skill in the art, the application can be applied to other similar scenarios without creative labor on the basis of these drawings. In addition, it can be understood that although the efforts made in this development process can be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the application, some design, manufacture or production changes based on the technical content disclosed in the application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the application.

[0042] Reference to an "example" in this application means that a particular feature, structure, or characteristic described in connection with the example can be included in at least one example of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same example, nor are they necessarily mutually exclusive of other examples. It is expressly understood that the examples described in this application are intended to be combined with each other in their broadest form unless otherwise expressly excluded.

[0043] Unless otherwise defined, technical terms or scientific terms used in the present application shall have the ordinary meanings commonly understood by one of ordinary skill in the art to which the present application pertains. The terms "a", "an", "one", "this", and the like, as used in the present application, do not denote a limitation of quantity and can mean one or more. The terms "including", "containing", "having", and the like, as used in the present application, are meant to be inclusive in a manner similar to the term "comprising". The terms "connected", "coupled", and the like, as used in the present application, are not intended to be limited to a direct connection or mechanical connection but can include an electrical connection, whether direct or indirect. The term "multiple" means two or more. The term "and / or" describes associated objects in association relationships, which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally means that the associated objects are in an "or" relationship. The terms "first", "second", "third", and the like, as used in the present application, are merely used to distinguish similar objects, and do not represent a specific order of the objects.

[0044] Different types of point data are uploaded separately, increasing the difficulty for users to model and use data. Efficient point data management helps enterprises achieve device monitoring, fault warning, and real-time monitoring, thereby improving the management capability of the device. Taking the Internet of Vehicles scenario as an example, various sensors are used to collect data such as battery consumption, vehicle speed, engine status, etc. Among them, the data type of battery consumption (elec) is integer, the data type of vehicle speed (speed) is double-precision floating point, and the data type of engine status (engineStatus) is string. The existing technology usually uses wide table or narrow table mode for data modeling.

[0045] (1) Table 1 is a data table obtained by using a wide table mode for modeling.

[0046] Table 1

[0047]

[0048] As shown in Table 1, each time sequence constitutes a row alone, and the wide table modeling has a data sparsity problem, wasting storage space. And with the increase of the point of the device, the data sparsity problem will be more and more obvious.

[0049] (2) A data table is maintained for each different data type point, and Table 2, Table 3 and Table 4 are battery power table (integer type point), vehicle speed table (double precision floating point type point) and engine state table (string type point) respectively.

[0050] Table 2

[0051]

[0052] Table 3

[0053]

[0054] Table 4

[0055]

[0056] As shown in Table 2 to Table 4, different data tables are maintained to store point data of different types and different frequencies. Although the data sparsity problem is solved, the number of tables to be maintained will increase with the increase of point types, which brings inconvenience to management, maintenance and use, such as when querying different points, different table objects need to be accessed.

[0057] (3) All point types are converted into strings for storage. In this way, only one table needs to be maintained to manage all point data, which is the current general solution for point data storage. Table 5 is a data table obtained by converting all point types into strings for storage.

[0058] Table 5

[0059]

[0060] As shown in Table 5, by using the method of converting all point types into strings for storage, the user only needs to access one table object when performing write and query operations, greatly improving the ease of use. However, converting data into strings increases storage costs and reduces query and calculation efficiency.

[0061] Therefore, an efficient and flexible point management method is needed to improve the point data management efficiency of Internet of Things enterprises.

[0062] Based on the above situation, the embodiments of the present application provide a point data management method, system and electronic device.

[0063] In a first aspect, the embodiments of the present application provide a point data management method, Figure 1 is a flowchart of a point data management method according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0064] In step S101, point data and at least two data types of the point data are obtained, substructure data is obtained by processing the point data according to a preset data structure and the data types, and an index offset of each point data in the corresponding substructure data is obtained.

[0065] Optionally, the preset data structure is a DolphinDB internal unified basic data structure Vector. By grouping the point data according to the data types, the consistency of the data types in each group is ensured, which is beneficial to improve the storage compression rate of the point data and effectively reduce the storage cost.

[0066] In one example, step S101 comprises: storing the point data of each data type according to the preset data structure to obtain substructure data corresponding to the data types and an index offset of each point data in the substructure data corresponding to the data type of the point data.

[0067] Optionally, a plurality of Vector data structures are used to store data of each basic data type (such as a string, a floating-point number, etc.), and an index of the point data in the corresponding Vector structure data is obtained.

[0068] In step S102, a data part is determined according to the substructure data, and an index part is obtained according to the data type of each point data and the corresponding index offset.

[0069] Optionally, all Vector structure data is taken as a data part of target structure data, and different types of data in the same logical row are associated by introducing an index (the data in the same logical row may be stored in different Vector data structures), so as to facilitate the storage of point data of different data types in the same structure.

[0070] In step S103, a target data structure is determined according to the index part and the data part, and the point data is stored in the target data structure.

[0071] Optionally, the target data structure is a self-named IOTANY type, representing an arbitrary data type in an Internet of Things (IOT) scenario, and an IOTANY Vector is divided into data (Data) and index (Index) two parts, and the data structure of the IOTANY Vector type can provide efficient physical storage support for any type. A new data type IOTANY is proposed, which unifies different types of point data in the Internet of Things and stores them in the same data structure Vector, and uses subscript index for access. Compared with the traditional scheme of simulating the storage of point data by using a string, this design not only improves the storage density, but also effectively reduces the memory usage.

[0072] For example, strings, floating-point numbers, and integer numbers can be placed in the same set of storage formats and have high efficiency. Figure 2 is a structural diagram of an IOTANY Vector according to an example embodiment, as Figure 2 shown, the data structure can store the same type of data in the same basic data structure Vector to form a subarray (SubVector) in the IOTANY Vector. The 0th, 1st, and 4th rows of the IOTANY Vector are INT types, which are sequentially recorded in the INT type subarray; the 2nd, 3rd, and 5th rows are DOUBLE types, which are sequentially recorded in the DOUBLE type subarray.

[0073] When a new element is written into the IOTANY Vector, the Figure 2 last element 21.49 is taken as an example, first, according to the type of the data (DOUBLE), it is determined that it should be written into the DOUBLE type subarray and become the last element of the subarray (i.e. the subscript is 2 in the subarray), then the type information of DOUBLE is written into the Types column in the index and the subscript 2 is written into the Indices column, and finally 21.49 is written into the DOUBLE subarray.

[0074] In this way, by the newly proposed target data type IOTANY, different types of point data in the Internet of Things are unified and stored in the same data structure Vector, and the index part containing the data type and the index is used for access. Compared with the traditional scheme of simulating the storage of point data by using a string, not only the storage density is improved, but also the memory usage is effectively reduced.

[0075] In one example, after obtaining the sub-structure data corresponding to the data type, the method further includes: sorting and grouping the sub-structure data according to a preset sorting condition to obtain secondary grouping structure data, and sequentially compressing and storing the secondary grouping structure data according to the sorting result.

[0076] Optionally, the point data is stored in a data file Level File of a TSDB (a time series data storage engine based on an LSM-Tree architecture developed by DolphinDB), and the Level File is a data storage file in a Partitioned Attributes Across (PAX) format. In the original persistent process, the data to be flushed to the disk is sorted and grouped according to the sorting key, and then each group is uniformly chunked according to the size and a Zone Map index is established, and then each data block is compressed and written to the disk in sequence according to the column to generate the Level File. The Zone Map index records the minimum value and the maximum value of multiple groups, and by comparing whether an input key value is between the minimum value and the maximum value of a group, it is determined whether the key value is in the group, so as to achieve the purpose of indexing.

[0077] Figure 3 is a schematic diagram of a point data grouping process according to an example embodiment, as shown in Figure 3 The present application considers that the point data usually has different data types, and when generating the Level File, the point data is first grouped according to the type of the point data. Figure 4 is a flowchart of a point data persistence process according to an example embodiment, as shown in Figure 4 After grouping according to the data type, the point data in each group is sorted and grouped according to the original persistent process based on the Sort Key (a sorting key specified when creating a TSDB distributed table), and an index is established. By ensuring that the point data types in each group are consistent, the compression rate of the point data is optimized, thereby significantly improving the storage efficiency. Moreover, a point data type index (Type Index) is introduced in the Level File. The index records the point data type and the offset (Offset) of the Zone Map index corresponding to the type data in the Level File.

[0078] In this way, the point data is grouped according to the type when stored in the Level File, and sorted according to the Sort Key, which ensures the consistency of the data types in each group, thereby improving the storage compression rate of the point data and effectively reducing the storage cost.

[0079] In one example, the point data is stored in a target data structure, including: storing the point data according to a preset storage order, and determining the index of each data in the target data structure according to the storage order. Optionally, the storage can be specified according to the time sequence of collection or the sequence of received data, the data index is generated according to the storage order, and the data and the index are stored.

[0080] In one example, the method further comprises querying the target data, and the querying the target data comprises:

[0081] At step S401, a target index of the target data in the target data structure is obtained, and a target data type and a target index offset are determined according to the target index.

[0082] At step S402, the target data is determined from the substructure data according to the target data type and the target index offset.

[0083] Optionally, when accessing data in the IOTANY Vector through the index, the index of the IOTANY is mapped to the index of the corresponding subarray through the index index. Refer to Figure 2 In the Figure 2 The second row of the IOTANY Vector is read, and it is found that it is an INT type, and the index in the INT subarray is 1. Then, data 90 at the index of 1 is read from the INT subarray.

[0084] Optionally, refer to Figure 4 When reading the point data from the Level File, first, the footer information at the end of the Level File is read to obtain the offset of the Type Index in the Level File. Then, all Type Indexes are located and read according to the offset to obtain the offset and data type of the Type Index, and then the Zone Map is located and read according to the offset in the Type Index. Finally, the target data to be read is determined according to the Zone Map combined with the query condition.

[0085] In one example, the method further comprises querying the target data, and the querying the target data comprises:

[0086] In response to the data types of the target data being the same, the target data is returned in a preset data structure.

[0087] In response to the data types of the target data being different, the target data is returned in a target data structure.

[0088] Optionally, if the data types corresponding to multiple indexes accessed are consistent, a basic Vector of the type is returned. If the data types are different, an IOTANY type Vector is returned. Refer to Figure 2When accessing the 2nd, 3rd and 5th elements in the IOTANY Vector, it can be determined by retrieving the type (Types) part in the index that these indices correspond to the 0th, 1st and 2nd elements in the DOUBLE sub-array, and finally the IOTANY Vector returns a Vector of type DOUBLE with element values [12.02, 9.97, 21.49]. When accessing the 1st, 2nd and 3rd elements in the IOTANY Vector, it can be determined by retrieving the type part in the index that their types are INT, INT and DOUBLE respectively, and are the 0th, 1st and 0th indices in the INT sub-array and the 0th index in the DOUBLE sub-array respectively, and finally a new IOTANY Vector is returned, storing the data content [89, 90, 12.02].

[0089] Generally, the point data types of the same device are the same, so when extracting data from the IOTANY Vector, if the data types are consistent, the result will be automatically converted into a normal Vector of that type, without additional data type conversion cost, but directly using the basic data type for calculation. In this way, the data type consistency can be intelligently judged during the query process, and the data type can be converted into the most suitable data structure, effectively avoiding the performance impact of data type conversion on query and calculation. Not only does it improve query performance, but it also better meets the practical application needs of Internet of Things point management.

[0090] As an example, the performance of the IOTANY type management point data proposed in the present application is tested. The test hardware environment is CPU: Intel Core i5-12500; memory size: 64 GiB; disk: PM9B1 NVMe Samsung 1TiB. The data size involved in the test is a total of 143,856,000 data, of which the point data includes 27,000,000 integer type data, 21,600,000 long integer type data, 62,856,000 floating point type data and 32,400,000 double precision floating point type data. Table 6 is a comparison table of write performance of using IOTANY type to store point data and using string to simulate storage of point data.

[0091] Table 6

[0092]

[0093] As shown in Table 6, compared with the traditional way of using string to simulate storage of point data, the IOTANY type storage of point data has higher compression ratio and higher data write throughput.

[0094] As an example, the query performance of the IOTANY type management point position data proposed in the application is tested. The data scale involved in the test is a total of 9,000,000 data, of which there are a total of 500 devices, each device has 300 point positions, and each point position has 60 data. Table 7 is a comparison table of query performance of DolphinDB and Apache IoTDB in the point position management scenario, wherein DolphinDB uses IOTANY type to store and manage point position data, and Apache IoTDB uses a string simulation method to store and manage point position data.

[0095] The query task is to aggregate and calculate (such as average value) the point position data of some devices at a certain time.

[0096] Table 7

[0097]

[0098] As shown in Table 7, DolphinDB uses IOTANY type to store and manage point position data, which has obvious advantages in point position data query and calculation.

[0099] In summary, the application unifies the management of different types of point position data through the target data structure, stores different types of point position data in the same data structure Vector in the Internet of Things, and uses an index for access. Compared with the traditional scheme of simulating point position data storage by using a string, not only the storage density is improved, but also the memory usage is effectively reduced. And using the target data structure can intelligently judge the data type consistency in the query process, and convert the data type to the most suitable data structure, effectively avoiding the performance impact of data type conversion on query and calculation, not only improving the query performance, but also more in line with the actual application requirements of Internet of Things point position management. When storing in Level File, the point position data is grouped according to the type and then sorted according to the Sort Key, ensuring the consistency of the data type in each group, thereby improving the storage compression rate of the point position data and effectively reducing the storage cost.

[0100] In a second aspect, the embodiments of the application provide a point position data management system, Figure 5 is a structural block diagram of a point position data management system according to an embodiment of the application, as Figure 5 shown, the system comprises:

[0101] The acquisition module 100 is configured to acquire point position data and at least two data types of the point position data, process the point position data according to a preset data structure and the data types to obtain substructure data, and an index offset of each point position data in the corresponding substructure data.

[0102] The determining module 200 is configured to determine the data part according to the substructure data, and obtain the index part according to the data type and the corresponding index offset of each point data.

[0103] The storage module 300 is configured to determine the target data structure according to the index part and the data part, and store the point data in the target data structure.

[0104] In one example, the obtaining module 100 comprises: a module configured to store the point data of each data type in a preset data structure respectively, to obtain the substructure data corresponding to the data type, and the index offset of each point data in the substructure data corresponding to the data type.

[0105] In one example, after obtaining the substructure data corresponding to the data type, the system further comprises:

[0106] sorting and grouping the substructure data according to a preset sorting condition to obtain secondary grouping structure data, and sequentially compressing and storing the secondary grouping structure data according to the sorting result.

[0107] In one example, the storage module 300 comprises: a module configured to store the point data according to a preset storage order, and determine the index of each data in the data part in the target data structure according to the storage order.

[0108] In one example, the system is further configured to query the target data, and the querying of the target data comprises:

[0109] obtaining the target index of the target data in the target data structure, and determining the target data type and the target index offset according to the target index.

[0110] determining the target data from the substructure data according to the target data type and the target index offset.

[0111] In one example, the querying of the target data comprises:

[0112] in response to the data type of the target data being the same, returning the target data in the preset data structure.

[0113] in response to the data type of the target data being different, returning the target data in the target data structure.

[0114] In summary, the application unifies the management of different types of point data through a target data structure, stores different types of point data in the Internet of Things in the same data structure Vector, and uses an index for access. Compared with the traditional scheme of simulating the storage of point data by using a string, the storage density is improved, and the memory usage is effectively reduced. Using the target data structure can intelligently judge the consistency of the data type in the query process, and convert the data type to the most suitable data structure, effectively avoiding the performance impact of data type conversion on query and calculation, not only improving the query performance, but also more in line with the actual application requirements of point management in the Internet of Things. When storing in the Level File, the point data is grouped according to the type and then sorted according to the Sort Key, ensuring the consistency of the data type in each group, thereby improving the storage compression rate of point data and effectively reducing the storage cost.

[0115] In a third aspect, an electronic device is provided, Figure 6 A structural schematic diagram of an electronic device provided by the embodiment of the application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the point data management method provided in the first aspect when executing the program, Figure 6 The electronic device 60 shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the application.

[0116] The electronic device 60 can be in the form of a general computing device, for example, it can be a server device. The components of the electronic device 60 can include but are not limited to the above-mentioned at least one processor 61, the above-mentioned at least one memory 62, and a bus 63 connecting different system components including the memory 62 and the processor 61.

[0117] The bus 63 includes a data bus, an address bus, and a control bus.

[0118] The memory 62 can include volatile memory, such as a random access memory (RAM) 621 and / or a cache memory 622, and can further include a read-only memory (ROM) 623.

[0119] The memory 62 can further include programs / utilities 625 having a set of (at least one) program modules 624, such as an operating system, one or more application programs, other program modules, and program data, each of which or some combination of which can include the implementation of a network environment.

[0120] The processor 61 performs various function applications and data processing by running the computer program stored in the memory 62, such as the point data management method provided in the first aspect of the present application.

[0121] The electronic device 60 can also communicate with one or more external devices 64 such as a keyboard, a pointing device, etc. via an input / output (I / O) interface 65. Further, the model generating device 60 can communicate with one or more networks such as a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet, via a network adapter 66. As depicted, the network adapter 66 communicates with the other modules of the model generating device 60 via the bus 63. It should be appreciated that other hardware and / or software modules can be used in conjunction with the model generating device 60, such as a microcode, device drivers, redundant processing units, external disk drive arrays, RAID (Redundant Array of Independent Disks) systems, tape drives, and data archival storage systems, etc., although not shown in the figure.

[0122] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. Indeed, according to an embodiment of the application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into several units / modules embodied by several units / modules.

[0123] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, having a program stored thereon, where the program, when executed by a processor, implements the point data management method provided in the first aspect.

[0124] More specifically, the computer readable storage medium can include, but is not limited to, portable discs, hard disks, random access memories, read-only memories, erasable programmable read-only memories, optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0125] In a possible implementation, the present application can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps of implementing the point data management method provided in the first aspect when the program product is run on the terminal device.

[0126] The program codes for implementing the present application can be written in any combination of one or more programming languages, and can be executed entirely on the user device, partly on the user device and partly on a remote device, or entirely on a remote device.

[0127] The technical features of the above-described embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features are described in the above description, but it is understood that the scope of the specification includes all possible combinations of the technical features.

[0128] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be construed as limiting the scope of the patent. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A point data management method characterized by comprising: The method comprises the following steps: obtaining point data and a data type of the point data, processing the point data according to a preset data structure and the data type to obtain substructure data and an index offset of each point data in the corresponding substructure data, and specifically comprising the following steps: storing the point data of each data type according to the preset data structure to obtain substructure data corresponding to the data type and an index offset of each point data in the substructure data corresponding to the data type of the point data; determining a data part according to the substructure data, and obtaining an index part according to the data type and the corresponding index offset of each point data, wherein all Vector structure data are taken as the data part of target structure data, and different types of data in the same logical line are associated by introducing an index to store point data of different data types in the same structure; determining a target data structure according to the index part and the data part, and storing the point data in the target data structure.

2. The point data management method according to claim 1, wherein After obtaining the substructure data corresponding to the data type, the method further comprises the following steps: sorting and grouping the substructure data according to a preset sorting condition to obtain secondary grouping structure data, and sequentially compressing and storing the secondary grouping structure data according to the sorting result.

3. The point data management method according to claim 1, wherein The step of storing the point data in the target data structure comprises the following steps: storing the point data according to a preset storage order, and determining the subscript of each data in the target data structure in the data part according to the storage order.

4. The point data management method according to claim 3, wherein The method further comprises querying target data, and the step of querying the target data comprises the following steps: obtaining a target subscript of the target data in the target data structure, determining a target data type and a target index offset according to the target subscript; determining the target data from the substructure data according to the target data type and the target index offset.

5. The point data management method according to claim 4, wherein The step of querying the target data comprises the following steps: in response to the data type of the target data being the same, returning the target data in the preset data structure; in response to the data type of the target data being different, returning the target data in the target data structure.

6. A point data management system characterized by, The method comprises the following steps: an obtaining module: used for obtaining point data and a data type of the point data, processing the point data according to a preset data structure and the data type to obtain substructure data and an index offset of each point data in the corresponding substructure data, and specifically comprising the following steps: storing the point data of each data type according to the preset data structure to obtain substructure data corresponding to the data type and an index offset of each point data in the substructure data corresponding to the data type of the point data; a determining module: used for determining a data part according to the substructure data, and obtaining an index part according to the data type and the corresponding index offset of each point data, wherein all Vector structure data are taken as the data part of target structure data, and different types of data in the same logical line are associated by introducing an index to store point data of different data types in the same structure; A storage module is configured to determine a target data structure according to the index part and the data part, and store the point data in the target data structure.

7. An electronic device, comprising: The computer program is stored in the memory and executable in the processor, and the processor executes the computer program to implement the point data management method according to any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executable in the processor to implement the point data management method according to any one of claims 1 to 5.

Images