Data export method, device and equipment of distributed native graph database and medium

Abstract

The application discloses a data export method and device of a distributed native graph database, equipment and medium, relates to the technical field of databases, and comprises the following steps: determining to-be-exported vertices according to a data export request, creating a hash index table set comprising a plurality of sub-hash tables according to the segment identifiers of the to-be-exported vertices; the sub-hash tables comprise a main index area, a collision chain area and a variable-length primary key storage area; scanning the to-be-exported vertices, writing business primary keys into the variable-length primary key storage area of a target sub-hash table to obtain an offset pointer; determining a target slot according to the hash value of a row identifier and writing an index record containing the offset pointer to generate a vertex export file; if a hash collision occurs, a collision slot is allocated in the collision chain area to move the original index record; and the target offset pointer is queried based on the start and end row identifiers of an edge to obtain business primary keys and export an edge export file. The application can reduce resource consumption in the export process and improve the efficiency of large data export.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a method, apparatus, device, and medium for exporting data from a distributed native graph database. Background Technology

[0002] In data export scenarios of distributed native graph databases, existing pure in-memory hash table solutions store all mappings in process memory or a single memory-mapped buffer. When the vertex size reaches tens of millions to hundreds of millions, the peak memory usage becomes excessive; memory amplification is even more pronounced when the primary key is long; and it is difficult to adapt to the segmented storage topology of distributed graph databases. Existing general-purpose external storage hash structures do not incorporate the segmented topology of graph databases and row identifier encoding rules, and are not optimized for the two-stage export process of writing indexes in the vertex stage and reading indexes in the edge stage. Index directories, conflict chains, and variable-length primary key values ​​are usually not stored separately or encoded in fixed-length slots according to the export scenario.

[0003] If the primary key is retrieved from the storage engine for each row identifier during the edge export phase, it will incur huge network and storage overhead, making it unsuitable for large-scale batch export. Some persistent hashing schemes preset a fixed capacity ratio between the main index area and the collision chain area. When the distribution of segmented data is skewed or the hash collision rate increases, the collision chain area is prone to premature exhaustion or wasted space; moreover, there is a lack of sharded indexes aligned with the underlying segment distribution, and a lack of real-time collision rate monitoring and dynamic expansion mechanism for the collision chain area during the export process.

[0004] In conclusion, improving the efficiency of exporting large amounts of data is a problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for exporting data from a distributed native graph database, which can improve the efficiency of exporting large amounts of data. The specific solution is as follows: Firstly, this application provides a method for exporting data from a distributed native graph database, including: Based on the data export request, the vertex to be exported is determined from the distributed native graph database. The corresponding segment identifier is obtained by reading the vertex to be exported. A set of hash index tables is created based on the segment identifier. The set of hash index tables includes several sub-hash tables that are routed according to the segment identifier. The sub-hash tables include a main index area, a collision chain area, and a variable-length primary key storage area. Scan the vertices to be exported, extract the corresponding row identifier, business primary key, attributes and graph storage segmentation information, determine the target sub-hash table from the hash index table set according to the graph storage segmentation information, write the business primary key into the variable-length primary key storage area of ​​the target sub-hash table, and obtain the corresponding offset pointer; The target slot is determined in the main index area of ​​the target sub-hash table based on the hash value of the row identifier. The index record containing the offset pointer is written into the target slot. A vertex export file is generated and exported based on the business primary key and the attribute. If a hash collision occurs when writing to the target slot, a new collision slot is allocated in the collision chain area, the original index record in the target slot is moved to the collision slot, and then the index record corresponding to the vertex to be exported is written to the target slot. Based on the data export request, the distributed native graph database is scanned to generate scan results. The target offset pointer is queried from the corresponding sub-hash table based on the starting row identifier and ending row identifier of the edge in the scan results. The corresponding business primary key is obtained from the variable-length primary key storage area based on the target offset pointer, so as to export the edge export file containing the business primary key and edge attributes.

[0006] Optionally, the step of determining the vertices to be exported from the distributed native graph database according to the data export request, reading the corresponding segment identifiers from the vertices to be exported, and creating a hash index table set based on the segment identifiers includes: The vertex to be exported is determined based on the data export request. The graph data of the vertex to be exported in the distributed native graph database is read to determine the data size of the vertex to be exported and the number of segments distributed by the vertex to be exported. A set of hash index tables is created based on the data size and the number of segments.

[0007] Optionally, determining the target sub-hash table from the hash index table set based on the graph storage segmentation information includes: Determine the segmented routing mapping table; the segmented routing mapping table records the correspondence between each segment and the sub-hash table; Parse the segment information of the graph storage to obtain the segment identifier, and query the segment routing mapping table to see if a record corresponding to the segment identifier already exists; If it does not exist, the query operation is locked based on double-checked locking technology, and the segmented routing mapping table is queried again. If it is not found, an idle sub-hash table is taken from the hash index table set as the target sub-hash table corresponding to the segment identifier, and the correspondence between the segment identifier and the sub-hash table is registered in the segmented routing mapping table. If the hash table exists or is matched after a second query, the sub-hash table corresponding to the segment identifier is determined as the target sub-hash table.

[0008] Optionally, determining the target slot in the main index area of ​​the target sub-hash table based on the hash value of the row identifier includes: The row identifier is hashed to determine the corresponding hash value, and the hash value is moduloed by the fixed capacity of the main index area, with the modulo result used as the slot index; The slot corresponding to the slot index in the main index area is determined as the target slot.

[0009] Optionally, the step of allocating a new conflict slot in the conflict chain region, moving the original index record in the target slot to the conflict slot, and then continuing to write the index record corresponding to the vertex to be exported into the target slot includes: At the end of the conflict chain area, a new conflict slot is assigned, and the original index record in the target slot is copied to the conflict slot as is to obtain the index of the conflict slot in the conflict chain area. Change the conflict flag of the target slot to "entered the conflict chain", write the offset pointer of the business primary key corresponding to the vertex to be exported into the offset pointer field of the target slot, and write the index of the conflicting slot into the predecessor pointer field of the target slot to generate a predecessor pointer.

[0010] Optionally, the step of scanning the distributed native graph database to generate scan results based on the data export request, and querying the target offset pointer from the corresponding sub-hash table based on the starting row identifier and ending row identifier of the edges in the scan results, includes: The starting row identifier and ending row identifier of the edge are determined based on the data export request. The target slot in the main index area is located based on the hash value of the starting row identifier or the ending row identifier; Based on the offset pointer in the target slot, the corresponding row identifier is read from the variable-length primary key storage area, and it is determined whether the read row identifier is the same as the starting row identifier or the ending row identifier. If they are the same, then the offset pointer in the target slot is taken as the target offset pointer; If they are not the same, then read the conflict flag bit of the target slot; If the conflict flag indicates that the query has not entered the conflict chain, the query is deemed to have failed. If the conflict flag indicates that a conflict chain has been entered, the predecessor pointer based on the target slot backtracks in the conflict chain area until it matches the starting row identifier or the ending row identifier, and the offset pointer of the conflict slot that satisfies the match is used as the target offset pointer.

[0011] Optionally, the method further includes: The collision rate is determined based on the total number of entries written to the sub-hash table and the number of entries with hash collisions, and the utilization rate is determined based on the used slots and total capacity of the collision chain area in the sub-hash table. If the conflict rate is greater than a preset high water level threshold, or the utilization rate is greater than a preset emergency threshold, then an additional storage block is added to the conflict chain area based on a preset step size.

[0012] Secondly, this application provides a data export device for a distributed native graph database, comprising: The index table creation module is used to determine the vertices to be exported from the distributed native graph database according to the data export request, read the vertices to be exported to obtain the corresponding segment identifiers, and create a set of hash index tables according to the segment identifiers; the set of hash index tables includes several sub-hash tables routed according to the segment identifiers, and the sub-hash tables include a main index area, a collision chain area, and a variable-length primary key storage area; The pointer acquisition module is used to scan the vertex to be exported, extract the corresponding row identifier, business primary key, attribute and graph storage segment information, determine the target sub-hash table from the hash index table set according to the graph storage segment information, write the business primary key into the variable-length primary key storage area of ​​the target sub-hash table, and obtain the corresponding offset pointer. The vertex export module is used to determine the target slot in the main index area of ​​the target sub-hash table according to the hash value of the row identifier, write the index record containing the offset pointer into the target slot, and generate and export the vertex export file based on the business primary key and the attribute. The record writing module is used to allocate a new conflict slot in the conflict chain area if a hash collision occurs when writing to the target slot, move the original index record in the target slot to the conflict slot, and then continue to write the index record corresponding to the vertex to be exported to the target slot. The edge export module is used to scan the distributed native graph database according to the data export request to generate scan results. Based on the starting row identifier and ending row identifier of the edge in the scan results, the module queries the target offset pointer from the corresponding sub-hash table. Based on the target offset pointer, the module obtains the corresponding business primary key from the variable-length primary key storage area to export an edge export file containing the business primary key and edge attributes.

[0013] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the data export method for the distributed native graph database as described above.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the data export method for the distributed native graph database as described above.

[0015] In summary, this application determines the vertices to be exported from the distributed native graph database based on the data export request, reads the corresponding segment identifiers from the vertices to be exported, and creates a set of hash index tables based on the segment identifiers. The hash index table set includes several sub-hash tables routed according to the segment identifiers. Each sub-hash table includes a main index area, a conflict chain area, and a variable-length primary key storage area. The application scans the vertices to be exported, extracts the corresponding row identifiers, business primary keys, attributes, and graph storage segment information, determines the target sub-hash table from the hash index table set based on the graph storage segment information, writes the business primary key into the variable-length primary key storage area of ​​the target sub-hash table, and obtains the corresponding offset pointer. Based on the hash value of the row identifier, the application determines the target slot in the main index area of ​​the target sub-hash table and... The index record containing the offset pointer is written to the target slot, and a vertex export file is generated and exported based on the business primary key and the attribute. If a hash collision occurs when writing to the target slot, a new collision slot is allocated in the collision chain area, the original index record in the target slot is moved to the collision slot, and then the index record corresponding to the vertex to be exported is written to the target slot. The distributed native graph database is scanned according to the data export request to generate scan results. Based on the starting row identifier and ending row identifier of the edge in the scan results, the target offset pointer is queried from the corresponding sub-hash table. Based on the target offset pointer, the corresponding business primary key is obtained from the variable-length primary key storage area to export the edge export file containing the business primary key and the edge attribute.

[0016] As described above, this application first determines the vertices to be exported based on the export request and reads the segment identifiers attached to these vertices. Then, it creates a corresponding hash index table set for each segment. This set is internally divided into multiple sub-hash tables according to segment routing. Each sub-hash table contains three parts: a main index area, a collision chain area, and a variable-length primary key storage area. During the scanning of the vertices to be exported, the system extracts the row identifier, business primary key, attributes, and graph storage segment information for each vertex. Based on the segment information, it locates the target sub-hash table, stores the business primary key in the variable-length primary key storage area of ​​the sub-hash table, and obtains the offset pointer. Simultaneously, based on the hash value of the row identifier, it determines the target slot in the main index area and writes the index record containing the offset pointer into the slot. If a hash collision occurs, the original slot is not overwritten; instead, a new slot is allocated in the collision chain area, the original record is moved to the collision slot, and then the new record is written, thus ensuring index integrity. After vertex processing is completed, a vertex export file containing the business primary key and attributes is generated and exported. For edge export, based on the start and end row identifiers of each edge obtained from the database scan results, the target offset pointer is retrieved from the corresponding sub-hash table. This pointer is then used to extract the business primary key from the variable-length primary key storage area, ultimately generating an edge export file containing the business primary key and edge attributes, thus achieving the export of the association between vertices and edges. In this way, through segmented routing and a layered design using hash indexes, the data export efficiency of the distributed graph database can be significantly improved. The variable-length primary key storage and conflict chain mechanism reduce memory redundancy and write amplification caused by hash collisions, while ensuring data consistency. Fast index lookup based on row identifiers reduces the primary key lookup complexity during edge record export to constant level, effectively supporting high-concurrency export and complex association scenarios for large-scale data while reducing resource consumption. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart of a data export method for a distributed native graph database disclosed in this application; Figure 2 This application discloses a specific method for exporting data from a distributed native graph database. Figure 3 This is a schematic diagram of the data export device structure for a distributed native graph database disclosed in this application; Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Currently, in data export scenarios of distributed native graph databases, existing pure in-memory hash table solutions store all mappings in process memory or a single memory-mapped buffer. When the vertex size reaches tens of millions to hundreds of millions, the peak memory usage becomes excessive; memory amplification is even more pronounced when the primary key is long; and it is difficult to adapt to the segmented storage topology of distributed graph databases. Existing general external storage hash structures do not incorporate the segmented topology of graph databases and row identifier encoding rules, and are not optimized for the two-stage export process of writing indexes in the vertex stage and reading indexes in the edge stage. Index directories, conflict chains, and variable-length primary key values ​​are usually not stored separately and encoded in fixed-length slots according to the export scenario. If the primary key is retrieved by querying the storage engine for each row identifier in the edge export stage, it will generate huge network and storage overhead, making it unsuitable for large-scale batch export. To solve the above technical problems, this application discloses a data export method, apparatus, device, and medium for distributed native graph databases, which can improve the efficiency of exporting large amounts of data.

[0021] See Figure 1 As shown in the figure, this invention discloses a data export method for a distributed native graph database, including: Step S11: Determine the vertex to be exported from the distributed native graph database according to the data export request, read the vertex to be exported to obtain the corresponding segment identifier, and create a hash index table set according to the segment identifier; the hash index table set includes several sub-hash tables routed according to the segment identifier, and the sub-hash tables include a main index area, a collision chain area, and a variable-length primary key storage area.

[0022] In this embodiment, the data export request is a batch export instruction initiated by the user or upper-layer business system to the distributed native graph database. The data export request may carry information such as the graph space identifier to be exported, vertex type, edge type, attribute filtering conditions, and export file format. Correspondingly, the vertices to be exported are the vertices in the distributed native graph database that match the graph space, vertex type, and other conditions specified in the data export request. It can be understood that in the distributed native graph database, graph data is divided into several segments according to preset segmentation rules and distributed across different storage nodes. Each segment has a unique segment identifier, and the row identifier of a vertex usually encodes the segment information of its segment. Therefore, the corresponding segment identifier can be obtained by reading the vertex to be exported.

[0023] Then, based on the data export request, the vertices to be exported are determined. The graph metadata of these vertices in the distributed native graph database is read to determine their data size and the number of segments they are distributed across. A set of hash index tables is created based on the data size and the number of segments. Specifically, the graph metadata is metadata in the distributed native graph database used to describe the organization of graph data. This metadata may include, for example, statistical information on the number of various vertices, the distribution information of vertices across segments, and the segment identifiers of each segment. By reading the graph metadata, the data size of the vertices to be exported (i.e., the total number of vertices or its order of magnitude) and the number of segments they are distributed across can be determined. When creating a hash index table set, the number of sub-hash tables can be determined based on the number of segments, for example, by assigning one sub-hash table to each segment. The fixed capacity of the main index area in each sub-hash table can be determined based on the data size. For example, the average number of vertices in each segment can be estimated by dividing the data size by the number of segments, and then amplified by a preset load factor to obtain the total number of slots in the main index area. Simultaneously, initial capacity is allocated to the collision chain area according to a preset ratio of the main index area capacity, and initial storage space is reserved for the variable-length primary key storage area. In this way, the capacity planning of the hash index table set matches the actual size and segment distribution of the data to be exported, avoiding both frequent expansions due to insufficient capacity estimation and space waste due to excessive capacity estimation.

[0024] Furthermore, when creating a set of hash index tables based on segment identifiers, a corresponding sub-hash table can be assigned to each segment identifier. These sub-hash tables are independent of each other, and subsequent index write and query operations are routed to the corresponding sub-hash table according to the segment identifier. This avoids lock contention and capacity bloat caused by concentrating all mappings in a single hash table, and keeps the index structure aligned with the underlying segment distribution. Specifically, each sub-hash table can be physically divided into three areas: a main index area, a collision chain area, and a variable-length primary key storage area. The main index area consists of fixed-length slots of fixed capacity, used to store index records addressed by row identifier hash values. The collision chain area also uses fixed-length slot encoding to store index records moved after hash collisions. The variable-length primary key storage area is used to store row identifiers, primary key lengths, and primary key content in an append-only manner. The fixed-length index slot is 10 bytes per slot. The first byte is the conflict flag; bytes 2 to 6 are the offset pointers, pointing to the starting position of the primary key data in the variable-length primary key storage area; bytes 7 to 10 are the inline data field or conflict chain predecessor pointers.

[0025] Step S12: Scan the vertex to be exported, extract the corresponding row identifier, business primary key, attribute and graph storage segment information, determine the target sub-hash table from the hash index table set according to the graph storage segment information, write the business primary key into the variable-length primary key storage area of ​​the target sub-hash table, and obtain the corresponding offset pointer.

[0026] In this embodiment, the row identifier is a unique identifier for a vertex within the distributed native graph database storage engine, such as the row key assigned to the vertex by the storage engine; the business primary key is a unique identifier for a vertex at the business level, such as user ID, device ID, order ID, etc. The length of the business primary key is often not fixed and may be quite long; attributes are various attribute fields carried by the vertex; graph storage segmentation information is used to characterize the segment to which the vertex belongs, for example, it can be parsed from the row identifier, or it can be directly obtained from the metadata attached to the scan results.

[0027] During the scanning of vertices to be exported, scanning tasks can be initiated in parallel according to segments. Each scanning task is responsible for reading vertices within one or more segments, thus fully utilizing the concurrency capabilities of the distributed cluster. For each vertex obtained from the scan, the segment identifier is parsed from its graph storage segment information, and the target sub-hash table corresponding to the segment identifier is determined from the hash index table set accordingly. Then, the business primary key of the vertex is appended to the variable-length primary key storage area of ​​the target sub-hash table. After the write is completed, the write position of the business primary key in the variable-length primary key storage area, i.e., the offset pointer, can be obtained. It can be understood that the offset pointer can include the starting offset of the business primary key in the variable-length primary key storage area, and can also include length information, so that the complete business primary key can be accurately read from the variable-length primary key storage area according to the offset pointer later. Since the variable-length primary key storage area stores all variable-length business primary keys in a centralized and sequential manner, while the main index area and the conflict chain area only store fixed-length index records, no space waste of index slots will occur regardless of the length of the business primary key, effectively alleviating the memory amplification problem when the primary key is long.

[0028] In this embodiment, after obtaining the graph storage segmentation information, a segmentation routing mapping table is determined. The segmentation routing mapping table registers the correspondence between each segment and a sub-hash table. The graph storage segmentation information is parsed to obtain segment identifiers. Based on the segment identifiers, a query is performed to check if a record corresponding to the segment identifier already exists in the segmentation routing mapping table. If not, a double-checked locking technique is used to lock the query operation, and the segmentation routing mapping table is queried again. If no match is found, a free sub-hash table is retrieved from the hash index table set as the target sub-hash table corresponding to the segment identifier, and the correspondence between the segment identifier and the sub-hash table is registered in the segmentation routing mapping table. If a match is found after another query, the sub-hash table corresponding to the segment identifier is determined as the target sub-hash table. Specifically, the segmentation routing mapping table registers the correspondence between each segment and a sub-hash table; for example, it can be a mapping table with the segment identifier as the key and the table identifier or table handle of the sub-hash table as the value. When determining the target sub-hash table, the segment identifier is first obtained by parsing the segment information stored in the graph. Then, based on the segment identifier, the segment routing map is queried to see if a record corresponding to the segment identifier already exists. If it exists, the sub-hash table corresponding to the segment identifier is directly determined as the target sub-hash table. If it does not exist, it means that the routing relationship for the segment has not yet been established. At this time, the query operation is locked using double-checked locking technology, and the segment routing map is queried again. If the query is successful again, it means that other concurrent threads have completed the registration of the segment during the gap before locking, and the successful sub-hash table is directly determined as the target sub-hash table. If the query is unsuccessful again, an idle sub-hash table is taken from the hash index table set as the target sub-hash table corresponding to the segment identifier, and the correspondence between the segment identifier and the sub-hash table is registered in the segment routing map table. Then, the lock is released.

[0029] It is understood that in a scenario where multiple threads scan vertices in parallel, multiple threads may encounter vertices in the same segment for the first time at the same time. By using double-checked locking technology, locking is only performed when the mapping relationship is established for the first time, and most other queries can be completed without locking. This ensures that the same segment identifier will only establish a correspondence with a unique sub-hash table, avoiding duplicate allocation, and minimizes lock overhead, thus guaranteeing routing performance under high-concurrency scanning.

[0030] Step S13: Determine the target slot in the main index area of ​​the target sub-hash table according to the hash value of the row identifier, write the index record containing the offset pointer into the target slot, and generate and export the vertex export file based on the business primary key and the attribute.

[0031] In this embodiment, the row identifier is hashed to determine the corresponding hash value. The hash value is then moduloed by the fixed capacity of the main index area, and the modulo result is used as the slot index. The slot corresponding to the slot index in the main index area is determined as the target slot. Specifically, a hash algorithm can be used to perform a hash operation on the row identifier to obtain a hash value. The hash value is then moduloed by the fixed capacity to determine the slot index, and the target slot is located based on the slot index. After determining the target slot, an index record containing an offset pointer is written to the target slot. Specifically, the index record can be a fixed-length structure, and its fields can include an offset pointer field for storing the offset pointer, a conflict flag bit for indicating whether the slot has entered the conflict chain, and a predecessor pointer field for pointing to the conflicting slot in the conflict chain area.

[0032] Simultaneously with the writing of the vertex index, a vertex export file is generated and exported based on the vertex's business primary key and attributes. The format of the vertex export file can be flexibly set according to the data export request; for example, it can be a comma-separated value file or other structured data files. Each record in the vertex export file can contain a vertex's business primary key and its various attributes. Thus, while completing the vertex data export, the vertex stage also completes the construction of the row identifier to business primary key mapping index, preparing for the primary key query in the subsequent edge export stage and avoiding the need for source lookups to the storage engine during the edge export stage.

[0033] Step S14: If a hash collision occurs when writing to the target slot, a new collision slot is allocated in the collision chain area, the original index record in the target slot is moved to the collision slot, and then the index record corresponding to the vertex to be exported is written to the target slot.

[0034] It is understood that, due to the limited slot capacity of the main index area, different row identifiers may be mapped to the same slot after hashing, i.e., a hash collision occurs. In this embodiment, when a hash collision occurs, the original index record in the target slot is not directly overwritten, nor is open addressing probing performed in the main index area. Instead, a new collision slot is allocated at the end of the collision chain area, and the original index record in the target slot is copied to the collision slot as is to obtain the index of the collision slot in the collision chain area. The collision flag of the target slot is changed to indicate that it has entered the collision chain, the offset pointer of the business primary key corresponding to the vertex to be exported is written into the offset pointer field of the target slot, and the index of the collision slot is written into the predecessor pointer field of the target slot to generate a predecessor pointer.

[0035] Specifically, in the collision chain area, collision slots can be managed using an append-only allocation method. That is, each time a hash collision occurs, a new collision slot is allocated at the end of the collision chain area. The collision slot and the slots in the main index area can use the same fixed-length structure. After copying the original index record from the target slot to the newly allocated collision slot, the offset pointer field, collision flag, and predecessor pointer field of the original index record are completely preserved. Therefore, even if the original index record is already in the collision chain, meaning its predecessor pointer field points to an earlier collision slot, the copy operation will not disrupt the existing chain relationship. Subsequently, the flag of the main index slot is set to indicate that it has entered the collision chain, bytes two through six are changed to the file offset of the new primary key, and bytes seven through ten are written to an index to generate a predecessor pointer. Thus, the main index slot always stores the last record written at that hash position, and the replaced original record is completely preserved in the collision chain area and can still be accessed through the predecessor index. It should be noted that the maximum capacity of the collision chain area cannot exceed one times the capacity of the main index area.

[0036] In this way, the target slot in the main index area stores the index record of the most recently written vertex, and its predecessor pointer points to the conflicting slot that stores the previous index record. If there are more than two vertices mapped to the same slot, the conflicting slots are sequentially linked through the predecessor pointer to form a backtracking chain starting from the target slot and running through the conflict chain area. Since the movement of conflicting records only involves copying fixed-length slots and rewriting pointers, and does not involve moving business primary keys in the variable-length primary key storage area, the write amplification caused by hash collisions is controlled to a very small range.

[0037] Step S15: Scan the distributed native graph database according to the data export request to generate scan results. Based on the starting row identifier and ending row identifier of the edge in the scan results, query the target offset pointer from the corresponding sub-hash table. Based on the target offset pointer, obtain the corresponding business primary key from the variable-length primary key storage area to export the edge export file containing the business primary key and edge attributes.

[0038] In this embodiment, the starting row identifier and ending row identifier of the edge are determined according to the data export request; the target slot in the main index area is located according to the hash value of the starting row identifier or the ending row identifier; the corresponding row identifier is read from the variable-length primary key storage area based on the offset pointer in the target slot, and it is determined whether the read row identifier is the same as the starting row identifier or the ending row identifier; if they are the same, the offset pointer in the target slot is used as the target offset pointer; if they are different, the conflict flag bit of the target slot is read; if the conflict flag bit indicates that it has not entered the conflict chain, the query is determined to be unsuccessful; if the conflict flag bit indicates that it has entered the conflict chain, the predecessor pointer of the target slot is used to backtrack in the conflict chain area until the starting row identifier or the ending row identifier is matched, and the offset pointer of the conflict slot that satisfies the match is used as the target offset pointer.

[0039] Specifically, based on the data export request, the edge data in the distributed native graph database is scanned to generate scan results. Each edge in the scan results must contain at least its start row identifier, end row identifier, and edge attributes. Since the edge records in the exported file need to express the start and end points in the form of business primary keys, the start row identifier and end row identifier need to be converted into their corresponding business primary keys. For the start or end row identifier, a hash value is calculated using the same hash algorithm as in the vertex writing stage, and the target slot in the main index area is located based on the hash value of the start or end row identifier. For example, the slot index is obtained by taking the modulo of the hash value with the fixed capacity of the main index area. Then, based on the offset pointer in the target slot, the corresponding row identifier is read from the variable-length primary key storage area, and it is determined whether the read row identifier is the same as the start or end row identifier.

[0040] If they are the same, it means that the index record stored in the target slot is the index record corresponding to the queried row identifier, and the offset pointer in the target slot is used as the target offset pointer; if they are different, the conflict flag of the target slot is read; if the conflict flag indicates that it has not entered the conflict chain, it means that there are no other candidate records under this slot, and the query is determined to fail; if the conflict flag indicates that it has entered the conflict chain, it backtracks in the conflict chain area based on the predecessor pointer of the target slot, that is, it jumps to the corresponding conflict slot according to the predecessor pointer, reads the row identifier from the variable-length primary key storage area based on the offset pointer in the conflict slot for comparison, if the comparison is inconsistent and the conflict slot still has a predecessor pointer, it continues to backtrack forward along the predecessor pointer until the starting row identifier or the ending row identifier is matched, and the offset pointer of the conflict slot that satisfies the match is used as the target offset pointer. The backtracking order is consistent with the chain building order, the main index slot is the top of the stack, storing the most recent write; the predecessor pointer points to an earlier occupation, and following the predecessor pointer can reach an earlier occupation, until the flag of a certain slot indicates that it will not continue. When querying a fixed primary index slot, the search proceeds along the predecessor. Therefore, records written later are compared first, and records written earlier are compared last. For example, if row identifiers A, B, and C are hashed to the same primary index slot and written in the order A, B, C, then a query for C will hit the primary index slot, a query for B will hit the predecessor once, and a query for A will hit the predecessor once more. The three primary keys are appended to the variable-length primary key storage area in the order they were written, and their offsets remain unchanged after a conflict.

[0041] After obtaining the target offset pointer, the corresponding business primary key is read from the variable-length primary key storage area based on the target offset pointer. This converts the starting and ending row identifiers of the edges into the starting and ending business primary keys, respectively, ultimately generating and exporting an edge export file containing the business primary keys and edge attributes. It should be noted that edge records where the query fails can be marked as abnormal records and added to the exception list for later review, or they can be skipped directly. Simultaneously, after the export is complete, index statistics are output, including conflict rate, peak memory usage, disk usage of each persistent file, insertion and query latency, number of times the conflict chain area has been expanded, and sampling correctness verification results.

[0042] As described above, this application first determines the vertices to be exported based on the export request and reads the segment identifiers attached to these vertices. Then, it creates a corresponding hash index table set for each segment. This set is internally divided into multiple sub-hash tables according to segment routing. Each sub-hash table contains three parts: a main index area, a collision chain area, and a variable-length primary key storage area. During the scanning of the vertices to be exported, the system extracts the row identifier, business primary key, attributes, and graph storage segment information for each vertex. Based on the segment information, it locates the target sub-hash table, stores the business primary key in the variable-length primary key storage area of ​​the sub-hash table, and obtains the offset pointer. Simultaneously, based on the hash value of the row identifier, it determines the target slot in the main index area and writes the index record containing the offset pointer into the slot. If a hash collision occurs, the original slot is not overwritten; instead, a new slot is allocated in the collision chain area, the original record is moved to the collision slot, and then the new record is written, thus ensuring index integrity. After vertex processing is completed, a vertex export file containing the business primary key and attributes is generated and exported. For edge export, based on the start and end row identifiers of each edge obtained from the database scan results, the target offset pointer is retrieved from the corresponding sub-hash table. This pointer is then used to extract the business primary key from the variable-length primary key storage area, ultimately generating an edge export file containing the business primary key and edge attributes, thus achieving the export of the association between vertices and edges. In this way, through segmented routing and a layered design using hash indexes, the data export efficiency of the distributed graph database can be significantly improved. The variable-length primary key storage and conflict chain mechanism reduce memory redundancy and write amplification caused by hash collisions, while ensuring data consistency. Fast index lookup based on row identifiers reduces the primary key lookup complexity during edge record export to constant level, effectively supporting high-concurrency export and complex association scenarios for large-scale data while reducing resource consumption.

[0043] See Figure 2 As shown, in order to achieve a balance between frequency and space occupancy during conflict chain expansion according to the following rules, this embodiment of the invention discloses a data export method for a distributed native graph database, including: Step S21: Determine the collision rate based on the total number of entries written to the sub-hash table and the number of entries with hash collisions, and determine the utilization rate based on the used slots and total capacity of the collision chain area in the sub-hash table.

[0044] In this embodiment, the running status of each sub-hash table can be monitored in real time during the export process. Specifically, the total number of writes and the number of entries with hash collisions in the sub-hash table are counted, and the ratio of the number of entries with hash collisions to the total number of writes is determined as the collision rate; at the same time, the number of used slots in the collision chain area is counted against the total capacity, and the ratio of the number of used slots to the total capacity is determined as the utilization rate. The above monitoring can be performed periodically or triggered once after each preset number of entries are written.

[0045] Step S22: If the conflict rate is greater than a preset high water level threshold, or the utilization rate is greater than a preset emergency threshold, then add storage blocks to the conflict chain area based on a preset step size.

[0046] In this embodiment, if the collision rate is greater than a preset high-water mark threshold and the number of insertions since the last expansion has exceeded the cooling interval, it indicates that hash collisions are currently frequent, and the consumption rate of slots in the collision chain area will significantly accelerate. If the utilization rate is greater than a preset emergency threshold, it indicates that the remaining space in the collision chain area is insufficient. When any of the above conditions are met, storage blocks are added to the collision chain area based on a preset step size, that is, a fixed number of slots corresponding to the preset step size are added to the tail of the collision chain area. The preset step size can be, for example, a fixed number of slots or a fixed proportion of the current capacity of the collision chain area. Among them, the high-water mark threshold is 0.12, the emergency utilization rate threshold is 0.85, the cooling interval can be 50,000 rows, and the preset step size is 0.1.

[0047] In addition, each expansion records an event in the expansion log, with fields including at least the time of expansion, the number of records inserted into the sub-hash table at that time, the conflict rate at that time, the utilization rate of the conflict chain area at that time, the relative capacity of the conflict chain area before expansion, the relative capacity of the conflict chain area after expansion, the graph label, and the segment identifier. This log is used to verify the number of expansions and changes in the conflict rate for each segment after export, avoiding repeated expansions during the cooling-off period and facilitating the analysis of the causes of increased conflicts. Furthermore, when remounting conflict chain area files already written to disk, the log verifies file integrity based on the block count and capacity ratio, preventing the predecessor index from exceeding the mounted range.

[0048] As shown above, the collision chain area adopts append allocation and fixed-length slot encoding. The expansion operation only requires appending storage blocks to the end of the storage file. There is no need to rehash or move existing index records. The expansion cost is extremely low and can be completed online during the export process. This avoids the problems of premature exhaustion of the collision chain area leading to export interruption or excessive space waste caused by skewed distribution of segmented data or increased hash collision rate under the fixed capacity ratio scheme. It can well adapt to the data distribution fluctuations in real-world scenarios.

[0049] Based on the previous embodiment, this application discloses a data export method for distributed native graph databases, which can improve the efficiency of exporting large amounts of data. Next, the data export method for distributed native graph databases will be described in detail, taking the full export of a social graph as an example. The graph contains vertex labels Person and Company, and the relationship type WORKS_AT. The data is stored in sixteen segments. The scale of Person vertices can reach tens of millions to hundreds of millions, and the business primary key is a variable-length string; edge records only carry the row identifiers of the start and end vertices, without carrying the primary key. The export requires generating vertex files and edge files, where the edge file must output the start primary key, end primary key, and edge attributes.

[0050] During export, the graph data is first read, and a sub-hash table resource pool is pre-created according to the label size and the number of segments. The correspondence between the segment identifier and the sub-hash table is registered when vertex data first appears in that segment. Then, Person and Company vertices are scanned: for each exported vertex, the mapping between its row identifier and primary key is written to the sub-hash table corresponding to the segment to which that row identifier belongs, the primary key is appended to the variable-length primary key storage area, and only the offset is recorded in the primary index slot. After scanning of vertices for a certain label is completed, the primary index area, conflict chain area, and variable-length primary key storage area for that label are written to disk. Afterward, only queries are provided, and no further modifications are made.

[0051] Then, WORKS_AT is exported. For each edge, the starting and ending row identifiers are read, and the primary key is queried in the inverse index of the corresponding label. After assembly, the edge file is written without needing to initiate a primary key query to the storage engine. If the vertex distribution on a certain segment is dense, leading to increased hash collisions, only the collision chain area of ​​the sub-hash table for that segment is added to the storage space, while the primary index area and variable-length primary key storage area already written to disk remain unchanged. The above process is the complete application of this invention in actual export tasks.

[0052] See Figure 3 As shown, this embodiment of the invention discloses a data export device for a distributed native graph database, comprising: The index table creation module 11 is used to determine the vertex to be exported from the distributed native graph database according to the data export request, read the vertex to be exported to obtain the corresponding segment identifier, and create a hash index table set according to the segment identifier; the hash index table set includes several sub-hash tables routed according to the segment identifier, and the sub-hash tables include a main index area, a collision chain area and a variable-length primary key storage area. The pointer acquisition module 12 is used to scan the vertex to be exported, extract the corresponding row identifier, business primary key, attribute and graph storage segment information, determine the target sub-hash table from the hash index table set according to the graph storage segment information, write the business primary key into the variable-length primary key storage area of ​​the target sub-hash table, and obtain the corresponding offset pointer. Vertex export module 13 is used to determine the target slot in the main index area of ​​the target sub-hash table according to the hash value of the row identifier, write the index record containing the offset pointer into the target slot, and generate and export the vertex export file based on the business primary key and the attribute. The record writing module 14 is used to allocate a new conflict slot in the conflict chain area if a hash collision occurs when writing to the target slot, move the original index record in the target slot to the conflict slot, and then continue to write the index record corresponding to the vertex to be exported to the target slot. The edge export module 15 is used to scan the distributed native graph database according to the data export request to generate scan results, query the target offset pointer from the corresponding sub-hash table based on the starting row identifier and ending row identifier of the edge in the scan results, and obtain the corresponding business primary key from the variable-length primary key storage area based on the target offset pointer, so as to export an edge export file containing the business primary key and edge attributes.

[0053] As described above, this application first determines the vertices to be exported based on the export request and reads the segment identifiers attached to these vertices. Then, it creates a corresponding hash index table set for each segment. This set is internally divided into multiple sub-hash tables according to segment routing. Each sub-hash table contains three parts: a main index area, a collision chain area, and a variable-length primary key storage area. During the scanning of the vertices to be exported, the system extracts the row identifier, business primary key, attributes, and graph storage segment information for each vertex. Based on the segment information, it locates the target sub-hash table, stores the business primary key in the variable-length primary key storage area of ​​the sub-hash table, and obtains the offset pointer. Simultaneously, based on the hash value of the row identifier, it determines the target slot in the main index area and writes the index record containing the offset pointer into the slot. If a hash collision occurs, the original slot is not overwritten; instead, a new slot is allocated in the collision chain area, the original record is moved to the collision slot, and then the new record is written, thus ensuring index integrity. After vertex processing is completed, a vertex export file containing the business primary key and attributes is generated and exported. For edge export, based on the start and end row identifiers of each edge obtained from the database scan results, the target offset pointer is retrieved from the corresponding sub-hash table. This pointer is then used to extract the business primary key from the variable-length primary key storage area, ultimately generating an edge export file containing the business primary key and edge attributes, thus achieving the export of the association between vertices and edges. In this way, through segmented routing and a layered design using hash indexes, the data export efficiency of the distributed graph database can be significantly improved. The variable-length primary key storage and conflict chain mechanism reduce memory redundancy and write amplification caused by hash collisions, while ensuring data consistency. Fast index lookup based on row identifiers reduces the primary key lookup complexity during edge record export to constant level, effectively supporting high-concurrency export and complex association scenarios for large-scale data while reducing resource consumption.

[0054] In some specific implementations, the step of determining the vertices to be exported from the distributed native graph database based on the data export request, reading the corresponding segment identifiers from the vertices to be exported, and creating a hash index table set based on the segment identifiers may specifically include: The vertex reading unit is used to determine the vertex to be exported according to the data export request, and read the graph data of the vertex to be exported in the distributed native graph database to determine the data size of the vertex to be exported and the number of segments distributed by the vertex to be exported. A set creation unit is used to create a set of hash index tables based on the data size and the number of segments.

[0055] In some specific implementations, determining the target sub-hash table from the hash index table set based on the graph storage segmentation information may specifically include: A mapping table acquisition unit is used to determine a segmented routing mapping table; the segmented routing mapping table registers the correspondence between each segment and a sub-hash table; The mapping table query unit is used to parse the segment information of the graph storage to obtain the segment identifier, and query whether the segment routing mapping table already contains a record corresponding to the segment identifier based on the segment identifier; The sub-hash table allocation unit is used to lock the query operation based on double-checked locking technology if the sub-hash table does not exist, and then query the segmented routing mapping table again. If the query fails to find the target sub-hash table, the unit retrieves an empty sub-hash table from the hash index table set as the target sub-hash table corresponding to the segment identifier and registers the correspondence between the segment identifier and the sub-hash table in the segmented routing mapping table. The target sub-hash table determination unit is used to determine the sub-hash table corresponding to the segment identifier as the target sub-hash table if it exists or is hit after a second query.

[0056] In some specific implementations, determining the target slot in the main index area of ​​the target sub-hash table based on the hash value of the row identifier may specifically include: A hash calculation unit is used to perform hash encryption on the row identifier to determine the corresponding hash value, and to take the hash value modulo the fixed capacity of the main index area, using the modulo result as the slot index. The slot determination unit is used to determine the slot corresponding to the slot index in the main index area as the target slot.

[0057] In some specific implementations, the step of allocating a new conflict slot in the conflict chain region, moving the original index record in the target slot to the conflict slot, and then continuing to write the index record corresponding to the vertex to be exported into the target slot may specifically include: The conflict slot allocation unit is used to allocate a new conflict slot at the end of the conflict chain area, copy the original index record in the target slot to the conflict slot as is, and obtain the index of the conflict slot in the conflict chain area. The record writing unit is used to change the conflict flag of the target slot to "entered the conflict chain", write the offset pointer of the business primary key corresponding to the vertex to be exported into the offset pointer field of the target slot, and write the index of the conflicting slot into the predecessor pointer field of the target slot to generate a predecessor pointer.

[0058] In some specific implementations, the step of scanning the distributed native graph database to generate scan results based on the data export request, and querying the target offset pointer from the corresponding sub-hash table based on the start row identifier and end row identifier of the edges in the scan results, may specifically include: The row identifier determination unit is used to determine the starting row identifier and ending row identifier of the edge based on the data export request. The slot positioning unit is used to locate the target slot in the main index area based on the hash value of the starting row identifier or the ending row identifier; The primary index matching unit is used to read the corresponding row identifier from the variable-length primary key storage area based on the offset pointer in the target slot, and determine whether the read row identifier is the same as the starting row identifier or the ending row identifier; if they are the same, the offset pointer in the target slot is used as the target offset pointer. The conflict chain processing unit is used to read the conflict flag of the target slot if they are not the same; if the conflict flag indicates that the target slot has not entered the conflict chain, the query is determined to be unsuccessful; if the conflict flag indicates that the target slot has entered the conflict chain, the unit backtracks in the conflict chain area based on the predecessor pointer of the target slot until the starting row identifier or the ending row identifier is matched, and the offset pointer of the matching conflict slot is used as the target offset pointer.

[0059] In some specific embodiments, the method further includes: The monitoring unit is used to determine the collision rate based on the total number of writes to the sub-hash table and the number of entries with hash collisions, and to determine the utilization rate based on the used slots and total capacity of the collision chain area in the sub-hash table. An expansion unit is used to add storage blocks to the conflict chain area based on a preset step size if the conflict rate is greater than a preset high water level threshold or the utilization rate is greater than a preset emergency threshold.

[0060] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 The diagram illustrates the structure of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data export method for the distributed native graph database disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0061] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0062] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0063] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the data export method of the distributed native graph database executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0064] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned data export method for a distributed native graph database. The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0065] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0066] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0067] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0068] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0069] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for exporting data from a distributed native graph database, characterized in that, include: Based on the data export request, determine the vertices to be exported from the distributed native graph database, read the vertices to be exported to obtain the corresponding segment identifiers, and create a set of hash index tables based on the segment identifiers; The hash index table set includes several sub-hash tables that are routed according to segment identifiers. Each sub-hash table includes a main index area, a collision chain area, and a variable-length primary key storage area. Scan the vertices to be exported, extract the corresponding row identifier, business primary key, attributes and graph storage segmentation information, determine the target sub-hash table from the hash index table set according to the graph storage segmentation information, write the business primary key into the variable-length primary key storage area of ​​the target sub-hash table, and obtain the corresponding offset pointer; The target slot is determined in the main index area of ​​the target sub-hash table based on the hash value of the row identifier. The index record containing the offset pointer is written into the target slot. A vertex export file is generated and exported based on the business primary key and the attribute. If a hash collision occurs when writing to the target slot, a new collision slot is allocated in the collision chain area, the original index record in the target slot is moved to the collision slot, and then the index record corresponding to the vertex to be exported is written to the target slot. Based on the data export request, the distributed native graph database is scanned to generate scan results. The target offset pointer is queried from the corresponding sub-hash table based on the starting row identifier and ending row identifier of the edge in the scan results. The corresponding business primary key is obtained from the variable-length primary key storage area based on the target offset pointer, so as to export the edge export file containing the business primary key and edge attributes.

2. The data export method for a distributed native graph database according to claim 1, characterized in that, The process of determining the vertices to be exported from the distributed native graph database based on the data export request, reading the corresponding segment identifiers from the vertices to be exported, and creating a hash index table set based on the segment identifiers includes: Based on the data export request, determine the vertex to be exported, read the graph metadata of the vertex to be exported in the distributed native graph database, and determine the data size of the vertex to be exported and the number of segments distributed by the vertex to be exported. A set of hash index tables is created based on the data size and the number of segments.

3. The data export method for a distributed native graph database according to claim 2, characterized in that, The step of determining the target sub-hash table from the hash index table set based on the graph storage segmentation information includes: Determine the segmented routing mapping table; the segmented routing mapping table records the correspondence between each segment and the sub-hash table; Parse the segment information of the graph storage to obtain the segment identifier, and query the segment routing mapping table to see if a record corresponding to the segment identifier already exists; If it does not exist, the query operation is locked based on double-checked locking technology, and the segmented routing mapping table is queried again. If it is not found, an idle sub-hash table is taken from the hash index table set as the target sub-hash table corresponding to the segment identifier, and the correspondence between the segment identifier and the sub-hash table is registered in the segmented routing mapping table. If the hash table exists or is matched after a second query, the sub-hash table corresponding to the segment identifier is determined as the target sub-hash table.

4. The data export method for a distributed native graph database according to claim 1, characterized in that, Determining the target slot in the main index area of ​​the target sub-hash table based on the hash value of the row identifier includes: The row identifier is hashed to determine the corresponding hash value, and the hash value is moduloed by the fixed capacity of the main index area, with the modulo result used as the slot index; The slot corresponding to the slot index in the main index area is determined as the target slot.

5. The data export method for a distributed native graph database according to claim 1, characterized in that, The step of allocating a new conflict slot in the conflict chain region, moving the original index record in the target slot to the conflict slot, and then continuing to write the index record corresponding to the vertex to be exported into the target slot includes: At the end of the conflict chain area, a new conflict slot is assigned, and the original index record in the target slot is copied to the conflict slot as is to obtain the index of the conflict slot in the conflict chain area. Change the conflict flag of the target slot to "entered the conflict chain", write the offset pointer of the business primary key corresponding to the vertex to be exported into the offset pointer field of the target slot, and write the index of the conflicting slot into the predecessor pointer field of the target slot to generate a predecessor pointer.

6. The data export method for a distributed native graph database according to claim 5, characterized in that, The step of generating scan results by exporting the data request to scan the distributed native graph database, and querying the target offset pointer from the corresponding sub-hash table based on the starting row identifier and ending row identifier of the edges in the scan results, includes: The starting row identifier and ending row identifier of the edge are determined based on the data export request. The target slot in the main index area is located based on the hash value of the starting row identifier or the ending row identifier; Based on the offset pointer in the target slot, the corresponding row identifier is read from the variable-length primary key storage area, and it is determined whether the read row identifier is the same as the starting row identifier or the ending row identifier. If they are the same, then the offset pointer in the target slot is taken as the target offset pointer; If they are not the same, then read the conflict flag bit of the target slot; If the conflict flag indicates that the query has not entered the conflict chain, the query is deemed to have failed. If the conflict flag indicates that a conflict chain has been entered, the predecessor pointer based on the target slot backtracks in the conflict chain area until it matches the starting row identifier or the ending row identifier, and the offset pointer of the conflict slot that satisfies the match is used as the target offset pointer.

7. The data export method for a distributed native graph database according to claim 1, characterized in that, Also includes: The collision rate is determined based on the total number of entries written to the sub-hash table and the number of entries with hash collisions, and the utilization rate is determined based on the used slots and total capacity of the collision chain area in the sub-hash table. If the conflict rate is greater than a preset high water level threshold, or the utilization rate is greater than a preset emergency threshold, then an additional storage block is added to the conflict chain area based on a preset step size.

8. A data export device for a distributed native graph database, characterized in that, include: The index table creation module is used to determine the vertices to be exported from the distributed native graph database according to the data export request, read the vertices to be exported to obtain the corresponding segment identifiers, and create a set of hash index tables according to the segment identifiers; the set of hash index tables includes several sub-hash tables routed according to the segment identifiers, and the sub-hash tables include a main index area, a collision chain area, and a variable-length primary key storage area; The pointer acquisition module is used to scan the vertex to be exported, extract the corresponding row identifier, business primary key, attribute and graph storage segment information, determine the target sub-hash table from the hash index table set according to the graph storage segment information, write the business primary key into the variable-length primary key storage area of ​​the target sub-hash table, and obtain the corresponding offset pointer. The vertex export module is used to determine the target slot in the main index area of ​​the target sub-hash table according to the hash value of the row identifier, write the index record containing the offset pointer into the target slot, and generate and export the vertex export file based on the business primary key and the attribute. The record writing module is used to allocate a new conflict slot in the conflict chain area if a hash collision occurs when writing to the target slot, move the original index record in the target slot to the conflict slot, and then continue to write the index record corresponding to the vertex to be exported to the target slot. The edge export module is used to scan the distributed native graph database according to the data export request to generate scan results. Based on the starting row identifier and ending row identifier of the edge in the scan results, the module queries the target offset pointer from the corresponding sub-hash table. Based on the target offset pointer, the module obtains the corresponding business primary key from the variable-length primary key storage area to export an edge export file containing the business primary key and edge attributes.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the data export method for a distributed native graph database as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the data export method for a distributed native graph database as described in any one of claims 1 to 7.

Images