A data management method, device, apparatus and storage medium

Abstract

The application provides a data management method, device and equipment and a storage medium, and can solve the problem of errors when managing data associated with metadata of different formats. The method comprises: obtaining data in at least one field in a data source according to a metadata template to construct metadata, wherein the metadata template records the correspondence between the at least one field in the data source and at least one attribute of the metadata, and the value of the at least one attribute of the metadata is determined according to the data in the at least one field in the data source; and constructing a label in a graph database according to the metadata.

Description

Technical Field

[0001] This application relates to the field of graph databases, and more particularly to a data management method, apparatus, device, and storage medium. Background Technology

[0002] With the advent of the big data era, the volume of data has increased dramatically, and the relationships between data have become more complex. Simply storing all data on a single server is clearly no longer sufficient to meet current needs. To store and manage even larger datasets, distributed graph databases are now commonly used. Distributed graph databases categorize data and store it on different servers, with the main server handling data operation and maintenance. This effectively addresses the challenge of handling massive datasets, and due to their inherent characteristics, graph databases can effectively preserve the relationships between data, ensuring fast retrieval of relational data.

[0003] Furthermore, graph databases primarily rely on metadata to describe the data and relationships within the database, enabling effective management of massive datasets. However, as business activities increase and evolve, the metadata in graph databases also needs continuous addition and updates to adapt to the latest business requirements. Due to the sheer volume of business activities, the metadata in graph databases often originates from different data sources, resulting in inconsistent formats and specifications. This can lead to errors when using inconsistent metadata to manage related data within the graph database. Existing methods of manually organizing metadata are inefficient and prone to secondary errors; therefore, an effective way to manage metadata in graph databases is currently lacking. Summary of the Invention

[0004] This application provides a data management method, apparatus, device, and storage medium to avoid errors that occur when managing related data using inconsistently formatted metadata, thus affecting the performance of the graph database.

[0005] Firstly, this application provides a data management method, which includes:

[0006] Based on the metadata template, data under at least one field in the data source is obtained to construct metadata, wherein the metadata template records the correspondence between the at least one field in the data source and at least one attribute of the metadata, and the value of at least one attribute of the metadata is determined based on the data under the at least one field in the data source;

[0007] Based on the metadata, construct the labels in the graph database.

[0008] Furthermore, the metadata template includes: a template for point metadata and a template for edge metadata;

[0009] If the metadata includes point metadata constructed based on a template of the point metadata, then constructing the tags in the graph database based on the metadata includes:

[0010] The point labels in the graph database are constructed based on the point metadata;

[0011] If the metadata includes edge metadata constructed based on the template of the edge metadata, then constructing the tags in the graph database based on the metadata includes:

[0012] The edge labels in the graph database are constructed based on the edge metadata.

[0013] Furthermore, the attributes of the point metadata in the template include: the name of the point label and the lifecycle of the point label, wherein the point labels in the graph database constructed based on the point metadata include the name and lifecycle of the point label;

[0014] The attributes of the edge metadata in the template include: the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left of the edge label, and the name of the vertex label associated to the right of the edge label. The edge labels in the graph database constructed based on the edge metadata include the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left, and the name of the vertex label associated to the right.

[0015] Furthermore, the attributes of the point metadata in the template further include: the name of the point label, the name of the attribute associated with the point label, the data type of the attribute, and the cardinality type of the attribute; after constructing the point labels in the graph database based on the point metadata, the method further includes:

[0016] Construct the attributes associated with the point label based on the point metadata;

[0017] The edge metadata template further includes the following attributes: the name of the edge label, the name of the attribute associated with the edge label, the data type of the attribute, and the cardinality type of the attribute; after constructing the edge labels in the graph database based on the metadata, the method further includes:

[0018] The attributes associated with the edge label are constructed based on the edge metadata.

[0019] Furthermore, the attributes of the point metadata and the edge metadata in the templates of the point metadata and the edge metadata also include: first indication information on whether to create a composite index and / or whether to create a hybrid index;

[0020] After constructing the attributes associated with the point label based on the point metadata, the method further includes:

[0021] Create a combined index and / or a mixed index for the attributes associated with the point label based on the first indication information in the point metadata;

[0022] After constructing the attributes associated with the edge label based on the edge metadata, the method further includes:

[0023] Create a combined index and / or a hybrid index for the attributes associated with the edge label based on the first indication information in the edge metadata.

[0024] Furthermore, the attributes of the edge metadata in the template of the edge metadata also include: second indication information on whether to create a composite index and / or whether to create a frequency index;

[0025] After constructing the attributes associated with the edge label based on the edge metadata, the method further includes:

[0026] Based on the second indication information in the edge metadata, create a combined index and / or a frequency index for the edge label.

[0027] Furthermore, the method also includes:

[0028] Based on the lifecycle of the point labels, remove point labels from the graph database that have exceeded their lifecycle;

[0029] Based on the lifecycle of the edge label, remove edge labels that have exceeded their lifecycle from the graph database.

[0030] Secondly, this application provides a data management device, the device comprising:

[0031] The acquisition module is used to acquire data under at least one field in the data source to construct metadata. The metadata template records the correspondence between the at least one field in the data source and at least one attribute of the metadata. The value of the at least one attribute of the metadata is determined based on the data under the at least one field in the data source.

[0032] The processing module is used to construct tags in the graph database based on the metadata.

[0033] Furthermore, the metadata template includes: a template for vertex metadata and a template for edge metadata; if the metadata includes vertex metadata constructed based on the template for vertex metadata, the processing module constructs tags in the graph database based on the metadata, specifically: constructing vertex tags in the graph database based on the vertex metadata; if the metadata includes edge metadata constructed based on the template for edge metadata, the processing module constructs tags in the graph database based on the metadata, specifically: constructing edge tags in the graph database based on the edge metadata.

[0034] Furthermore, the attributes of the point metadata in the template include: the name of the point label and the lifecycle of the point label, wherein the point labels in the graph database constructed by the processing module based on the point metadata include the name and lifecycle of the point label;

[0035] The attributes of the edge metadata in the template include: the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left of the edge label, and the name of the vertex label associated to the right of the edge label. The edge labels in the graph database constructed by the processing module based on the edge metadata include the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left, and the name of the vertex label associated to the right.

[0036] Furthermore, the attributes of the point metadata in the template also include: the name of the point label, the name of the attribute associated with the point label, the data type of the attribute, and the cardinality type of the attribute; after the processing module constructs the point labels in the graph database based on the point metadata, it is also used to construct the attributes associated with the point labels based on the point metadata;

[0037] The edge metadata template also includes the following attributes: the name of the edge label, the name of the attribute associated with the edge label, the data type of the attribute, and the cardinality type of the attribute. After the processing module constructs the edge label in the graph database based on the edge metadata, it is also used to construct the attribute associated with the edge label based on the edge metadata.

[0038] Furthermore, the attributes of the point metadata and the edge metadata in the templates of the point metadata and the edge metadata also include: first indication information on whether to create a composite index and / or whether to create a hybrid index;

[0039] After the processing module constructs the attributes associated with the point label based on the point metadata, it is further configured to create a combined index and / or a mixed index for the attributes associated with the point label based on the first indication information in the point metadata.

[0040] After constructing the attributes associated with the edge label based on the edge metadata, the processing module is further configured to create a combined index and / or a mixed index for the attributes associated with the edge label based on the first indication information in the edge metadata.

[0041] Furthermore, the attributes of the edge metadata in the template of the edge metadata also include: second indication information on whether to create a composite index and / or whether to create a frequency index;

[0042] After constructing the attributes associated with the edge tag based on the edge metadata, the processing module is further configured to create a joint index and / or frequency index for the edge tag based on the second indication information in the metadata.

[0043] Furthermore, the processing module is also used for:

[0044] Based on the lifecycle of the point labels, remove point labels that have exceeded their lifecycle from the graph database; and based on the lifecycle of the edge labels, remove edge labels that have exceeded their lifecycle from the graph database.

[0045] Thirdly, this application provides an electronic device, which includes at least a processor and a memory, wherein when the processor executes a computer program or instructions stored in the memory, it implements the method described in the first aspect.

[0046] Fourthly, this application provides a computer-readable storage medium storing a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0047] In this application, data under at least one field in the data source is obtained from a metadata template that records the correspondence between at least one field in the data source and at least one attribute of the metadata to construct metadata. The value of at least one attribute of the metadata is determined based on the data under at least one field in the data source. This enables the construction and management of point labels and edge labels in the graph database using metadata with a unified format, avoiding errors that occur when using metadata with inconsistent formats to manage related data in the graph database. At the same time, using metadata with a unified format to efficiently construct and manage data in the graph database can improve the performance of the graph database. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart of a data management method provided in this application.

[0050] Figure 2 This is a template example diagram of a possible point and edge metadata provided in this application.

[0051] Figure 3 This is a template example diagram of another possible point and edge metadata provided in this application.

[0052] Figure 4 This is a schematic diagram illustrating the association information of a car theft incident with a side tag provided in this application.

[0053] Figure 5 This is a schematic diagram illustrating a possible construction process for point labels and edge labels in a graph database provided in this application.

[0054] Figure 6 A flowchart illustrating one possible way to add a new associated attribute as provided in this application.

[0055] Figure 7 This application provides one possible method for constructing a graph database.

[0056] Figure 8 This is a schematic diagram of the structure of a data management device provided in this application.

[0057] Figure 9 This is a schematic diagram of an electronic device structure provided in this application. Detailed Implementation

[0058] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0059] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0060] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or textual entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0061] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0062] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0063] Graph databases primarily consist of vertex labels, edge labels, and attributes. Vertex labels, also known as points or vertices in a graph database, are similar to entities and concepts in a knowledge graph, representing concrete things like people and cars, as well as concepts like theft and fraud cases. Edge labels, also known as edges in a graph database, are similar to relationships in a knowledge graph, representing the connections between each vertex label, such as someone stealing a car or someone visiting a certain place. Attributes correspond to the characteristics and features associated with vertex or edge labels. For example, attributes associated with a person might include age, gender, father, and school attended, while attributes associated with someone stealing a car might include the location of the theft and the theft event.

[0064] To effectively manage data in graph databases, metadata is typically used to describe the data and relationships between it. This involves using metadata to construct and manage vertex labels, edge labels, and attributes. However, as business needs increase and evolve, the metadata in the graph database also needs to be updated to meet the latest requirements. Due to the sheer volume of business data, the metadata often originates from different data sources, leading to inconsistencies in format and specifications. This can cause errors when using inconsistent metadata to manage related data in the graph database. For example, in data source 1, the metadata used to construct vertex label A includes an English name, description, and lifecycle. After an electronic device creates vertex label A based on the metadata from data source 1, users cannot query vertex label A using its corresponding Chinese name and description. In data source 2, the metadata used to construct vertex label A includes a Chinese name, description, and data type. After an electronic device creates vertex label A using the metadata from data source 2, it cannot automatically clean up vertex label A according to its lifecycle, and the constructed vertex label A may contain other unnecessary restrictions such as data type limitations. This impacts the performance of the graph database and the user experience. Based on this, this application provides a data management method and apparatus to effectively manage the metadata of a graph database and ensure the performance of the graph database.

[0065] Figure 1This is a flowchart of a data management method that can be applied to an electronic device. The method includes:

[0066] S101: Based on the metadata template, obtain data from at least one field in the data source to construct metadata, wherein the metadata template records the correspondence between the at least one field in the data source and at least one attribute of the metadata, and the value of the at least one attribute of the metadata is determined based on the data from the at least one field in the data source.

[0067] Metadata templates can be a metadata construction format or a metadata construction specification. Specifically, since graph databases mainly contain vertex labels and edge labels, and in this application, the metadata used to construct vertex labels is called vertex metadata, and the metadata used to construct edge labels is called edge metadata, the metadata template can include templates for vertex metadata and templates for edge metadata. The vertex metadata template records the correspondence between at least one field in the data source and at least one attribute of the metadata, which can include a first correspondence between at least one first field in the data source and at least one attribute of the vertex metadata. The edge metadata template records a second correspondence between at least one second field in the data source and at least one attribute of the edge metadata. By pre-importing the metadata templates into an electronic device, the electronic device can retrieve data under the first field recorded in the vertex metadata template from the data source, and match the retrieved data with each attribute of the vertex metadata according to the first correspondence between each first field and each attribute of the vertex metadata. Similarly, the electronic device can retrieve data under the second field recorded in the edge metadata template from the data source, and match the retrieved data with each attribute of the edge metadata according to the second correspondence between each second field and each attribute of the edge metadata. The data source can be an Excel file, JSON file, or other file containing graph database metadata information, constructed according to the first and second correspondences recorded in the metadata template. This Excel file, JSON file, or other file containing graph database metadata information can contain at least one first field and / or at least one second field, and may also contain other fields besides the first and second fields. The data source can also be online resources or legacy graph databases. Electronic devices can retrieve data from the internet or legacy graph databases that corresponds to the fields recorded in the metadata template to construct the metadata.

[0068] In one possible implementation, this application provides a possible metadata template, wherein the attributes of the point metadata in the template include: the name of the point label and the lifecycle of the point label, wherein the point labels in the graph database constructed based on the point metadata include the name and lifecycle of the point label;

[0069] The attributes of the edge metadata in the template include: the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left of the edge label, and the name of the vertex label associated to the right of the edge label. The edge labels in the graph database constructed based on the edge metadata include the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left, and the name of the vertex label associated to the right.

[0070] by Figure 2 For example, Figure 2 This is a template example diagram of a possible point and edge metadata provided for an embodiment of this application. For example... Figure 2 As shown, where Figure 2 The top image shows the node metadata template, and the bottom image shows the edge metadata template. The first correspondence of the node metadata template record can include the first field NODE.CN corresponding to the Chinese description of the node label, the first field NODE.EN corresponding to the English name of the node label, and the first field TTL (D / W / M) corresponding to the lifecycle of the node label. Similarly, the second correspondence of the edge metadata template record can include the second field RELATIONSHIP.CN corresponding to the Chinese description of the edge label, the second field RELATIONSHIP.EN corresponding to the English name of the edge label, the second field TTL (D / W / M) corresponding to the lifecycle of the edge label, the second field LEFTNODE.CN corresponding to the Chinese description of the node label associated to the left of the edge label, the second field RIGHTNODE.CN corresponding to the English name of the node label associated to the left of the edge label, and the second field LEFTNODE.EN corresponding to the Chinese description of the node label associated to the right of the edge label, and the second field RIGHTNODE.EN corresponding to the English name of the node label associated to the right of the edge label. The English names of the tags are used to construct vertex or edge labels in the graph database, while the Chinese descriptions describe the actual meaning of the tags for easy user viewing. Additionally, the tag lifecycle manages the timeliness of the corresponding metadata, effectively cleans up data, reduces storage pressure on electronic devices, and improves the performance of the graph database.

[0071] Taking an Excel file as an example, the electronic device only retrieves data from the same fields as the first or second field recorded in the metadata template. The retrieved dot tag metadata can include data under the dot tag's Chinese description (NODE.CN), English name (NODE.EN), and time-to-live (TTL), which are then organized into the final metadata. For example, Table 1 shows the dot tag metadata obtained by organizing data from the data source according to the metadata template. As shown in Table 1, the retrieved dot tags include people, vehicles, theft cases, and fraud cases. The English description for a person is "person," with a time-to-live of 3 months; the English description for a vehicle is "vehicle," with a time-to-live of 3 months; the English description for a theft case is "stealCase," with a time-to-live of 6 months; and the English description for a fraud case is "defraudCase," with a time-to-live of 6 months.

[0072] Table 1: Example of metadata for dot labels

[0073]

[0074] If the data source is a JSON file, the same electronic device can retrieve the data under the NODE.CN, NODE.EN, and TTL fields in the JSON file to obtain the metadata used to construct the point labels. The method for obtaining the metadata of the edge labels can be the same as that for obtaining the metadata of the point labels. In addition, metadata information describing the attributes associated with the points can also be included in the point metadata template, and metadata information describing the attributes associated with the edges can also be included in the edge metadata template.

[0075] It should be understood that this application does not limit a single attribute of point metadata or edge metadata to a single field. For example, the field corresponding to the Chinese description of a point label can be NODE.CN, or Vertex.CN, or other fields used to represent the Chinese description of a point label.

[0076] Alternatively, the electronic device can be a server, computer, or other electronic device with certain computing power and the ability to manage the metadata of the graph database.

[0077] S102: Based on the metadata, construct the labels in the graph database.

[0078] If the metadata includes point metadata constructed from a template of point metadata, then point labels in the graph database are constructed based on the point metadata.

[0079] Once the point metadata is retrieved from the data source based on the point metadata template, since the template includes at least the English names of the point labels, corresponding point labels can be constructed in the graph database based on these English names. For example, if the retrieved point metadata includes the English names of three different point labels: person, vehicle, and stealCase, the electronic device can create three different point labels—person, vehicle, and stealCase—in the graph database to represent three different entities: a person, a vehicle, and a theft case.

[0080] If the metadata includes edge metadata constructed from a template of edge metadata, then the edge labels in the graph database are constructed based on the edge metadata.

[0081] Similarly, after obtaining edge metadata from the data source based on the edge metadata template, since the template includes at least the English name of the edge label, edge labels can be constructed in the graph database based on the English name of the edge label. The edge label can represent the relationship between any two entities; for example, a car theft event (hasPersonStealVehicleEvent) can represent the relationship between two entities (i.e., point labels) – a person and a car.

[0082] In this application, data under at least one field in the data source is obtained from a metadata template that records the correspondence between at least one field in the data source and at least one attribute of the metadata to construct metadata. The value of at least one attribute of the metadata is determined based on the data under at least one field in the data source. This enables the construction and management of the graph database's metadata in a unified format, avoiding errors that occur when using inconsistently formatted metadata to manage related data in the graph database. At the same time, using consistently formatted metadata enables the efficient construction and management of data in the graph database, thereby improving the performance of the graph database.

[0083] Furthermore, based on the above embodiments, the attributes of the point metadata in the template of the point metadata also include: the name of the point label, the name of the attribute associated with the point label, the data type of the attribute, and the cardinality type of the attribute;

[0084] After constructing the point labels in the graph database based on the point metadata, the method further includes constructing the attributes associated with the point labels based on the point metadata.

[0085] The attributes of the edge metadata in the template also include: the name of the edge label, the name of the attribute associated with the edge label, the data type of the attribute, and the cardinality type of the attribute;

[0086] After constructing the edge labels in the graph database based on the edge metadata, the method further includes constructing the attributes associated with the edge labels based on the edge metadata.

[0087] In this graph database, vertex or edge labels can be associated with zero or any integer number of attributes. When a vertex or edge label is associated with zero attributes, it indicates that the label only serves as an identifier. When a vertex or edge label is associated with at least one attribute, each associated attribute corresponds to an attribute data type and an attribute cardinality type. Attribute data types can include String, Integer, Long, Single, and Double. Attribute cardinality types can be Single, Set, or List. Single indicates that the current attribute can only have one value, such as a person's ID number, gender, or age. Set indicates that the current attribute can have multiple distinct values, such as the names of schools a person has attended. List indicates that the current attribute can have multiple repeatable values.

[0088] Furthermore, in order to achieve efficient search of data for point and edge label management, the attributes of point metadata and edge metadata in the metadata template also include: first indication information on whether to create a composite index and / or whether to create a hybrid index;

[0089] After constructing the attribute associated with the point label based on the point metadata, the method further includes creating a combined index and / or a mixed index for the attribute associated with the point label based on the first indication information in the aforementioned metadata.

[0090] After constructing the attributes associated with the edge label based on the edge metadata, the method further includes creating a combined index and / or a mixed index for the attributes associated with the edge label based on the first indication information in the edge metadata.

[0091] The attributes associated with a point label can be some characteristic information possessed by the entity represented by the point label. The characteristic information can be values, strings, identifiers, etc. For example, the attributes associated with a point label representing a vehicle can include license plate number, vehicle brand, vehicle color, and vehicle pictures, etc. Similarly, the attributes associated with an edge label can also be some characteristic information possessed by the relationship represented by the edge label. For example, the attributes associated with an edge label representing a car theft event can include the time of car theft, the location of car theft, and the nature of the event, etc. In order to quickly find the corresponding point labels and edge labels, corresponding composite indexes and / or hybrid indexes can be constructed for the point labels and edge labels. The composite index can define one or more key attributes in the attributes associated with the point label or edge label. For example, a composite index is created for the age and name associated with the point label representing a person. The electronic device will store the composite index of age and name and the corresponding person in a composite index list. When using age and / or name for an equality search in the query statement, the electronic device can quickly find the person that meets the query statement from the composite index list. Opposite to the equality search of the composite index, a hybrid index can also be created for the attributes associated with the point label or edge label. The hybrid index can perform range searches, fuzzy searches, etc. based on the composite index. For example, a hybrid index is created for the license plate number associated with the edge label representing a vehicle. When querying "the owner information of vehicles with license plate numbers starting with Zhe A123", "all vehicles with license plate numbers containing Zhe X45", etc., which contain partial determined license plate number information, the electronic device can efficiently find the vehicles that meet the query conditions and their associated attribute information according to the constructed hybrid index. In addition, creating a hybrid index requires the help of external search engines such as Elasticsearch, Solr, and Lucene, etc. That is, the attribute data with the hybrid index constructed needs to be written into the external search engine, and the external search engine implements the hybrid index search.

[0092] Optionally, in order to further improve the retrieval efficiency, the attributes of the edge metadata in the template of the edge metadata further include: second indication information on whether to create a combined index and / or whether to create a frequency index;

[0093] After constructing the attributes associated with the edge label according to the edge metadata and constructing the attributes associated with the point label according to the point metadata, it further includes creating a combined index and / or a frequency index for the edge label according to the second indication information in the metadata.

[0094] Both composite indexes and frequency indexes require the assistance of external search engines. Unlike hybrid indexes, which focus only on the attributes associated with point labels, composite indexes use the attributes associated with edge labels and the attributes associated with the point labels associated with those edge labels as key index attributes. Taking a car theft event with a composite index as an example, the edge label "car theft event" is associated with the point labels "people" and "stolen car." Using the attributes associated with "people," "stolen car," and "car theft event" as query statements, such as "a male car theft incident that occurred by the river" or "a man around 30 years old stole a car at 2:00 AM," the electronic device can quickly retrieve the edge labels of the car theft events related to the query statement, as well as the attribute data associated with those edge labels. Similarly, frequency indexes are implemented by storing the frequency of occurrences of the relationships represented by edge labels in a frequency index statistics table. This frequency index statistics table allows for the quick identification of edge labels that match the query frequency, as well as the point labels associated with those edge labels.

[0095] Figure 3 This is an example template diagram of another possible point and edge metadata provided for an embodiment of this application. For example... Figure 3 As shown, where Figure 3The top image shows the node metadata template, and the bottom image shows the edge metadata template. The node metadata template includes the following attributes: the Chinese description and English name of the node label; the Chinese description of the attribute associated with the node label; the English name of the attribute associated with the node label; the attribute data type; the attribute cardinality type; and first indication information regarding whether to create a composite index and / or a mixed index. The first fields corresponding to these attributes in the node metadata template are NODE.CN, NODE.EN, PROPERTYKEY.CN, PROPERTYKEY.EN, PROPERTYKEY.DATATYPE, PROPERTYKEY.CARDINALITY, ISCOMPOSITEINDEX, and ISMIXEDINDEX, respectively. Correspondingly, the attributes of the edge metadata in the template include the Chinese description and English name of the edge label, the Chinese description and English name of the point label associated to the left of the edge label, the Chinese description and English name of the point label associated to the right of the edge label, the Chinese description and English name of the attribute associated with the edge label, the attribute data type, the attribute cardinality type, and first indication information on whether to create a composite index and / or a mixed index, second indication information on whether to create a joint index and / or a frequency index, and third indication information on whether it is a sorting key for a point center index. The second fields recorded in the edge metadata template corresponding to the above attributes in sequence are RELATIONSHIP.CN, RELATIONSHIP.EN, TTL, ISUNITEINDEX, ISFREQUENCYINDEX, PROPERTKEY.CN, PROPERTKEY.EN, PROPERTKEY.DATATYPE, ISCOMPOSITEINDEX, ISMI XEDINDEX,ISVERTEXCENTRICINDSEXSORTKEY,LEFTNODE.CN,RIGHTNODE.CN,LEFTNODE.EN,RIGHTNODE.EN.

[0096] Taking Excel as the data source, the metadata of the attributes associated with the point labels and the metadata of the attributes associated with the edge labels are shown in Tables 2 and 3 below, respectively, based on the metadata template.

[0097] Table 2: Metadata Examples of Attributes Associated with Dot Labels

[0098]

[0099] Table 3: Metadata Examples of Edge Labels and Associated Attributes

[0100]

[0101] The electronic device can retrieve at least the Chinese descriptions and English names of four point tags—person, vehicle, theft case, and fraud case—from Table 1 based on the point metadata template, as well as the lifecycle of each point tag. Additionally, the electronic device can retrieve at least the attributes associated with the point tag "vehicle" from Table 2, including license plate number, energy type, vehicle type, vehicle brand, vehicle model, vehicle color, and vehicle tag. It can also retrieve the Chinese descriptions and English names, data type, and data cardinality for each attribute. Furthermore, the electronic device can retrieve at least the attributes associated with the edge tag "car theft event" from Table 3, including relation start address, relation start time, relative longitude, relative latitude, relation type, and relation geohash. It has also created composite indexes and frequency indexes for the edge tag "car theft event" and composite indexes for the relation start address associated with the edge tag. Taking the car theft event in Table 3 as an example... Figure 4 This application provides a schematic diagram illustrating the association of car theft incident information with side tags. For example... Figure 4 As shown, attributes associated with the point tag "person" such as name, age, and height, and attributes associated with the point tag "vehicle" such as license plate number and vehicle color, are all associated with the side tag "vehicle event". Furthermore, the attributes associated with the side tag "vehicle event" can include theft location (relationStartAddress) and theft time (relationStarTime), enabling electronic devices to find the associated tags and attribute information based on any attribute of any tag.

[0102] Taking the acquisition of metadata from an Excel file and the construction of point labels and their attributes, as well as edge labels and their attributes, into a graph database as an example... Figure 5 This application provides a schematic diagram illustrating a possible construction process for point and edge labels in a graph database. For example... Figure 5 As shown:

[0103] S501: Verify Excel file.

[0104] The electronic device verifies whether the Excel file contains the first field of the point metadata template or the second field of the edge metadata template, and verifies whether the data under the first or second field in the Excel file is empty or whether the data type is correct. For example, the data under the Chinese description or English name of the point label should be a string, and the data under the lifecycle should be a number.

[0105] S502: Retrieve metadata from an Excel file based on the metadata template.

[0106] The electronic device searches for and retrieves data in the Excel file that is identical to the first or second field recorded in the metadata template, based on the metadata template. The electronic device can obtain the Chinese description and English name of point labels, edge labels, and attributes, the association between point labels and attributes, the association between edge labels and attributes, and the association between point labels and edge labels. It can also read the lifecycle of point and edge labels, as well as the command information for whether to create composite indexes, mixed indexes, joint indexes, frequency indexes, and other metadata attributes.

[0107] S503: Create point labels, edge labels, and attributes.

[0108] Electronic devices can construct corresponding point labels, edge labels, and attributes in a graph database based on the point labels, edge labels, and attribute names read from an Excel file.

[0109] S504: In the graph database, add relationships to point labels, edge labels, and attributes.

[0110] Based on the relationships between point labels and attributes, edge labels and attributes, and point labels and edge labels read from the metadata, the electronic device adds corresponding relationships to the point labels, edge labels, and attributes constructed in step S503, thus completing the basic construction of the graph database. Of course, it can also add lifecycles to point labels, edge labels, and attributes, and create indexes, based on metadata such as the lifecycle of point and edge labels, and command information regarding whether to create composite indexes, mixed indexes, joint indexes, or frequency indexes.

[0111] Based on the above embodiments, in this embodiment of the application, the method further includes:

[0112] Based on the lifecycle of the point labels, remove point labels from the graph database that have exceeded their lifecycle;

[0113] Based on the lifecycle of the edge label, remove edge labels that have exceeded their lifecycle from the graph database.

[0114] Specifically, the electronic device writes the lifecycles of point and edge labels from the metadata into a configuration file (configfile). A script, set according to the configuration file, is then executed via shell command to clear point or edge labels in the graph database that have exceeded their lifecycle. The lifecycle can be calculated when a point or edge label is created, or it can be calculated based on a specified start time entered by the user.

[0115] To illustrate the process when adding attributes to point labels or edge labels in a graph database, Figure 6 For example, Figure 6 This application provides a flowchart illustrating a possible method for adding new associated attributes. For example... Figure 6 As shown, after obtaining the metadata of the new attribute, the electronic device first iterates through the existing attributes in the graph database to determine if the new attribute already exists. If no attribute exists that matches the new attribute, the new attribute is constructed in the graph database and associated with the corresponding vertex label or edge label. If the new attribute is associated with vertex label A, the composite index under all edge labels associated with vertex label A needs to be updated, followed by the combined index and mixed index associated with vertex label A. If the new attribute is associated with edge label B, the combined index and mixed index associated with edge label B are updated directly. Optionally, the index update task is executed in the background, and the front end displays the status of the index update as complete, allowing the user to continue using the front end during the index update process.

[0116] Figure 7 This application provides a possible method for constructing a graph database. For example... Figure 7As shown, the electronic device receives an Excel or JSON file designed according to the metadata template. The Excel or JSON file includes at least one first field and / or at least one second field, as well as the attributes of the point metadata corresponding to each first field and the attributes of the edge metadata corresponding to each second field. The electronic device then verifies the header information of the Excel or JSON file, specifically whether the file contains the first or second field recorded in the metadata template, and whether the data under the first and second fields is empty or of a preset data type. Next, it retrieves the point metadata and edge metadata from the Excel or JSON file that conform to the metadata template. This point metadata and edge metadata contain metadata information used to construct point labels, edge labels, and attributes in the graph database. It also includes metadata information such as the association between point labels and attributes, the association between edge labels and attributes, the association between point labels and edge labels, the lifecycle of points and edge labels, and command information for creating composite indexes, mixed indexes, joint indexes, and frequency indexes. If the metadata obtained above is used to create a new graph database, then vertex labels, edge labels, and attributes are constructed sequentially based on the metadata. Associations between vertex labels and attributes, edge labels and attributes, and vertex labels and edge labels are added. Then, composite indexes, mixed indexes, and joint indexes are created for the attributes associated with vertex labels, edge labels, and their associated attributes. Finally, corresponding lifecycles are configured for vertex labels and edge labels to periodically clear data and improve the performance of the graph database. If the metadata obtained above is used to update the graph database, then the existing vertex labels, edge labels, and attributes in the graph database need to be traversed to determine if the current metadata has already been used to construct corresponding vertex labels and edge labels in the graph database. If it is an existing vertex label or edge label, the newly added attribute is associated with the existing vertex label or edge label, and the corresponding index is updated. If it is a vertex label or edge label that has not yet been constructed in the graph database, the process is the same as when constructing a new graph database: vertex labels, edge labels, and attributes are created in the graph database, and associations are added, indexes are created, and lifecycles are configured.

[0117] Based on the above data management method, this application provides a data management device. Figure 8 This is a schematic diagram of a data management device provided in an embodiment of this application. The device includes:

[0118] The acquisition module 801 is used to acquire data under at least one field in the data source according to the metadata template for constructing metadata, wherein the metadata template records the correspondence between the at least one field in the data source and at least one attribute of the metadata, and the value of the at least one attribute of the metadata is determined according to the data under the at least one field in the data source;

[0119] Processing module 802 is used to construct tags in the graph database based on the metadata.

[0120] Furthermore, the metadata template includes: a template for vertex metadata and a template for edge metadata; if the metadata includes vertex metadata constructed based on the template for vertex metadata, the processing module 802 constructs tags in the graph database based on the metadata, specifically: constructing vertex tags in the graph database based on the vertex metadata; if the metadata includes edge metadata constructed based on the template for edge metadata, the processing module 802 constructs tags in the graph database based on the metadata, specifically: constructing edge tags in the graph database based on the edge metadata.

[0121] Furthermore, the attributes of the point metadata in the template include: the name of the point label and the lifecycle of the point label, wherein the point labels in the graph database constructed by the processing module 802 based on the point metadata include the name and lifecycle of the point label;

[0122] The attributes of the edge metadata in the template include: the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left of the edge label, and the name of the vertex label associated to the right of the edge label. The edge labels in the graph database constructed by the processing module 802 based on the edge metadata include the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left, and the name of the vertex label associated to the right.

[0123] Furthermore, the attributes of the point metadata in the template of the point metadata also include: the name of the point label, the name of the attribute associated with the point label, the data type of the attribute, and the cardinality type of the attribute; after the processing module 802 constructs the point label in the graph database based on the point metadata, it is also used to construct the attribute associated with the point label based on the point metadata.

[0124] The attributes of the edge metadata in the template also include: the name of the edge label, the name of the attribute associated with the edge label, the data type of the attribute, and the cardinality type of the attribute; after the processing module 802 constructs the edge label in the graph database based on the edge metadata, it is also used to construct the attribute associated with the edge label based on the edge metadata.

[0125] Furthermore, the attributes of the point metadata and the edge metadata in the templates of the point metadata and the edge metadata also include: first indication information on whether to create a composite index and / or whether to create a hybrid index;

[0126] After the processing module 802 constructs the attributes associated with the point label based on the point metadata, it is further configured to create a combined index and / or a mixed index for the attributes associated with the point label based on the first indication information in the point metadata.

[0127] After the processing module 802 constructs the attributes associated with the edge label based on the edge metadata, it is further configured to create a combined index and / or a mixed index for the attributes associated with the edge label based on the first indication information in the edge metadata.

[0128] Furthermore, the attributes of the edge metadata in the template of the edge metadata also include: second indication information on whether to create a composite index and / or whether to create a frequency index;

[0129] After the processing module 802 constructs the attributes associated with the edge tag based on the edge metadata, it is further configured to create a joint index and / or frequency index for the edge tag based on the second indication information in the metadata.

[0130] Furthermore, the processing module 802 is also used for:

[0131] Based on the lifecycle of the point labels, remove point labels that have exceeded their lifecycle from the graph database; and based on the lifecycle of the edge labels, remove edge labels that have exceeded their lifecycle from the graph database.

[0132] Figure 9 This is a schematic diagram of an electronic device structure. (Example) Figure 9 As shown, the electronic device includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0133] The memory 903 stores a computer program, which, when executed by the processor 901, causes the processor 901 to implement the steps of any of the data management methods described above.

[0134] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0135] The communication interface 902 is used for communication between the above-mentioned electronic device and other devices.

[0136] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0137] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0138] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, the electronic device executes the steps of any of the above-described data management methods.

[0139] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0141] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0142] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0147] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data management method, characterized in that, The method includes: Based on the first field recorded in the template of the point metadata, the data under the first field is obtained from the data source, and the obtained data is matched one by one with the attributes of the point metadata according to the first correspondence between each first field and each attribute of the point metadata; based on the second field recorded in the template of the edge metadata, the data under the second field is obtained from the data source, and the obtained data is matched one by one with the attributes of the edge metadata according to the second correspondence between each second field and each attribute of the edge metadata. Based on the vertex metadata and the edge metadata, construct the labels in the graph database.

2. The method according to claim 1, characterized in that, The attributes of the point metadata in the template include: the name of the point label and the lifecycle of the point label, wherein the point labels in the graph database constructed based on the point metadata include the name and lifecycle of the point label; The attributes of the edge metadata in the template include: the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left of the edge label, and the name of the vertex label associated to the right of the edge label. The edge labels in the graph database constructed based on the edge metadata include the name of the edge label, the lifespan of the edge label, the name of the vertex label associated to the left, and the name of the vertex label associated to the right.

3. The method according to claim 2, characterized in that, The point metadata template further includes the following attributes: the name of the point label, the name of the attribute associated with the point label, the data type of the attribute, and the cardinality type of the attribute; after constructing the point labels in the graph database based on the point metadata, the method further includes: Construct the attributes associated with the point label based on the point metadata; The edge metadata template further includes the following attributes: the name of the edge label, the name of the attribute associated with the edge label, the data type of the attribute, and the cardinality type of the attribute; after constructing the edge labels in the graph database based on the edge metadata, the method further includes: The attributes associated with the edge label are constructed based on the edge metadata.

4. The method according to claim 3, characterized in that, The attributes of the point metadata and the edge metadata in the templates of the point metadata and the edge metadata also include: first indication information on whether to create a composite index and / or whether to create a hybrid index; After constructing the attributes associated with the point label based on the point metadata, the method further includes: Create a combined index and / or a mixed index for the attributes associated with the point label based on the first indication information in the point metadata; After constructing the attributes associated with the edge label based on the edge metadata, the method further includes: Create a combined index and / or a hybrid index for the attributes associated with the edge label based on the first indication information in the edge metadata.

5. The method according to claim 3, characterized in that, The attributes of the edge metadata in the template also include: second indication information on whether to create a composite index and / or whether to create a frequency index; After constructing the attributes associated with the edge label based on the edge metadata, the method further includes: Based on the second indication information in the edge metadata, create a combined index and / or a frequency index for the edge label.

6. The method according to claim 2, characterized in that, The method further includes: Based on the lifecycle of the point labels, remove point labels from the graph database that have exceeded their lifecycle; Based on the lifecycle of the edge label, remove edge labels that have exceeded their lifecycle from the graph database.

7. A data management device, characterized in that, The device includes: The acquisition module is used to retrieve data under the first field recorded in the template of the point metadata from the data source, and match the retrieved data with the attributes of the point metadata one by one according to the first correspondence between each first field and each attribute of the point metadata; and to retrieve data under the second field recorded in the template of the edge metadata from the data source, and match the retrieved data with the attributes of the edge metadata one by one according to the second correspondence between each second field and each attribute of the edge metadata. The processing module is used to construct labels in the graph database based on the vertex metadata and the edge metadata.

8. An electronic device, characterized in that, The electronic device includes at least a processor and a memory, wherein when the processor executes a computer program or instructions stored in the memory, it implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores a computer program or instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

Images