Data writing method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a data writing method, electronic equipment and a computer readable storage medium. The method comprises the steps of creating an initial mixed index in response to to-be-written data in a data writing instruction, and determining a first sub-index and a second sub-index in the initial mixed index according to each initial sub-index in the initial mixed index and a preset graph database, the preset graph database comprises preset entity data, preset attribute data and an index relationship between the preset entity data and the preset attribute data, and the attribute data in the first sub-index exists in the preset graph database; performing index state conversion processing on the first sub-index in the preset graph database until the first sub-index in an available state is obtained; and determining a target mixed index of the available state based on the first sub-index and the second sub-index of the available state, and writing the to-be-written data into a preset graph database based on the target mixed index corresponding to the to-be-written data. Therefore, the success rate of data writing is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data writing method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] JanusGraph is a distributed graph database that effectively supports querying and processing large-scale graph data through indexes, supporting businesses such as knowledge graphs. The use of indexes is key to improving data query efficiency. When data is accessed through JanusGraph, it is first written to an HBase (Hadoop Database, a distributed columnar database) table, and then the indexed data is written to Elasticsearch (ES, a distributed search engine).

[0003] Existing data writing methods typically involve creating an index based on string data types and then writing data based on that index. However, when creating a new index for an existing attribute in a graph database, an index state transition is required. This transition process may result in the index becoming unavailable due to service failures, external operations, or other factors, thus preventing successful data writing after index creation. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a data writing method, an electronic device, and a computer-readable storage medium to improve the success rate of data writing.

[0005] To address the aforementioned technical problems, this application provides a data writing method, comprising: responding to receiving a data writing instruction, creating an initial hybrid index based on the data to be written in the data writing instruction; determining a first sub-index and a second sub-index in the initial hybrid index based on each initial sub-index in the initial hybrid index and a preset graph database, wherein the preset graph database includes preset entity data, preset attribute data, and an index relationship between the preset entity data and the preset attribute data, wherein the attribute data in the first sub-index exists in the preset graph database, the preset entity data is used to describe objects and / or things, and the preset attribute data is used to describe the attributes of objects and / or things; performing index state transformation processing on the first sub-index in the preset graph database until a usable first sub-index is obtained; determining a usable target hybrid index based on the usable first sub-index and the second sub-index; and writing the data to be written into the preset graph database based on the target hybrid index corresponding to the data to be written.

[0006] In one embodiment, the step of performing index state transition processing on the first sub-index until a first sub-index in an available state is obtained includes: performing initial transition processing on the initial index state of the first sub-index to obtain a first sub-index after state transition; and performing a second transition processing on the first sub-index after state transition in response to the index state of the first sub-index being unavailable to obtain a first sub-index in the available state.

[0007] In one embodiment, the initial index state includes an initial unavailable state and a registered state. The step of performing an initial transformation process on the initial index state of the first sub-index to obtain the first sub-index after state transformation includes: in response to the initial index state of the first sub-index being the initial unavailable state, transforming the first sub-index from the initial unavailable state to the registered state; and in response to the first sub-index being in the registered state, transforming the first sub-index from the registered state to the available state to obtain the first sub-index after state transformation.

[0008] In one embodiment, the initial sub-index includes attribute data. The step of determining the first sub-index and the second sub-index in the initial hybrid index based on each initial sub-index in the initial hybrid index and the preset graph database includes: determining whether there is preset attribute data in the preset graph database that is the same as the attribute data in the initial sub-index, and obtaining a determination result; in response to the determination result indicating existence, determining the corresponding initial sub-index as the first sub-index; in response to the determination result indicating non-existence, determining the corresponding initial sub-index as the second sub-index.

[0009] In one embodiment, before the step of determining the first sub-index and the second sub-index in the initial hybrid index based on each initial sub-index in the initial hybrid index and the preset graph database, the method further includes: determining whether each initial sub-index is a composite index based on the attribute data in each initial sub-index in the initial hybrid index; if so, determining the sub-index associated with the composite index from the preset graph database; and adding the sub-index associated with the composite index to the initial sub-index of the hybrid index.

[0010] In one embodiment, the data writing instruction includes target attribute data that correspond to the image data to be written, and the step of creating an initial hybrid index based on the data to be written in the data writing instruction includes: creating an initial sub-index between the data to be written and each target attribute data; and determining each initial sub-index as the initial hybrid index.

[0011] In one embodiment, after the step of determining the target hybrid index of the availability status based on the first sub-index and the second sub-index of the availability status, the method further includes: in response to receiving a data query instruction, determining the target hybrid index corresponding to the data to be queried from the preset graph database according to the data to be queried in the data query instruction; and determining a query result from the preset graph database based on the target hybrid index corresponding to the data to be queried.

[0012] In one embodiment, after the step of determining the query result from the preset graph database based on the target hybrid index corresponding to the query data, the method further includes: comparing the query data in the query result with preset written data to obtain a comparison result; in response to the comparison result indicating that the query data and the preset written data are completely consistent, the hybrid index optimization is determined to be successful.

[0013] To address the aforementioned technical problems, this application provides an electronic device, including a memory and a processor. The memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the aforementioned data writing method.

[0014] To address the aforementioned technical problems, this application provides a computer-readable storage medium, comprising: storing program data, wherein the program data, when executed by a processor, is used to implement the aforementioned data writing method.

[0015] The above scheme, in response to the data to be written in the data write instruction, creates an initial hybrid index. Based on the initial sub-indexes in the initial hybrid index and a preset graph database, it determines the first and second sub-indexes within the initial hybrid index. The preset graph database includes preset entity data, preset attribute data, and the index relationships between the preset entity data and preset attribute data. The attribute data in the first sub-index exists within the preset graph database. It then performs index state transition processing on the first sub-index in the preset graph database until a usable first sub-index is obtained. Based on the usable first and second sub-indexes, it determines a usable target hybrid index. Finally, it writes the data to be written into the preset graph database based on the target hybrid index corresponding to the data to be written. Thus, by performing index state transition processing on the first sub-index in the preset graph database until a usable first sub-index is obtained, the usability of the first sub-index is ensured. This, in turn, makes the target hybrid index composed of the first and second sub-indexes usable, guaranteeing successful writing of the data to be written based on the target hybrid index, thereby improving the data writing success rate. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating an exemplary embodiment of the data writing method shown in this application; Figure 2 This is a schematic diagram illustrating the unavailability state of a hybrid index as shown in this application; Figure 3 This is a flowchart illustrating an exemplary embodiment of an index conversion failure shown in this application; Figure 4 This is a flowchart illustrating an exemplary embodiment of the index optimization process shown in this application; Figure 5 This is a block diagram illustrating a data writing apparatus according to an exemplary embodiment of this application; Figure 6 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application; Figure 7 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] First, it's important to clarify that JanusGraph is a distributed graph database. Through indexes, it effectively supports querying and processing large-scale graph data, supporting businesses such as knowledge graphs. The use of indexes is key to improving data query efficiency. When data is accessed through JanusGraph, it is first written to an HBase (Hadoop Database, a distributed columnar database) table, and then the indexed data is written to Elasticsearch (ES). Existing index creation methods typically create indexes based on string data types. However, when creating new indexes for existing attributes in a graph database, a state transition is required. This state transition process may result in the index becoming unavailable due to service failures, external operations, or other factors, rendering the index unusable after creation.

[0019] Based on this, this application provides a data writing method, an electronic device, and a computer-readable storage medium. For details, please refer to... Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of a data writing method shown in this application.

[0020] The execution entity of a data writing method can be a terminal device, a server, or other processing device. The terminal device can be a computer, mobile device, terminal, computing device, vehicle-mounted device, etc. The execution entity of the data writing method can also be a data writing device. In some possible implementations, the data writing method can be implemented by a processor calling computer-readable instructions stored in memory. The execution entity of the data writing method can also be a big data cluster. A big data cluster is a computer system architecture formed by multiple computers connected through a network. Big data clusters can be deployed on a private cloud built using K8S (Kubernetes, a container orchestration engine).

[0021] It's important to note that JanusGraph graph data writing includes writing HBase table data and writing Elasticsearch (ES) index data. After receiving a data write request from the client, the JanusGraph server performs data verification and organization, first writing the HBase table data. Once the HBase table data is successfully written, the ES index data is then written. If an exception occurs while writing HBase table data, the current write request will be rolled back, and the client will detect the failure and retry. If the HBase table data is written successfully, but the ES index write fails or is not written correctly due to the index being unwritable, the client will be unable to utilize the index's capabilities to accelerate queries, resulting in slow or even unqueryable queries, and in severe cases, even service crashes. Therefore, this application provides a data writing method that optimizes the index before data writing, ensuring index availability and successful ES index data writing, thereby ensuring the integrity and consistency of graph data and improving data query efficiency.

[0022] Specifically, one data writing method in this embodiment includes the following steps: Step S110: In response to receiving a data write instruction, create an initial hybrid index based on the data to be written in the data write instruction.

[0023] A data write instruction is an instruction to write data into a pre-defined graph database. The data write instruction includes the data to be written. The data to be written can be text data, which can be entity text or relational text. For example, the data to be written could be "Zhang San," "relative," etc. The data write instruction can also include attribute data that corresponds to the data to be written.

[0024] The initial hybrid index refers to the hybrid index before optimization. The hybrid index is an external index of JanusGraph, creating indexes on the attributes of entities and relationships in a predefined graph database, supporting fuzzy matching, range queries, equality queries, etc. The initial hybrid index includes multiple initial sub-indexes. These initial sub-indexes can include the mapping between entities and attributes, and can also include the mapping between relationships and attributes.

[0025] Specifically, the data writing device creates initial sub-indexes between the data to be written and the corresponding attribute data, and determines each initial sub-index as an initial hybrid index.

[0026] Step S120: Determine the first sub-index and the second sub-index in the initial mixed index based on each initial sub-index in the initial mixed index and the preset graph database. The preset graph database includes preset entity data, preset attribute data, and the index relationship between preset entity data and preset attribute data. The attribute data in the first sub-index exists in the preset graph database. The preset entity data is used to describe objects and / or things, and the preset attribute data is used to describe the attributes of objects and / or things.

[0027] Preset graph databases are a type of NoSQL (Not Only SQL, non-relational database). In a preset graph database, nodes correspond to entity data, edges between nodes represent relationships between entities, and descriptive information on each node and edge is attribute data. Applications of preset graph databases include knowledge graphs, recommendation systems, and more.

[0028] In one embodiment, the knowledge graph simultaneously includes multiple static nodes, dynamic nodes, dynamic relationships, and static relationships. Static nodes can contain various document data, such as biological document information, vehicle document information, and event document information. Dynamic nodes can contain various spatiotemporal event data, such as violation times and travel trajectories; the data in dynamic nodes continuously increases over time. Static relationships can include kinship relationships and item ownership relationships, such as parent-child relationships, colleague relationships, and vehicle owner relationships. Dynamic relationships can include biological event relationships and vehicle event relationships, such as relationships involving appearing in the same location and violation relationships; the data in dynamic relationships also continuously increases over time.

[0029] Predefined entity data refers to nodes in a graph database that represent an object or thing in reality. Predefined entity data can be predefined text data. For example, the biological entity "person" has attributes such as identification number "number", name "name", age "age", and height "height". Predefined entity data includes multiple entities.

[0030] Preset attribute data refers to descriptive information attached to nodes or edges in a graph database. Attribute data consists of key-value pairs used to describe entities or relationships. Preset attribute data can be preset text data. Preset attribute data includes multiple preset attributes.

[0031] Preset relation data in a preset graph database refers to the edges in the graph database, used to represent the relationships between entities. For example, "likes" can have "since: 2022" and "score: 5", representing the relationship of liking, and can have attributes such as start: liking since 2022, and score: rating 5 stars.

[0032] The data writing device determines the first and second sub-indexes in the initial mixed index based on the initial sub-indexes in the initial mixed index and the preset graph database. Specifically, the data writing device compares the attribute data in each initial sub-index with the preset attribute data in the preset graph database to obtain a comparison result. If the comparison result indicates that there is preset attribute data in the preset graph database that matches the attribute data in the initial sub-index, then the corresponding initial sub-index is determined as the first sub-index. If the comparison result indicates that there is no preset attribute data in the preset graph database that matches the attribute data in the initial sub-index, then the corresponding initial sub-index is determined as the second sub-index.

[0033] Step S130: Perform index state transition processing on the first sub-index in the preset graph database until the first sub-index in the usable state is obtained.

[0034] Specifically, the data writing device creates the first sub-index to obtain the first sub-index in the preset graph database. The initial index state of the first sub-index is the initial unavailable state.

[0035] The data writing device also includes: creating a second sub-index to obtain a second sub-index in a preset graph database, wherein the initial index state of the second sub-index is an available state.

[0036] In one embodiment, JanusGraph creates initial sub-indexes for each attribute sequentially. If the attribute corresponding to the initial sub-index does not exist in the preset graph database, the created initial sub-index becomes the second sub-index, and its initial index state is set to available after creation. If JanusGraph creates initial sub-indexes for existing attributes, the created initial sub-index becomes the first sub-index, and its initial index state is set to initially unavailable after successful creation.

[0037] The first sub-index includes index field values. An index field value of INSTALLED indicates that the index is initially unavailable; an index field value of REGISTERED indicates that the index is registered; an index field value of DISABLED indicates that the index is disabled; and an index field value of ENABLED indicates that the index is available.

[0038] The available state refers to the state where indexing can be successful.

[0039] The data writing device performs index state transition processing on the first sub-index in the preset graph database until a usable first sub-index is obtained. Specifically, the data writing device modifies the index state data source of the first sub-index in the data source of the preset graph database to realize the index state transition of the first sub-index. In response to the first sub-index becoming usable after the index state transition, a usable first sub-index is obtained.

[0040] Step S140: Determine the target hybrid index of the available state based on the first and second sub-indexes of the available state.

[0041] A target hybrid index refers to an optimized hybrid index. A target hybrid index includes a first sub-index of available state and a second sub-index of available state.

[0042] The data writing device determines the target hybrid index of the available state based on the first and second sub-indexes of the available state. Specifically, the data writing device determines the first and second sub-indexes of the available state as the target hybrid index of the available state.

[0043] In one embodiment, the data writing device creates an index for attributes that do not exist in the preset graph database to obtain a second sub-index of the available state; creates an index for attributes that exist in the preset graph database to obtain a first sub-index of the unavailable state; performs index state transformation processing on the first sub-index of the unavailable state until a first sub-index of the available state is obtained; and determines the first sub-index of the available state and the second sub-index of the available state as the target hybrid index of the available state.

[0044] In one embodiment, JanusGraph contains attributes named "number", "name", "age", and "height". A new entity "person" is added, and a hybrid index is created based on the attributes "number", "name", "age", and "height". The data types corresponding to the attributes "number", "name", "age", and "height" are String, String, Integer, and Double, respectively. Initially, the sub-index corresponding to the attributes "number", "name", "age", and "height" is the first sub-index, requiring a state transition. Combined with... Figure 1As shown, if the first sub-indexes corresponding to the entity "person" experience a state transition anomaly during the state transition process, resulting in the first sub-indexes of "number" and "name" being in the ENABLED state, while the indexes of "age" and "height" are in the REGISTERED state, then the mixed index of person is in an abnormal state, with some parts available and some parts unavailable. At this time, there is a risk of anomalies and data inconsistencies in the graph database's write and query operations, that is, the index data and the original data do not match.

[0045] For example, a client writes entity data of the form (id, number, name, age, height) as (1, "32032XYZ", "Zhang San", 28, 185). This data can be written to the HBase table normally, but when written to Elasticsearch (ES), only the "number" and "name" data can be written, while ES loses the index data for "age" and "height". When the client performs an index query using "age" or "height", it cannot retrieve the index information for "id=1", and therefore cannot perform a query on the HBase table, resulting in an empty query result. To avoid this situation where the HBase table has data, but cannot be queried due to index anomalies, leading to inconsistencies in written data between the HBase table and ES, and unexpected queries, this application, in response to the first sub-index being in an unavailable state after a state transition, performs another transformation on the first sub-index until a usable first sub-index is obtained. This ensures that the target hybrid index is in a usable state, thereby avoiding inconsistent written data and unexpected queries.

[0046] Step S150: Write the data to be written into the preset graph database based on the target hybrid index corresponding to the data to be written.

[0047] Specifically, the data writing device performs the writing operation of the data to be written according to the writing position corresponding to the target hybrid index, thereby writing the data to be written into the preset graph database.

[0048] As can be seen, in response to creating an initial hybrid index based on the data to be written in the data write instruction, the first and second sub-indexes in the initial hybrid index are determined based on the initial sub-indexes in the initial hybrid index and the preset graph database. The preset graph database includes preset entity data, preset attribute data, and the index relationship between preset entity data and preset attribute data. The attribute data in the first sub-index exists in the preset graph database. The first sub-index in the preset graph database undergoes index state transformation processing until a usable first sub-index is obtained. Based on the usable first and second sub-indexes, a usable target hybrid index is determined. Based on the target hybrid index corresponding to the data to be written, the data to be written is written to the preset graph database. Thus, by performing index state transformation processing on the first sub-index in the preset graph database until a usable first sub-index is obtained, the usability of the first sub-index is ensured, thereby making the target hybrid index composed of the first and second sub-indexes usable. This ensures successful writing of the data to be written based on the target hybrid index, thereby improving the data writing success rate.

[0049] Before the step of determining the first and second sub-indexes in the initial mixed index based on each initial sub-index in the initial mixed index and the preset graph database, the method further includes: determining whether each initial sub-index is a composite index based on the attribute data in each initial sub-index in the initial mixed index; if so, determining the sub-index associated with the composite index from the preset graph database; and adding the sub-index associated with the composite index to the initial sub-indexes of the mixed index.

[0050] A composite index is an index created for each relation in a predefined graph database. A composite index simultaneously writes the attribute data of the left and right nodes of the relation in the predefined graph database into the corresponding index of the relation, combining multiple attributes into an ordered index structure, resulting in high query efficiency for spatiotemporal relation data.

[0051] The initial sub-index includes attribute data, which includes the index type and index name of each attribute. The index type can be a composite index.

[0052] In one embodiment, in response to the fact that the index type in the initial sub-index is a composite index, the data writing device obtains the sub-index associated with the composite index from a preset graph database, adds the associated sub-index to the initial sub-index of the composite index, and uses the added sub-index as the first sub-index.

[0053] The initial sub-index includes attribute data. The step of the data writing device determining the first sub-index and the second sub-index in the initial mixed index based on each initial sub-index in the initial mixed index and the preset graph database includes: determining whether preset attribute data with the same attribute data as the attribute data in the initial sub-index exists in the preset graph database, and obtaining a determination result; in response to the determination result indicating existence, determining the corresponding initial sub-index as the first sub-index; in response to the determination result indicating non-existence, determining the corresponding initial sub-index as the second sub-index.

[0054] Specifically, when the data writing device receives the optimization instruction, it queries the attribute data in each initial sub-index in the preset graph database. If preset attribute data that matches the attribute data in each initial sub-index is found, the created initial sub-index is determined as the first sub-index; if preset attribute data that matches the attribute data in each initial sub-index is not found, the created initial sub-index is determined as the second sub-index.

[0055] In one embodiment, the data writing device receives an input target object and determines the links between the target object and multiple attributes of the target object as initial sub-indexes, and each initial sub-index forms an initial mixed index of the target object. In response to a target object attribute being a preset attribute existing in a preset graph database, the data writing device determines the initial sub-index between the target object and its attributes as a first sub-index; in response to a target object attribute not existing in the preset graph database, the data writing device determines the initial sub-index between the target object and its attributes as a second sub-index.

[0056] The step of the data writing device performing index state transition processing on the first sub-index until a usable first sub-index is obtained includes: performing initial transition processing on the initial index state of the first sub-index to obtain a state-transitioned first sub-index; and, in response to the index state of the state-transitioned first sub-index being unusable, performing a second transition processing on the state-transitioned first sub-index to obtain a usable first sub-index.

[0057] An unavailable state refers to a state where indexing cannot be successful. Unavailable states can include initial unavailable state, registered state, and disabled state. Specifically, during the initial transformation process of the first sub-index by the data writing device, abnormal situations such as concurrency issues or power outages may cause the index to enter an unavailable state.

[0058] The initial index state includes the initial unavailable state and the registered state. The initial index state can also be the disabled state.

[0059] The step of the data writing device performing an initial transformation process on the initial index state of the first sub-index to obtain the first sub-index after state transformation includes: in response to the initial index state of the first sub-index being an initial unavailable state, transforming the first sub-index from the initial unavailable state to the registered state; in response to the first sub-index being in the registered state, transforming the first sub-index from the registered state to the available state to obtain the first sub-index after state transformation.

[0060] In response to the initial index state of the first sub-index being initially unavailable, the data writing device transitions the first sub-index from the initially unavailable state to the registered state. Specifically, the data writing device modifies the index state data source of the first sub-index from the initially unavailable state to the registered state in the data source of the preset graph database.

[0061] After the data writing device converts the first sub-index from an initially unavailable state to a registered state, the conversion is successful if the first sub-index is in a registered state, and fails if the first sub-index is in an initially unavailable state or a disabled state.

[0062] After the step of converting the first sub-index from a registered or disabled state to an available state, the data writing device determines the current index state of the first sub-index. If the first sub-index is in an available state, the initial conversion is successful; if the first sub-index is in an initially unavailable state, a registered state, or a disabled state, the initial conversion fails.

[0063] If the data writing device responds to an initial conversion failure and the number of initial conversions is less than the preset number of conversions, it will convert the first sub-index from the initial unavailable state to the registered state, or convert the first sub-index from the registered state or the disabled state to the available state.

[0064] For example, in response to the first sub-index being in a registered state and the initial number of transformations being less than a preset number of transformations, the data writing device transforms the first sub-index from a registered state to an available state, thus obtaining the first sub-index after the state transformation. Specifically, the data writing device modifies the index state data source of the first sub-index from a registered state to an available state in the data source of a preset graph database.

[0065] For example, in response to the first sub-index being in a disabled state and the initial number of transformations being less than the preset number of transformations, the data writing device transforms the first sub-index from a disabled state to an available state, obtaining the first sub-index after the state transformation. Specifically, the data writing device modifies the index state data source of the first sub-index from a disabled state to an available state in the data source of the preset graph database.

[0066] For example, in response to the first sub-index being in an initially unavailable state and the initial number of transformations being less than the preset number of transformations, the data writing device transforms the first sub-index from the initially unavailable state to the registered state; then transforms it from the registered state to the available state to obtain the first sub-index after the state transformation.

[0067] In response to the first sub-index being in an unavailable state after a state transition, the data writing device performs another state transition on the first sub-index to obtain an available first sub-index. As an example, in response to the first sub-index being in an unavailable state after a state transition, the data writing device stores the index information of the first sub-index in a preset storage area of ​​the data writing device to obtain a first sub-index to be processed; and performs index state transition processing on each first sub-index to be processed to obtain a first sub-index in an available state.

[0068] As another example, in response to the first sub-index being in an unavailable state after a state transition, the data writing device sends the index information of the first sub-index to a persistent node in the cluster management engine service, triggering the persistent node to create a child node containing the index information of the first sub-index; if the data writing device detects the existence of a child node at the persistent node, it obtains the index information at the child node, performs an index state transition on the first sub-index in the index information, and obtains the first sub-index after a second state transition; if the first sub-index after the second state transition is in an available state, it obtains the first sub-index in an available state.

[0069] The data writing device further includes: in response to the first sub-index being in an unavailable state after another state transition, sending an index warning message to trigger a repair process for the first sub-index, thereby obtaining the first sub-index in an available state.

[0070] The data writing device includes a recording module and a detection module. The recording module records the first sub-index that failed to be converted, and the detection module detects the index information stored in a preset storage area or persistent node.

[0071] In one embodiment, combined with Figure 3As shown, the JanusGraph service includes the created IndexProtectModule and IndexStatusTransfer modules. The IndexProtectModule serves as the detection module, and the IndexStatusTransfer module serves as the recording module. The JanusGraph service triggers the Zookeeper (cluster management engine) service to create a persistent node ` / index / graphName`, initialized as an empty node, used to store index information where index status transitions failed. Its child nodes represent index information in `graphName` with abnormal index statuses; the index information can be the index name. For example, ` / index / graphtest / vehicle` indicates that the `vehicle` index in `graphtest` is in an unavailable state. This application, through the collaborative interaction of the IndexStatusTransfer and IndexProtectModule modules with JanusGraph and Zookeeper, automatically detects and ensures the availability of hybrid indexes, thereby guaranteeing the integrity and consistency of graph database data writing and querying.

[0072] If the data writing device fails to convert the index state during the conversion process, the IndexStatusTransfer module records the index information of the first sub-index that failed to convert and triggers the persistent node to create a new sub-node to record the index information. The IndexProtectModule module monitors whether the / index / graphName node has changed. If the / index / graphName node has changed, it retrieves the index information from the changed sub-node, re-triggers the index state conversion of the first sub-index in the index information, converts the index state to an available state to ensure the availability of the first sub-index, and updates the JanusGraph information and deletes the corresponding sub-node in ZooKeeper. As can be seen, this application performs an initial transformation process on the initial index state of the first sub-index to obtain the first sub-index after state transformation. In response to the first sub-index being in an unavailable state after state transformation, it performs another transformation process to obtain a first sub-index in an available state. Therefore, by optimizing the initial hybrid index before data writing, a target hybrid index in an available state is obtained, thus avoiding index state transformation every time data is written, improving data writing efficiency. Simultaneously, it avoids creating indexes during business operations, consuming service resources, and affecting online services. It also effectively supports complex queries such as equality queries, range queries, and text matching for different types of business operations.

[0073] The data writing device's data writing instruction includes target attribute data that corresponds to the data to be written. The step of creating an initial hybrid index for the data to be written in the data writing instruction includes: creating an initial sub-index between the data to be written and each target attribute data; and determining each initial sub-index as an initial hybrid index.

[0074] In one embodiment, the data to be written is "person", and the corresponding attribute data is "number", "name", "age", and "height". An initial sub-index between "person" and the corresponding attribute data "number", "name", "age", and "height" is created in a preset graph database using a preset index creation method or code.

[0075] In response to receiving a data write command, the data writing device determines the target hybrid index corresponding to the data to be written in the preset graph database. Specifically, in response to receiving a data write command, the data writing device queries the preset graph database for the data to be written, obtains the target hybrid index corresponding to the data to be written, acquires the current index status of the target hybrid index, and if it is in an available state, performs the operation of writing the data to be written.

[0076] The data writing device further includes: in response to receiving a data writing instruction, querying the data to be written in a preset graph database to obtain the target hybrid index corresponding to the data to be written, obtaining the current index status of the target hybrid index, and if it is in an unavailable state, triggering a status activation task for the target hybrid index.

[0077] The steps of triggering the state activation task of the target hybrid index by the data writing device include: performing index state transition processing on each sub-index in the target hybrid index until a sub-index in an available state is obtained.

[0078] In one embodiment, after obtaining the target hybrid index in an available state, in order to prevent the optimized target hybrid index from becoming abnormal due to related tasks executed in the preset graph database, the data writing device, upon receiving the data writing instruction, determines whether the target hybrid index corresponding to the data to be written is in an available state. If it is, the writing operation is performed; otherwise, the state activation task of the target hybrid index is triggered to ensure that the target hybrid index is in an available state before the data writing is performed, thereby avoiding the problem of inconsistency between ES index data and HBase table data.

[0079] After the step of determining the target hybrid index of availability based on the first sub-index and the second sub-index of availability, the method further includes: in response to receiving a data query instruction, determining the target hybrid index corresponding to the data to be queried from a preset graph database according to the data to be queried in the data query instruction; and determining the query result from the preset graph database based on the target hybrid index corresponding to the data to be queried.

[0080] A data query command is an instruction to retrieve data from a pre-defined graph database. A data query command includes the data to be queried. The data to be queried can be one or more of the following: entities, relations, and attributes.

[0081] Specifically, in response to receiving a data query instruction, the data writing device queries the data to be queried in the preset graph database, obtains the target hybrid index corresponding to the data to be queried, jumps to the corresponding target position based on the target hybrid index corresponding to the data to be queried, determines the data at each target position as the query result, and displays the query result.

[0082] After the step of determining the query result from the preset graph database based on the target hybrid index corresponding to the data to be queried, the method further includes: comparing the query data in the query result with the preset write data to obtain a comparison result; in response to the comparison result indicating that the query data and the preset write data are completely consistent, the method determines that the hybrid index optimization is successful.

[0083] Pre-written data refers to data that has been written to the pre-defined graph database beforehand. There is a correspondence between the queried data and the pre-written data; the pre-written data can be retrieved by querying the queried data. For example, if the pre-written data is entity data (1, "32032XYZ", "Zhang San", 28, 185), the queried data would be Zhang San, etc.

[0084] The comparison results can include either completely identical query data and preset write data, or they can include queries that are not completely identical to preset write data.

[0085] The data writing device compares the query data in the query results with the preset write data to obtain a comparison result. Specifically, the data writing device uses a deep learning module to convert the query data and the preset write data into vectors in a high-dimensional space and obtains the similarity between the two vectors. If the similarity is greater than a preset similarity threshold, the comparison result is determined to be that the query data and the preset write data are completely consistent; otherwise, the comparison result is determined to be that the query data and the preset write data are not completely consistent.

[0086] The data writing device also includes: in response to the comparison result indicating that the query data is not completely consistent with the preset write data, determining that the hybrid index optimization has failed, and optimizing the target hybrid index again.

[0087] In one embodiment, combined with Figure 4 As shown, the data writing device creates a first sub-index for an existing attribute in the preset graph database, triggering the corresponding index state transition task. The index state transition task adds the first sub-index to a Container; obtains the attribute information of each initial sub-index in the hybrid index; determines whether each initial sub-index is a composite index. If it is, it obtains the sub-index associated with the composite index and adds the associated sub-index to the Container. If not, it traverses each first sub-index in the Container, determining whether the index state of the current first sub-index is INSTALLED based on the index field value associated with the currently traversed first sub-index. If yes, it modifies the index state of the current first sub-index to REGISTERED, and then modifies it to ENABLED; otherwise, it determines whether the index state of the current first sub-index is REGISTERD or DISABLED. If the index state of the current first sub-index is REGISTERD or DISABLED, it directly modifies the index state of the current first sub-index to ENABLED. After the data writing device completes the state modification, it checks whether the current index state of the first sub-index is available. If yes, the index state transition is considered successful, and the index cache information is updated. If not, it checks whether the number of state transitions exceeds the preset number. If not, the index state transition task is re-executed; if yes, a child node is added under the ` / index / graphName` node in Zookeeper, and the index information of the current first sub-index is recorded under the child node. It can be seen that by increasing the number of transitions in the index state transition task, the frequency of interaction with Zookeeper can be reduced, while the success rate of the index state transition task execution can be improved.

[0088] Figure 5 This is a block diagram illustrating a data writing apparatus according to an exemplary embodiment of this application. Figure 5 As shown, the exemplary data writing device 500 includes: an index creation module 510, a sub-index determination module 520, a state transition module 530, a target hybrid index determination module 540, and a data writing module 550. Specifically: The index creation module 510 is used to create an initial hybrid index based on the data to be written in the data write instruction in response to receiving a data write instruction.

[0089] The sub-index determination module 520 is used to determine the first sub-index and the second sub-index in the initial mixed index based on each initial sub-index in the initial mixed index and the preset graph database. The preset graph database includes preset entity data, preset attribute data, and the index relationship between the preset entity data and the preset attribute data. The attribute data in the first sub-index exists in the preset graph database. The preset entity data is used to describe objects and / or things, and the preset attribute data is used to describe the attributes of objects and / or things.

[0090] The state transition module 530 is used to perform index state transition processing on the first sub-index in the preset graph database until the first sub-index in the usable state is obtained.

[0091] The target hybrid index determination module 540 is used to determine the target hybrid index with available status based on the first sub-index and the second sub-index.

[0092] The data writing module 550 is used to write the data to be written into the preset graph database based on the target hybrid index corresponding to the data to be written.

[0093] In this exemplary data writing device, in response to creating an initial hybrid index based on the data to be written in the data writing instruction, a first sub-index and a second sub-index in the initial hybrid index are determined based on each initial sub-index in the initial hybrid index and a preset graph database. The preset graph database includes preset entity data, preset attribute data, and index relationships between the preset entity data and the preset attribute data. The attribute data in the first sub-index exists in the preset graph database. The first sub-index in the preset graph database undergoes index state transition processing until a usable first sub-index is obtained. Based on the usable first and second sub-indexes, a usable target hybrid index is determined. Based on the target hybrid index corresponding to the data to be written, the data to be written is written to the preset graph database. Thus, by performing index state transition processing on the first sub-index in the preset graph database until a usable first sub-index is obtained, the first sub-index is ensured to be usable, thereby ensuring that the target hybrid index composed of the first and second sub-indexes is in a usable state, guaranteeing successful writing of the data to be written based on the target hybrid index, and improving the data writing success rate.

[0094] The functions of each module can be found in the data writing method implementation example, and will not be repeated here.

[0095] To implement the data writing method of the above embodiments, this application proposes another electronic device, please refer to [link / reference needed]. Figure 6 , Figure 6 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0096] Electronic device 600 includes memory 601 and processor 602, wherein memory 601 and processor 602 are coupled together.

[0097] The memory 601 is used to store program data, and the processor 602 is used to execute the program data to implement the data writing method of the above embodiment.

[0098] In this embodiment, processor 602 can also be referred to as CPU (Central Processing Unit). Processor 602 may be an integrated circuit chip with signal processing capabilities. Processor 602 can also be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 602 can be any conventional processor.

[0099] This application also provides a computer-readable storage medium, such as Figure 7 As shown, the computer-readable storage medium 700 is used to store program data 701, which, when executed by the processor, is used to implement the data writing method as described in the method embodiment of this application.

[0100] The methods involved in the data writing method embodiments of this application, when implemented as software functional units and sold or used as independent products, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A data writing method, characterized in that, The method includes: In response to receiving a data write instruction, an initial hybrid index is created based on the data to be written in the data write instruction; The first sub-index and the second sub-index in the initial hybrid index are determined based on each initial sub-index in the initial hybrid index and the preset graph database. The preset graph database includes preset entity data, preset attribute data, and the index relationship between the preset entity data and the preset attribute data. The attribute data in the first sub-index exists in the preset graph database. The preset entity data is used to describe objects and / or things, and the preset attribute data is used to describe the attributes of objects and / or things. The first sub-index in the preset graph database is subjected to index state transition processing until a usable first sub-index is obtained; The target hybrid index of the available state is determined based on the first sub-index and the second sub-index of the available state; The data to be written is written into the preset graph database based on the target hybrid index corresponding to the data to be written.

2. The method according to claim 1, characterized in that, The step of performing index state transition processing on the first sub-index until a usable first sub-index is obtained includes: The initial index state of the first sub-index is subjected to an initial transformation process to obtain the first sub-index after the state transformation; In response to the first sub-index being in an unavailable state after the state transition, the first sub-index is transformed again to obtain the first sub-index in the available state.

3. The method according to claim 2, characterized in that, The initial index state includes an initial unavailable state and a registered state. The step of performing an initial transformation process on the initial index state of the first sub-index to obtain the first sub-index after state transformation includes: In response to the initial index state of the first sub-index being the initial unavailable state, the first sub-index is transitioned from the initial unavailable state to the registered state. In response to the first sub-index being in the registered state, the first sub-index is transitioned from the registered state to the available state to obtain the first sub-index after the state transition.

4. The method according to claim 1, characterized in that, The initial sub-index includes attribute data. The step of determining the first and second sub-indexes in the initial mixed index based on each initial sub-index in the initial mixed index and the preset graph database includes: Determine whether there is any preset attribute data in the preset graph database that is the same as the attribute data in the initial sub-index, and obtain the determination result; In response to the existence indicated by the judgment result, the corresponding initial sub-index is determined as the first sub-index; In response to the judgment result indicating that it does not exist, the corresponding initial sub-index is determined as the second sub-index.

5. The method according to claim 1, characterized in that, Before the step of determining the first and second sub-indexes in the initial hybrid index based on each initial sub-index in the initial hybrid index and the preset graph database, the method further includes: Determine whether each initial sub-index is a composite index based on the attribute data in each initial sub-index of the initial hybrid index; If so, then determine the sub-index associated with the composite index from the preset graph database; The sub-indexes associated with the composite index are added to the initial sub-indexes of the hybrid index.

6. The method according to claim 1, characterized in that, The data write instruction includes target attribute data that correspond to the data to be written. The step of creating an initial hybrid index for the data to be written in the data write instruction includes: Create an initial sub-index between the data to be written and the target attribute data; Each initial sub-index is determined as the initial hybrid index.

7. The method according to claim 1, characterized in that, After the step of determining the target hybrid index of availability based on the first sub-index and the second sub-index of availability, the method further includes: In response to receiving a data query instruction, the target hybrid index corresponding to the data to be queried is determined from the preset graph database according to the data to be queried in the data query instruction; The query result is determined from the preset graph database based on the target hybrid index corresponding to the data to be queried.

8. The method according to claim 7, characterized in that, After the step of determining the query result from the preset graph database based on the target hybrid index corresponding to the data to be queried, the method further includes: The query data in the query results is compared with the preset write data to obtain the comparison result; If the comparison result indicates that the queried data is completely consistent with the preset written data, then the hybrid index optimization is determined to be successful.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, include: The system stores program data, which, when executed by a processor, is used to implement the method as described in any one of claims 1-8.

Images