SparkSql-based GQL graph database data importing method and device
By introducing SparkSql in the GQL graph database data import process, it supports custom data processing logic, and solves the problem of lack of flexibility in data import in the existing technology, achieving a more efficient and flexible data import process.
Patent Information
- Application Number
- CN202510001043.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-02
- Publication Date
- 2025-02-07
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, there is a lack of flexibility in the data import process, which makes it difficult to meet the user's custom data processing needs, resulting in inefficient import of data from GQL graph database.
Provide a GQL graph database data import method based on SparkSql, by obtaining the data to be imported, pre-processing according to preset data processing rules or custom SparkSql expressions, map the data into the format supported by GQL graph database and converting it into batch writing. The entered GQL statement is finally sent to the graph database service for import.
It improves the flexibility of data processing for different data sources during data import, meets the needs of different business scenarios, greatly improves the efficiency of data import, and reduces the workload of data source maintenance.
Smart Images

Figure CN119397067A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of graph database and data processing technology, and in particular to a method and device for importing GQL graph database data based on SparkSql. Background Art
[0002] With the rapid development of big data technology, different types of database systems are widely used in various business scenarios. Graph Database has received more and more attention and application due to its advantages in processing complex relational data. GQL (Graph Query Language), as the standard query language for graph databases, provides powerful query functions. However, the data formats and structures of different database systems are different, and how to efficiently and flexibly import these data into the GQL graph database has become a major challenge. In the prior art, due to the various data preprocessing requirements of different source data systems for different businesses, such as null value conversion, selecting and importing multiple columns according to business needs, etc., the data import process often lacks flexibility and is difficult to meet the user's customized data processing needs, resulting in users needing to preprocess the data themselves, making the GQL graph database data import inefficient and requiring users to preprocess data. Summary of the invention
[0003] The purpose of this application is to overcome the problem of lack of flexibility in source data preprocessing in the data import process in the prior art, and to provide a GQL graph database data import method and device based on SparkSql.
[0004] In a first aspect, a method for importing GQL graph database data based on SparkSql is provided, comprising:
[0005] Get the data to be imported;
[0006] Preprocess the imported data according to preset data processing rules or custom SparkSql expressions;
[0007] Map the preprocessed data to be imported into a data format supported by the GQL graph database, and convert the mapped data into GQL statements for batch writing;
[0008] The GQL statement is sent to the graph database service, which executes the GQL statement to import data and returns the import result.
[0009] In some possible implementations, obtaining the data to be imported includes:
[0010] Obtain a configuration file, wherein the configuration file includes connection information of a data source, a database to be read, a data table, and a file path;
[0011] A connection is established with the source data system according to the configuration information in the configuration file, and the specified data is read as the data to be imported.
[0012] In some possible implementations, obtaining the data to be imported further includes:
[0013] Define the configuration items of the new data source in the configuration file, wherein the configuration items include the data source category, link address, user name, password and reading concurrency, and the data source category is the fully qualified class name of the newly defined data source;
[0014] Inherit the SourceConfigResolver abstract class and implement the getSourceConfigEntry method in the SourceConfigResolver abstract class to perform configuration parsing, which is used to parse the new data source configuration from the configuration file;
[0015] Inherit the DataSourceReader abstract class in the import tool and implement the readData method in the DataSourceReader abstract class to read the new data source as the data to be imported.
[0016] In some possible implementations, the preset data processing rule includes at least one of the following:
[0017] Merge multiple columns into one;
[0018] Split a column into multiple columns based on the specified delimiter;
[0019] Filter data according to preset conditions;
[0020] Concatenate a constant to the specified column;
[0021] Use the specified constant as a new column of data.
[0022] In some possible implementations, preprocessing is performed on the imported data according to the custom SparkSql expression, including: adding a configuration UDFSql of the custom SparkSql expression in the configuration file, converting the custom SparkSql expression into an optimized physical execution task, which is executed by the Spark engine to complete the preprocessing of the imported data.
[0023] In some possible implementations, the import result includes at least one of the following: whether the data writing request is successful, error information, a delay in executing the requested operation, and an amount of vertex-edge data affected on the database.
[0024] In some possible implementations, it also includes: printing the processing results of the graph database service, wherein the processing results include at least one of the following: the graph database service that executes the GQL statement, the delay in executing the operation, and the amount of data processed by the current GQL statement.
[0025] In a second aspect, a GQL graph database data import device based on SparkSql is provided, comprising:
[0026] The acquisition module is used to obtain the data to be imported;
[0027] The preprocessing module is used to preprocess the imported data according to preset data processing rules or custom SparkSql expressions;
[0028] The conversion module is used to map the preprocessed data to be imported into a data format supported by the GQL graph database, and convert the mapped data into GQL statements for batch writing;
[0029] The import module is used to send the GQL statement to the graph database service, and the graph database service executes the GQL statement to import data and return the import result.
[0030] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable medium stores a program code for execution by a device, wherein the program code includes steps for executing the method in any one of the implementations of the first aspect described above.
[0031] In a fourth aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements a method as in any one of the implementations in the first aspect above.
[0032] This application has the following beneficial effects:
[0033] 1. During the data import process, this application supports the powerful operators of SparkSql, which enables users to flexibly respond to a variety of complex business needs. Through complex SQL expressions, the structure of source data can be dynamically adjusted to meet the needs of different business scenarios, which greatly improves the flexibility of data processing for different data sources during data import;
[0034] 2. This application can realize dynamic loading of data source plug-ins, and can expand new data sources while keeping the code of the import tool itself unchanged, and can support a wider range of data sources. At the same time, the data source expansion capability is handed over to the user, which can enhance the flexibility of expanding the data source and reduce the maintenance work of the data source on the import tool. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0037] Figure 1 It is a flowchart of the method for importing GQL graph database data based on SparkSql in Example 1 of the present application; Figure 2 It is a flowchart of data preprocessing in the GQL graph database data import method based on SparkSql in Example 1 of the present application; Figure 3 It is a schematic diagram of splicing columns in the GQL graph database data import method based on SparkSql in Example 1 of the present application; Figure 4 It is a schematic diagram of splitting columns in the GQL graph database data import method based on SparkSql in Example 1 of the present application; Figure 5 It is a schematic diagram of filtering according to conditions in the GQL graph database data import method based on SparkSql in Example 1 of the present application; Figure 6 It is a schematic diagram of splicing constants in the GQL graph database data import method based on SparkSql in Example 1 of the present application; Figure 7 It is a schematic diagram of specifying a constant as a new column in the GQL graph database data import method based on SparkSql in Example 1 of the present application; Figure 8 It is a structural block diagram of a GQL graph database data import device based on SparkSql in Example 2 of the present application; Fig. 9 It is a schematic diagram of the internal structure of the electronic device of Example 4 of the present application.
[0038] Reference numerals:
[0039] 100, acquisition module; 200, preprocessing module; 300, conversion module; 400, import module. DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] Example 1
[0042] like Figure 1 As shown, a method for importing GQL graph database data based on SparkSql involved in Example 1 of the present application includes:
[0043] S100, obtaining data to be imported;
[0044] In this embodiment, data to be imported is obtained from different database systems. Data sources may include graph databases, relational databases, NoSQL databases, file systems, streaming data systems, etc. Connection information of different data sources, databases to be read, data tables, file paths and other information are configured in a configuration file. The data import application reads the required information from the configuration file, establishes a connection with the source data system, and reads data in the specified data table or specified file.
[0045] Although a variety of different data sources are built-in, it may not cover all data sources. When users have data from other data systems, they can customize the reading of data sources based on this embodiment. This embodiment provides an abstraction for data source reading, including data source configuration, source data reading interface, and data processing flow. When users customize data sources, they only need to inherit the DataSourceReader abstract class in the import tool and implement the readData method in this class, inherit the SourceConfigResolver abstract class and implement the getSourceConfigEntry method in this class for configuration parsing. How to read data is defined by the user's code, and the defined data source is handed over to the import tool for the next step of data mapping and conversion and written into the graph database.
[0046] When a user customizes a new data source, the steps for custom development are as follows: first create a code module, introduce the dependency of the import tool data source module, define the configuration items of the new data source, such as link address, user name, password, reading concurrency, etc., inherit the SourceConfigResolver abstract class and implement the getSourceConfigEntry method in the SourceConfigResolver abstract class for configuration parsing, which is used to parse the new data source configuration from the configuration file; inherit the DataSourceReader abstract class in the import tool and implement the readData method in the DataSourceReader abstract class to read the source data.
[0047] When using a custom data source, users need to configure the data source category in the configuration file as the newly defined fully qualified class name. In this way, the import tool dynamically loads the user-defined data source class through reflection when loading the data source, and calls the user-defined readData method to read the source data.
[0048] S200, preprocessing the imported data according to preset data processing rules or custom SparkSql expressions;
[0049] After obtaining the data to be imported, different businesses may need different processing requirements for the source data, for example: merging multiple columns as one column to import into a certain attribute of the graph database, splitting a column into multiple columns and importing them into multiple attributes of the graph database, filtering data rows according to the business, or performing data calculations on a certain column, etc. By summarizing the common preprocessing requirements in different business fields, the data import of this embodiment supports built-in data preprocessing rules, for example: by configuring concat.oldFields, concat.newField, concat.splicer to realize the requirement of splicing multiple columns of data into one column according to the splicing character, by configuring separate.oldField, separate.newFields,concat.sep to realize the requirement of splitting a column into multiple columns according to the separator, etc.
[0050] In this embodiment, the implementation logic of the data preprocessing rules and processing methods built into the import tool is as follows: 1. If the preprocessing configuration is a concatenation configuration, obtain the configured column name to be concatenated and the new column name after concatenation, perform string concatenation on each row of data in the data set, and concatenate the values of the column to be concatenated together using the specified concatenation character. For example, Figure 3As shown, the data in column a of the source data set is "123", the data in column b is "abc", and the concatenation character is "_". Column a and column b of the source data set are concatenated into a new column c, and the data in column c of the concatenated result is "123_abc".
[0051] 2. If the preprocessing configuration is a split configuration, obtain the configured column name to be split and the new column name after splitting, perform string splitting on each row of data in the data set, and split the string into multiple substrings according to the specified delimiter as new column values. For example, as shown in Figure 4, the data of column c of the source data set is "1,2", and the column c in the source data set is split into columns a and b according to the comma, and the split result is: the data of column a is "1", and the data of column b is "2".
[0052] 3. If the preprocessing configuration is a filtering configuration, filtering is performed on each row of the data set, and the filtered data set is returned. For example, Figure 5 As shown in the figure, column a in the source data set contains three rows of data, namely "1", "2" and "3", and column b contains three rows of data, namely "10", "11" and "12". To filter the source data set with a>1, you need to filter out the row of data where a is 1. The result is: column a contains two rows of data, namely "2" and "3", and column b contains two rows of data, namely "11" and "12".
[0053] 4. If the preprocessing configuration is to concatenate constants, obtain the column name and constant value to be concatenated, perform string concatenation on each row of data in the data set, and concatenate the column to be concatenated and the constant together. For example, Figure 6 As shown, the data in column a of the source data set is "1", a constant value "test" is added to column a of the source data set, the new added column is c, and the result is that the data in column c is "1 test".
[0054] 5. If the preprocessing configuration specifies a constant as a new column, a column is appended to the data set, and all values in the column are the specified constant values. For example, Figure 7 As shown, column a in the source data set contains three rows of data, "1", "2" and "3", and column b contains three rows of data, "10", "11" and "12". The constant "test" is specified as a new column c in the source data set. The result is that a new column c is added to the source data set, in which column c contains three rows of data and the content is "test".
[0055] In addition to the built-in preprocessing rules, in order to meet various data preprocessing requirements in different business fields and for the flexibility of data preprocessing, this application also supports custom SparkSql expressions to support the preprocessing requirements of special businesses.
[0056] Specifically, during the data import process, users can define specific SparkSql syntax expressions according to actual business needs to flexibly process source data. By providing custom SparkSql expressions, users can specify various data transformation, cleaning, aggregation and other operations during the data import process. Users can enter SparkSql syntax expressions through the configuration interface or API interface to specify data processing logic. These custom SparkSql expressions can include common data conversion operations, such as: SELECT, JOIN, GROUP BY, WHERE, ORDER BY, CONCAT, etc., and also support adding columns of specified data, such as: SELECT*, current_timestamp as newCol, etc. The corresponding Spark execution plan can be generated based on these custom SparkSql expressions, and the expressions can be converted into optimized physical execution tasks, which are finally executed efficiently by the Spark engine. For example: the custom SparkSql expression select *, least(col1,col2) as new_colfrom exchange will add a new column of data based on the source data. The column name is new_col, and the value of this column is the smaller value of column col1 and column col2. For this expression, the import tool will first read the source data, create a data view called exchange for the source data (which can be understood as a virtual table with the table name exchange and the data in the table being the two-dimensional table data of Spark Dataframe), then generate a new column of data in each row of data, take the smaller value of column col1 and column col2 as the value of the new column, and finally read all the data from exchange.
[0057] During the data import process, the powerful operators that support SparkSql enable users to flexibly respond to a variety of complex business needs. Whether it is data cleaning, data merging, data aggregation or data format conversion, users can process it through custom SparkSql expressions. In this way, data preprocessing during the import process is not limited to simple field mapping, but can dynamically adjust the structure of source data through complex SparkSql expressions to meet the needs of different business scenarios.
[0058] The data preprocessing process executes the custom SparkSql expression as follows:
[0059] if (UDFSqlConfig.sql != null && UDFSqlConfig.sql.nonEmpty) {
[0060] data.createTempView("exchange")
[0061] data = data.sparkSession.sql(UDFSqlConfig.sql)
[0062] }
[0063] When users use the import tool to customize data source preprocessing, they only need to add a custom expression configuration UDFSql in the configuration file, as follows:
[0064] pre_processes: {
[0065] UDFSql: select *, current_timestamp as new_col from exchange
[0066] }
[0067] like Figure 2 As shown, in this embodiment, the complete data preprocessing process is as follows: first, read and parse the configuration file, establish a connection with the data source and read the data from the data source, determine whether there is a data preprocessing configuration in the configuration file, if there is a data preprocessing configuration, traverse the data preprocessing configuration, and perform built-in preprocessing operations, including: filtering, splicing columns and splitting columns, etc., and then continue to determine whether there is a custom preprocessing configuration in the configuration file, that is, a custom SparkSql expression. If there is a custom preprocessing configuration, create an exchange view name for the data after the built-in preprocessing, and convert the custom SparkSql expression into an optimized physical execution task, which is executed by the Spark engine to complete the preprocessing of the imported data. If there is no custom preprocessing configuration, it means that the user has not configured a custom SparkSql expression, and directly enters the next step of data mapping and conversion; if there is no data preprocessing configuration, it means that there is no need to preprocess the source data, and directly enters the next step of data mapping and conversion.
[0068] S300, mapping the pre-processed data to be imported into a data format supported by the GQL graph database, and converting the mapped data into GQL statements for batch writing;
[0069] Specifically, converting the mapped data into a GQL statement for batch writing includes converting the mapped data into a GQL statement of an import point and a GQL statement of an import edge, wherein the process of converting the data into a GQL statement of an import point is as follows: The table header is constructed according to the source data schema column name, and the batch data set is constructed into a data format that meets the requirements of the graph database to piece together a two-dimensional table with a header and data. That is, the data type of each attribute is obtained by obtaining the metadata of the graph database, and the source data is converted according to the data type. That is, the string data type needs to be enclosed in double quotes for the source data, and the time type needs to be enclosed in date() for the source data, etc.
[0070] Traverse each row of data in the two-dimensional table, perform the insert operation, and concatenate the mapping relationship between the graph database attribute name and the two-dimensional table header in the insert statement. The mapping relationship is obtained through the mapping relationship configured in the imported configuration file.
[0071] The process of converting into a GQL statement for importing edges is as follows: The table header is constructed according to the source data schema column name, and the batch data set is constructed into a data format that meets the requirements of the graph database to piece together a two-dimensional table with a header and data. That is, the data type of each attribute is obtained by obtaining the metadata of the graph database, and the source data is converted according to the data type. That is, the string data type needs to be enclosed in double quotes for the source data, and the time type needs to be enclosed in date() for the source data, etc.
[0072] Traverse each row of data in the two-dimensional table, query the internal id of the source node according to the primary key data value of the source node, query the internal id of the target node according to the primary key data of the target node, perform the insert operation through the internal id of the source node and the internal id of the target node, and concatenate the mapping relationship between the graph database attribute name and the two-dimensional table header in the insert statement. The mapping relationship is obtained through the mapping relationship configured in the imported configuration file.
[0073] S400. Send the GQL statement to the graph database service, which executes the GQL statement to import data and returns the import result.
[0074] In this embodiment, the converted GQL statement is sent to the graph database service through the database client, and the import result is returned to the import tool, including whether the current data write request is successful, error information, execution delay of the request, and the amount of vertex and edge data affected by the database. The processing result of the current request is selectively printed in the import tool, including which graph database service executes the request, the execution delay, and the amount of data processed by the current request, so as to complete the data import process.
[0075] Whether the import tool prints the processing results of the request is controlled by the configuration. This control can be set by the user in the configuration file. The specific printing information is as follows:
[0076] write Person succeed, host{192.168.8.6:3820}, batch size(1892),latency(753384us), response(1031ms)
[0077] This application provides powerful data processing capabilities by introducing SparkSql, allowing users to customize data processing logic according to business needs during data import, greatly improving the flexibility and efficiency of data import. At the same time, this application supports multiple data sources and custom data sources, and has wide applicability.
[0078] Example 2
[0079] like Figure 8 As shown, a GQL graph database data import device based on SparkSql involved in Example 2 of the present application includes:
[0080] An acquisition module 100 is used to acquire data to be imported;
[0081] A preprocessing module 200 is used to preprocess the imported data according to preset data processing rules or custom SparkSql expressions;
[0082] The conversion module 300 is used to map the pre-processed data to be imported into a data format supported by the GQL graph database, and convert the mapped data into GQL statements for batch writing;
[0083] The import module 400 is used to send the GQL statement to the graph database service, and the graph database service executes the GQL statement to import data and return the import result.
[0084] It should be noted that other specific implementations of the GQL graph database data import device based on SparkSql in this embodiment can refer to the specific implementations of the GQL graph database data import method based on SparkSql mentioned above. To avoid redundancy, they will not be repeated here.
[0085] Example 3
[0086] A computer-readable storage medium according to Embodiment 3 of the present application, wherein the computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing a method in any one of the implementations in Embodiment 1 of the present application;
[0087] Among them, the computer-readable storage medium can be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM); the computer-readable storage medium can store program code, and when the program stored in the computer-readable storage medium is executed by the processor, the processor is used to execute the steps of the method in any one of the implementation methods in Example 1 of the present application.
[0088] Example 4
[0089] like Fig. 9 As shown, an electronic device involved in Embodiment 4 of the present application includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, a method in any one of the implementations in Embodiment 1 of the present application is implemented;
[0090] Among them, the processor can adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the method in any one of the implementation methods in Example 1 of the present application.
[0091] The processor may also be an integrated circuit electronic device with signal processing capability. In the implementation process, each step of the method in any implementation of Embodiment 1 of the present application may be completed by an integrated logic circuit of hardware in the processor or by instructions in software form.
[0092] The above-mentioned processor may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the functions required to be performed by the unit included in the data processing device of the embodiment of the present application in combination with its hardware, or executes the method in any one of the implementation modes in the embodiment 1 of the present application.
[0093] The above are only preferred specific implementations of the present application; however, the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and its improved ideas of the present application within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.
Claims
1. A method for importing GQL graph database data based on SparkSql, characterized in that: include: Get the data to be imported; Preprocess the imported data according to preset data processing rules or custom SparkSql expressions; Map the preprocessed data to be imported into a data format supported by the GQL graph database, and convert the mapped data into GQL statements for batch writing; The GQL statement is sent to the graph database service, which executes the GQL statement to import data and returns the import result.
2. The method for importing GQL graph database data based on SparkSql according to claim 1, characterized in that: Get the data to be imported, including: Obtain a configuration file, wherein the configuration file includes connection information of a data source, a database to be read, a data table, and a file path; A connection is established with the source data system according to the configuration information in the configuration file, and the specified data is read as the data to be imported.
3. The method for importing GQL graph database data based on SparkSql according to claim 2, characterized in that: Obtain the data to be imported, including: Define the configuration items of the new data source in the configuration file, wherein the configuration items include the data source category, link address, user name, password and reading concurrency, and the data source category is the fully qualified class name of the newly defined data source; Inherit the SourceConfigResolver abstract class and implement the getSourceConfigEntry method in the SourceConfigResolver abstract class to perform configuration parsing, which is used to parse the new data source configuration from the configuration file; Inherit the DataSourceReader abstract class in the import tool and implement the readData method in the DataSourceReader abstract class to read the new data source as the data to be imported.
4. The method for importing GQL graph database data based on SparkSql according to claim 1, characterized in that: The preset data processing rules include at least one of the following: Merge multiple columns into one; Split a column into multiple columns based on the specified delimiter; Filter data according to preset conditions; Concatenate a constant to the specified column; Use the specified constant as a new column of data.
5. The method for importing GQL graph database data based on SparkSql according to claim 4, characterized in that: Preprocessing the imported data according to the custom SparkSql expression includes: adding a configuration UDFSql of the custom SparkSql expression in the configuration file, converting the custom SparkSql expression into an optimized physical execution task, which is executed by the Spark engine to complete the preprocessing of the imported data.
6. The method for importing GQL graph database data based on SparkSql according to claim 1, characterized in that: The import result includes at least one of the following: whether the data writing request is successful, error information, delay in executing the requested operation, and the amount of vertex-edge data affected on the database.
7. The method for importing GQL graph database data based on SparkSql according to any one of claims 1 to 6, characterized in that: Also includes: Print the processing result of the graph database service, wherein the processing result includes at least one of the following: the graph database service that executes the GQL statement, the delay in executing the operation, and the amount of data processed by the current GQL statement.
8. A GQL graph database data import device based on SparkSql, characterized in that: include: The acquisition module is used to obtain the data to be imported; The preprocessing module is used to preprocess the imported data according to preset data processing rules or custom SparkSql expressions; The conversion module is used to map the preprocessed data to be imported into a data format supported by the GQL graph database, and convert the mapped data into GQL statements for batch writing; The import module is used to send the GQL statement to the graph database service, and the graph database service executes the GQL statement to import data and return the import result.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing the method according to any one of claims 1 to 7.
10. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the method as claimed in any one of claims 1 to 7 when executed by the processor.
Citation Information
Patent Citations
Data importing method and device for graph database
CN110110108A
Graph data processing method and device
CN110427359A
Data processing method and device based on Spark component, equipment and medium
CN113934752A
Method and device for importing and exporting multi-source heterogeneous data source and graph database
CN114461712A
Multi-source data importing method and device for graph database system
CN115658978A
Cited By
User batch initialization method for graph database document authentication system
CN121351119A
A user batch initialization method for a graph database document authentication system
CN121351119B