Data operation method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to the field of data processing, and specifically discloses a data operation method and device, electronic equipment and storage medium. The method comprises: obtaining a database identifier and a data table identifier contained in a data table operation request; obtaining database configuration information corresponding to the database identifier, and determining a front-end data node and a back-end data node corresponding to the data table identifier according to the database configuration information; obtaining data table attribute information corresponding to the data table identifier from the front-end data node; and performing data operation on a target data table corresponding to the data table identifier stored in the back-end data node according to the data table attribute information. This method greatly improves the operation efficiency of the data table, avoids the problem of manual input error, and improves accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, specifically to a data operation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, big data has become a technological hotspot in the industry, especially with the deployment of cloud computing services. As its most important business application, big data's development prospects are highly anticipated. The main characteristics of big data are its large volume, diverse types, high speed, and high value, while also being accompanied by multi-source, heterogeneous, and massive data characteristics.

[0003] To facilitate the storage of large amounts of data, various types of databases have emerged. Databases store massive amounts of data tables, and in some business scenarios, it is necessary to perform various types of data operations on the data tables already created in the database.

[0004] In related technologies, in order to perform data operations on an existing data table, the user needs to manually input various information about the created data table. This information is then used to locate the corresponding data table in the database and perform the relevant data operations. Therefore, in application scenarios with a large number of data tables, the above implementation method suffers from drawbacks such as cumbersome operation and low efficiency. Summary of the Invention

[0005] In view of the above problems, this disclosure is made in order to provide a data operation method, apparatus, electronic device and storage medium that overcomes or at least partially solves the above problems.

[0006] According to one aspect of the embodiments of this disclosure, a data manipulation method is provided, including:

[0007] Retrieve the database identifier and data table identifier contained in the data table operation request;

[0008] Obtain the database configuration information corresponding to the database identifier, and determine the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information;

[0009] Obtain the data table attribute information corresponding to the data table identifier from the front-end data node;

[0010] Based on the data table attribute information, perform data operations on the target data table stored in the backend data node that corresponds to the data table identifier.

[0011] In one optional implementation, the data operation includes: a write operation; the step of performing data operations on the target data table corresponding to the data table identifier stored in the backend data node according to the data table attribute information specifically includes:

[0012] Obtain the data to be written corresponding to the data table operation request; wherein, the data to be written includes: multiple fields to be written arranged in the order of the first field;

[0013] Based on the data table attribute information, determine the order of the second field corresponding to the target data table;

[0014] If the order of the second field does not match the order of the first field, the order of the multiple fields to be written is adjusted, and the multiple adjusted fields to be written are written into the target data table.

[0015] In one optional implementation, obtaining the data to be written corresponding to the data table operation request specifically includes:

[0016] Based on the database identifier, determine the target data format corresponding to the database identifier;

[0017] Obtain the original operation data corresponding to the data table operation request, and determine the original data format of the original operation data;

[0018] The original operation data is subjected to format conversion processing to obtain the data to be written that matches the target data format.

[0019] In one optional implementation, before performing data operations on the target data table corresponding to the data table identifier stored in the backend data node, the method further includes:

[0020] Obtain the partition adjustment parameters corresponding to the data table operation request, and determine the initial partition information of the target data table based on the data table attribute information;

[0021] Based on the adjustment type and virtual partition field included in the partition adjustment parameters, the initial partition information of the target data table is adjusted to obtain the adjusted partition information of the target data table.

[0022] The specific steps of performing data operations on the target data table corresponding to the data table identifier stored in the backend data node include: performing data operations on the target data table according to the adjusted partition information of the target data table.

[0023] In one optional implementation, adjusting the initial partition information of the target data table according to the adjustment type and virtual partition field included in the partition adjustment parameters to obtain the adjusted partition information of the target data table specifically includes:

[0024] Obtain the field value of the virtual partition field, and determine the target partition corresponding to the field value of the virtual partition field based on the partition configuration information contained in the data table attribute information;

[0025] The target partition is adjusted according to the adjustment type to obtain the adjusted partition information; wherein, the adjustment type includes: size adjustment type and / or data deletion type.

[0026] In one optional implementation, determining the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information specifically includes:

[0027] Based on the database configuration information, determine the cluster topology corresponding to the database identifier;

[0028] Based on the cluster topology, multiple front-end data nodes and multiple back-end data nodes corresponding to the database identifier are determined;

[0029] The process involves determining a front-end data node corresponding to the data table identifier from among the plurality of front-end data nodes, and determining a back-end data node corresponding to the data table identifier from among the plurality of back-end data nodes.

[0030] In one optional implementation, obtaining the data table attribute information corresponding to the data table identifier from the front-end data node includes:

[0031] Based on the cluster topology, obtain the node address and authentication information of the front-end data node corresponding to the data table identifier;

[0032] Based on the node address and authentication information, obtain the data table attribute information corresponding to the data table identifier from the front-end data node; wherein, the authentication information includes: account information and / or password information.

[0033] In one alternative implementation, the method further includes the following steps before execution:

[0034] Generate the database identifier based on the database type of the target database;

[0035] Configure and load the target database service corresponding to the database identifier;

[0036] The steps of obtaining the database identifier and the data table identifier contained in the data table operation request, and their subsequent steps, are executed through the target database service.

[0037] In one optional implementation, the target database is a distributed relational columnar database;

[0038] Furthermore, the target database service includes at least one of the following interfaces: an information acquisition interface, a data operation interface, and a partition operation interface;

[0039] The steps of obtaining database configuration information corresponding to the database identifier, determining the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information, and obtaining data table attribute information corresponding to the data table identifier from the front-end data node are implemented by calling the information acquisition interface.

[0040] The step of performing data operations on the target data table corresponding to the data table identifier stored in the backend data node is implemented by calling the data operation interface;

[0041] The step of adjusting the initial partition information of the target data table is implemented by calling the partition operation interface.

[0042] In one alternative implementation, the target database service includes: a plug-in service based on a batch processing computing framework; or, the target database service includes: an object-relational mapping framework service based on microservices.

[0043] According to another aspect of this disclosure, a data manipulation apparatus is provided, the apparatus comprising:

[0044] The identifier acquisition module is suitable for acquiring the database identifier and data table identifier contained in the data table operation request;

[0045] The configuration acquisition module is adapted to acquire database configuration information corresponding to the database identifier, and determine the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information.

[0046] The attribute acquisition module is adapted to acquire data table attribute information corresponding to the data table identifier from the front-end data node;

[0047] The data operation module is adapted to perform data operations on the target data table stored in the backend data node that corresponds to the data table identifier, based on the data table attribute information.

[0048] In one optional implementation, the data operation includes: a write operation; the data operation module is specifically adapted to:

[0049] Obtain the data to be written corresponding to the data table operation request; wherein, the data to be written includes: multiple fields to be written arranged in the order of the first field;

[0050] Based on the data table attribute information, determine the order of the second field corresponding to the target data table;

[0051] If the order of the second field does not match the order of the first field, the order of the multiple fields to be written is adjusted, and the multiple adjusted fields to be written are written into the target data table.

[0052] In one alternative implementation, the data manipulation module is specifically adapted to:

[0053] Based on the database identifier, determine the target data format corresponding to the database identifier;

[0054] Obtain the original operation data corresponding to the data table operation request, and determine the original data format of the original operation data;

[0055] The original operation data is subjected to format conversion processing to obtain the data to be written that matches the target data format.

[0056] In one alternative implementation, the data manipulation module is further adapted to:

[0057] Obtain the partition adjustment parameters corresponding to the data table operation request, and determine the initial partition information of the target data table based on the data table attribute information;

[0058] Based on the adjustment type and virtual partition field included in the partition adjustment parameters, the initial partition information of the target data table is adjusted to obtain the adjusted partition information of the target data table.

[0059] The specific steps of performing data operations on the target data table corresponding to the data table identifier stored in the backend data node include: performing data operations on the target data table according to the adjusted partition information of the target data table.

[0060] In one alternative implementation, the data manipulation module is specifically adapted to:

[0061] Obtain the field value of the virtual partition field, and determine the target partition corresponding to the field value of the virtual partition field based on the partition configuration information contained in the data table attribute information;

[0062] The target partition is adjusted according to the adjustment type to obtain the adjusted partition information; wherein, the adjustment type includes: size adjustment type and / or data deletion type.

[0063] In one alternative implementation, the configuration acquisition module is specifically adapted to:

[0064] Based on the database configuration information, determine the cluster topology corresponding to the database identifier;

[0065] Based on the cluster topology, multiple front-end data nodes and multiple back-end data nodes corresponding to the database identifier are determined;

[0066] The process involves determining a front-end data node corresponding to the data table identifier from among the plurality of front-end data nodes, and determining a back-end data node corresponding to the data table identifier from among the plurality of back-end data nodes.

[0067] In one alternative implementation, the attribute acquisition module is specifically adapted to:

[0068] Based on the cluster topology, obtain the node address and authentication information of the front-end data node corresponding to the data table identifier;

[0069] Based on the node address and authentication information, obtain the data table attribute information corresponding to the data table identifier from the front-end data node; wherein, the authentication information includes: account information and / or password information.

[0070] In one alternative implementation, the apparatus further includes:

[0071] The configuration loading module is adapted to generate the database identifier based on the database type of the target database; configure and load the target database service corresponding to the database identifier;

[0072] The database identifier and data table identifier included in the data table acquisition operation request, as well as the subsequent operations, are executed through the target database service.

[0073] In one optional implementation, the target database is a distributed relational columnar database;

[0074] Furthermore, the target database service includes at least one of the following interfaces: an information acquisition interface, a data operation interface, and a partition operation interface;

[0075] The operation of obtaining database configuration information corresponding to the database identifier, determining the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information, and obtaining data table attribute information corresponding to the data table identifier from the front-end data node is implemented by calling the information acquisition interface.

[0076] The operation of performing data operations on the target data table corresponding to the data table identifier stored in the backend data node is implemented by calling the data operation interface;

[0077] The operation of adjusting the initial partition information of the target data table is implemented by calling the partition operation interface.

[0078] In one alternative implementation, the target database service includes: a plug-in service based on a batch processing computing framework; or, the target database service includes: an object-relational mapping framework service based on microservices.

[0079] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0080] The memory is used to store at least one executable instruction that causes the processor to perform the data manipulation method described above.

[0081] According to another aspect of this disclosure, a computer storage medium is provided, the storage medium storing at least one executable instruction that causes a processor to perform the data manipulation method described above.

[0082] In the embodiments of this disclosure, database configuration information corresponding to the database identifier can be obtained based on the database identifier and data table identifier contained in the data table operation request. This allows for the determination of the front-end data node and back-end data node corresponding to the data table identifier. Data table attribute information corresponding to the data table identifier can then be obtained from the front-end data node. Based on this data table attribute information, data operations are performed on the target data table stored in the back-end data node corresponding to the data table identifier. Therefore, this method pre-configures the database configuration information corresponding to the database identifier. This database configuration information enables the rapid determination of the front-end and back-end data nodes corresponding to the data table identifier, and automatically retrieves the data table attribute information corresponding to the data table identifier from the front-end data node. Users do not need to manually input the node address of the front-end data node and the data table attribute information; the corresponding data table in the database can be automatically located directly based on the database identifier and data table identifier. This significantly improves the efficiency of data table operations, avoids errors caused by manual input, and enhances accuracy.

[0083] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0084] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this disclosure. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0085] Figure 1 A flowchart of a data manipulation method provided in one embodiment of this disclosure is shown;

[0086] Figure 2 A flowchart illustrating a data manipulation method provided by a specific example of this disclosure is shown;

[0087] Figure 3 A schematic diagram of a data manipulation apparatus provided in yet another embodiment of this disclosure is shown;

[0088] Figure 4 A schematic diagram of the structure of an electronic device provided in yet another embodiment of this disclosure is shown. Detailed Implementation

[0089] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0090] Figure 1 A flowchart illustrating a data manipulation method according to an embodiment of this disclosure is shown. Figure 1 As shown, the method includes:

[0091] Step S110: Obtain the database identifier and data table identifier contained in the data table operation request.

[0092] Data table operation requests are used to perform data operations on data tables. Specific data operation types include various types such as delete, write, and update operations.

[0093] In practice, data table operation requests can be generated using various syntax formats. For example, they can be implemented using Structured Query Language (SQL) statements. Alternatively, they can be implemented using other forms of program code. Regardless of the specific implementation form, the data table operation request must include parameter information indicating the database identifier and the data table identifier. The database identifier is used to determine the database where the data table resides, and the data table identifier is used to locate the specific data table.

[0094] Step S120: Obtain the database configuration information corresponding to the database identifier, and determine the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information.

[0095] Specifically, for each target database corresponding to a database identifier, corresponding database configuration information is pre-configured. This database configuration information stores various basic information such as the cluster topology and authentication information of the target database. The database configuration information can be maintained through a metadata service or a configuration center.

[0096] Because the database configuration information stores the cluster topology of the target database, it's possible to determine the front-end and back-end data nodes corresponding to the data table identifiers based on this configuration information. Within the database cluster, several front-end and back-end data nodes are configured to enable fast data access. The front-end data nodes (FE) are responsible for implementing the various query operations required for normal database operation; for example, they store the data table attribute information for each table. The back-end data nodes (BE) are responsible for storing the actual data content of each table.

[0097] Step S130: Obtain the data table attribute information corresponding to the data table identifier from the front-end data node.

[0098] By accessing the front-end data node, one can obtain the data table attribute information stored in the front-end data node, which corresponds to the data table identifier. This data table attribute information describes various attributes of the target data table corresponding to the data table identifier, specifically including: format information, number of fields and names of each field, order of multiple fields, partition settings, etc.

[0099] Whether you need to write data to a data table or delete data from a data table, you need to obtain the data table attribute information in advance to determine the storage format and storage order of each field in the data table in order to carry out the write or delete operation.

[0100] Step S140: Perform data operations on the target data table corresponding to the data table identifier stored in the backend data node according to the data table attribute information.

[0101] Specifically, based on the data table attribute information, it is possible to determine the number of fields, field order, field format, and other specific information of the target data table, thereby automatically performing data operations that match the target data table.

[0102] Therefore, this method pre-configures database configuration information corresponding to the database identifier. Through this database configuration information, the front-end data node and back-end data node corresponding to the data table identifier can be quickly identified, and the data table attribute information corresponding to the data table identifier can be automatically obtained from the front-end data node. Users do not need to manually input the node address of the front-end data node and the data table attribute information. The corresponding data table in the database can be automatically located directly based on the database identifier and the data table identifier, which greatly improves the efficiency of data table operation and avoids the problem of errors caused by manual input, thus improving accuracy.

[0103] In one optional implementation, the above data operation includes a write operation. This data operation method is mainly used to write data to a data table already created in the database. Since the data to be written may be obtained from a third-party data table, the field order in the user input data may differ from the field order in the target data table. In related technologies, the user needs to obtain the field order in the target data table before writing and adjust the user input data according to the field order in the target data table so that the adjusted field order of the user input data is the same as the field order of the target data table. However, the above method has the disadvantage of being cumbersome and inconvenient. To solve the above problem, in this embodiment, the field order of the data to be written can be automatically adjusted according to the field order corresponding to the target data table to improve the efficiency and accuracy of data writing.

[0104] For example, when performing data operations on a target data table corresponding to a data table identifier stored in a backend data node based on data table attribute information, the specific implementation is as follows: First, obtain the data to be written corresponding to the data table operation request; wherein, the data to be written includes multiple fields to be written arranged in the order of the first field. Then, determine the order of the second field corresponding to the target data table based on the data table attribute information. Finally, if the order of the second field does not match the order of the first field, adjust the order of the multiple fields to be written, and write the adjusted fields to the target data table. For example, suppose the multiple fields to be written include: first field, second field, and third field, and the order of the first field is: first field, third field, second field; suppose the order of the second field corresponding to the target data table is: first field, second field, third field, then the order of the multiple fields to be written needs to be adjusted so that the adjusted order of the multiple fields to be written is consistent with the order of the fields in the target data table. As can be seen, the above method can automatically adjust the order of fields to be written, eliminating the need for users to manually specify the content of each field, greatly reducing the amount of data that users manually enter, and avoiding the possibility of errors.

[0105] In another alternative implementation, due to the diverse sources of user input data, the data format of the user input data may not match the data format of the target database corresponding to the database identifier, leading to data writing errors. To solve this problem, in this embodiment, automatic format conversion can be used to match the format of the user input data with that of the target database. Accordingly, when obtaining the data to be written corresponding to the data table operation request, it is specifically implemented as follows: First, based on the database identifier, the target data format corresponding to the database identifier is determined. Then, the original operation data corresponding to the data table operation request is obtained, and the original data format of the original operation data is determined. Finally, format conversion processing is performed on the original operation data to obtain the data to be written that matches the target data format. The original operation data corresponding to the data table operation request is usually data input by the user through a preset interface. Since the original data format of the user-input original operation data is usually different from the target data format corresponding to the database identifier, format conversion processing can obtain the data to be written that matches the target data format.

[0106] In one optional implementation, considering that big data scenarios often involve partitioning operations, and traditional data manipulation methods do not support partitioning, a virtual partition field is provided in this embodiment to address the aforementioned issue. This virtual partition field can point to a specific partition, enabling operations on that partition. Furthermore, since the data table attribute information provided by the front-end data node includes the initial partition information of the target data table, the target partition pointed to by the virtual partition field can be quickly located based on the initial partition information, thus enabling partition adjustment operations on the target partition. Specifically, before performing data operations on the target data table corresponding to the data table identifier stored in the back-end data node, the following operations are further performed: First, the partition adjustment parameters corresponding to the data table operation request are obtained, and the initial partition information of the target data table is determined based on the data table attribute information. The partition adjustment parameters corresponding to the data table operation request can be included in the data table operation request itself; alternatively, the partition adjustment parameters corresponding to the data table operation request can also be included in the partition adjustment instruction associated with the data table operation request, which can be various instructions such as code instructions. In addition, the data table attribute information includes the initial partition information of the target data table. This initial partition information represents the number of partitions in the target data table, as well as the partition name, size, and other related information of each partition. Then, based on the adjustment type and virtual partition field included in the partition adjustment parameters, the initial partition information of the target data table is adjusted to obtain the adjusted partition information of the target data table. The adjustment types included in the partition adjustment parameters specifically include various types such as partition size adjustment, partition data deletion, and partition creation. Accordingly, when performing data operations on the target data table corresponding to the data table identifier stored in the backend data nodes, the data operations are specifically performed on the target data table based on the adjusted partition information.

[0107] In one specific implementation, when adjusting the initial partition information of the target data table based on the adjustment type and virtual partition field included in the partition adjustment parameters to obtain the adjusted partition information of the target data table, the following method is used: First, the value of the virtual partition field is obtained. Based on the partition configuration information contained in the data table attribute information, the target partition corresponding to the value of the virtual partition field is determined. The partition configuration information includes partition identifier information for each partition; accordingly, the partition whose partition identifier information matches the value of the virtual partition field is determined as the target partition. Then, the target partition is adjusted according to the adjustment type to obtain the adjusted partition information. Through this method, the virtual partition field can be used to perform partition adjustment operations, thus providing a foundation for subsequent data writing operations.

[0108] In one optional implementation, the determination of the front-end data nodes and back-end data nodes corresponding to the data table identifier based on the database configuration information is specifically achieved as follows: First, based on the database configuration information, the cluster topology corresponding to the database identifier is determined. The cluster topology characterizes the cluster architecture of the target database corresponding to the database identifier, such as the number and distribution of front-end data nodes and the number and distribution of back-end data nodes within the cluster. Then, based on the cluster topology, multiple front-end data nodes and multiple back-end data nodes corresponding to the database identifier are determined. Finally, the front-end data node corresponding to the data table identifier is determined from the multiple front-end data nodes, and the back-end data node corresponding to the data table identifier is determined from the multiple back-end data nodes. For example, the entry node of the target database is determined from the multiple front-end data nodes, and this entry node is used as the front-end data node corresponding to the data table identifier.

[0109] In another alternative implementation, to enhance database security, user authentication is required before database access to verify user security. In related technologies, users typically need to manually input authentication information, leading to complex input code and the vulnerability of authentication information in plaintext. To address these issues, this embodiment automatically retrieves authentication information based on the cluster topology, avoiding the cumbersome and easily leaked nature of manual input. Specifically, when retrieving data table attribute information corresponding to the data table identifier from the front-end data nodes, the following steps are taken: First, based on the cluster topology, the node address and authentication information of the front-end data node corresponding to the data table identifier are retrieved. The authentication information, used to authenticate the user accessing the front-end data node, includes account information and / or password information. The cluster topology stores the node addresses and authentication information of each front-end data node. Then, based on the node address and authentication information, the data table attribute information corresponding to the data table identifier is retrieved from the front-end data node. The data table attribute information describes various related information such as the number of fields, field names, data table source, data table purpose, and partitioning.

[0110] The data manipulation method in this embodiment can be applied to various database scenarios. To facilitate operations on a specified target database, a target database service needs to be pre-configured for the target database. This target database service is used to perform data operations on the data tables stored in the target database. Accordingly, before executing the data manipulation method in this embodiment, the following operations are further performed: generating a database identifier based on the database type of the target database; configuring and loading the target database service corresponding to the database identifier. In this approach, the steps of obtaining the database identifier and data table identifier contained in the data table operation request, and their subsequent steps, are executed through the target database service. That is, the execution subject in this embodiment is the target database service. Therefore, in the above approach, a database identifier corresponding to the target database is pre-defined, and a target database service corresponding to the database identifier is configured. Consequently, upon receiving a data table operation request containing the database identifier, the target database service can be automatically invoked to realize the operation function on the data tables in the target database.

[0111] In one optional implementation, the target database is a distributed relational columnar database. Accordingly, the target database service includes at least one of the following interfaces: an information retrieval interface, a data operation interface, and a partition operation interface. The information retrieval interface is used to obtain various information required for data operations based on database configuration information, including the node address of the front-end data node, authentication information, and other types of information. Accordingly, the steps of retrieving database configuration information corresponding to the database identifier, determining the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information, and retrieving the data table attribute information corresponding to the data table identifier from the front-end data node are implemented by calling the aforementioned information retrieval interface. The data operation interface is used to access the target data table in the target database to perform specific operations on the target data table, such as write operations and delete operations. Accordingly, the steps of performing data operations on the target data table corresponding to the data table identifier stored in the back-end data node are implemented by calling the data operation interface. The partition operation interface is used to manage the partition settings information in the target data table; for example, it can adjust the partitions of the target data table and delete partition data. Accordingly, the steps of adjusting the initial partition information of the target data table are implemented by calling the partition operation interface. By encapsulating different interfaces, the corresponding interfaces can be directly called to implement different functions, improving the convenience and accuracy of the operation.

[0112] In one optional implementation, the target database service includes a plug-in service based on a batch processing computing framework. For example, the batch processing computing framework can be various big data frameworks capable of batch processing, such as the Spark framework. Due to the diversity of database types, to facilitate the batch processing computing framework's flexible handling of various database types, a plug-in service is pre-loaded into the batch processing framework, and this plug-in service enables data operation functions targeting the target database. In another optional implementation, the target database service may further include an object-relational mapping framework service based on microservices. In summary, this application does not limit the specific implementation of the target database service, as long as it can achieve the function of performing data operations on the target database.

[0113] To facilitate understanding, the specific implementation details of the above embodiments are explained in detail below using two concrete examples. In the following two examples, the target database is a distributed relational columnar database, specifically a Doris database. The Doris database is a high-performance, real-time analytical database implemented based on an MPP architecture. The Doris database can store various types of business data, including but not limited to: business data from game applications, report analysis data from advertising applications, etc. Furthermore, since business data in big data scenarios is dynamically updated, different types of data operations need to be performed on the data tables in the Doris database according to the dynamic update status of the business data. For example, a large amount of new business data needs to be imported into the Doris database every day; therefore, data write operations or data deletion operations need to be performed on the data tables already created in the Doris database. The data import methods mainly include: a distributed computing-based import method (corresponding to the business scenario in Example 1 below) and a microservice single-node-based import method (corresponding to the business scenario in Example 2 below).

[0114] In related technologies, data import methods based on the Doris database mainly include the following: Broker asynchronous import method, Spark Load import method, and streaming synchronous import method. However, the above import methods generally have the following drawbacks: redundant import syntax, complex asynchronous query interactions, and a high risk of data errors. For example, in one implementation, when operating on Doris data tables using the Spark framework, users need to manually write the following code:

[0115] CREATE TEMPORARY VIEW spark_doris

[0116] USING doris

[0117] "table.identifier" = "$YOUR_DORIS_DATABASE_NAME.$YOUR_DORIS_TABLE_NAME", / / Enter the name of the data table;

[0118] "fenodes" = "$YOUR_DORIS_FE_HOSTNAME:$YOUR_DORIS_FE_RESFUL_PORT", / / Enter the node address of the front-end node;

[0119] "doris.write.fields" = "COL1,COL2,…", / / Enter the column names related to the data table so that you can operate on the columns;

[0120] "user"="$YOUR_DORIS_USERNAME",

[0121] "password"="$YOUR_DORIS_PASSWORD"

[0122] / / Enter username and password in plaintext

[0123] INSERT INTO spark_doris

[0124] VALUES("VALUE1","VALUE2",...); / / Write data to the data table stored on the backend node;

[0125] Therefore, it is evident that the above methods have at least the following problems:

[0126] (1) Users need to manually enter their account and password information, and the account and password information is explicitly configured. Therefore, not only is the operation cumbersome, but it is also easy to cause information leakage.

[0127] (2) Users need to manually input the field information of each field to be operated on in the data table, which makes the configuration of table meta information complex and redundant.

[0128] (3) The above methods do not support partitioning operations on data tables. Because the above methods do not include code definitions related to partitioning, operations such as partition adjustment and partition creation cannot be performed. However, partitioning functionality is crucial for the Online Analytical Processing (OLAP) engine.

[0129] In addition to the above code implementation methods, data manipulation functions can also be implemented based on the native streaming import interface in a microservice environment. However, this method is more complex and also suffers from the same three drawbacks mentioned above.

[0130] To address the aforementioned issues, this application provides two implementation methods as shown in Example 1 and Example 2 below:

[0131] Example 1

[0132] Example 1 provides a data import method based on distributed computing, primarily using a plugin service implemented based on a batch processing computing framework as the target database service mentioned above. This batch processing computing framework can include various big data frameworks, such as Spark, Flink, and DataX (a data synchronization framework that supports JSON configuration). In Example 1, the batch processing computing framework is Spark. Since Spark itself does not support data manipulation functions for the Doris database, related technologies require writing a large amount of complex code to enable operations on Doris database tables based on the Spark framework.

[0133] To address the aforementioned issues, this example pre-generates a database identifier corresponding to the Doris database based on its database type, and configures and loads the target database service corresponding to this identifier. This target database service is specifically a plugin service implemented based on the Spark framework (such as the Catalog plugin, which manages database metadata). This plugin service enables operations on data tables within the Doris database. Specifically, this plugin service implements at least the three types of interfaces mentioned above: information retrieval interface, data operation interface, and partition operation interface.

[0134] Figure 2 The diagram illustrates a data manipulation method provided in Example 1, as shown below. Figure 2 As shown, this example mainly includes the following steps:

[0135] Step S201: Receive a data table operation request and obtain the database identifier and data table identifier contained in the data table operation request.

[0136] For example, data table operation requests can be implemented using the following Spark SQL code statement:

[0137] INSERT INTO doris.alias_name.expamle_list_tbl

[0138] Here, `doris` is the database identifier, used to refer to the Doris database, and `alias_name` is used to identify the plugin service corresponding to the Doris database. For example, if the Doris database comprises multiple physical databases, different plugin service names can be configured for each physical database. Correspondingly, the plugin service corresponding to the physical database to be accessed can be called through `alias_name`. `expamle_list_tbl` is the table identifier, used to identify the specific data table. Therefore, a data table operation request must at least include the database identifier and the data table identifier to call the plugin service corresponding to the database and access the corresponding data table.

[0139] Step S202: Send a configuration information retrieval request containing the database identifier and the data table identifier to the configuration storage center corresponding to the database identifier.

[0140] Among them, the configuration storage center corresponding to the database identifier is used to uniformly store the configuration information corresponding to the target database, which can be a metadata service center.

[0141] Step S203: Obtain the database configuration information returned by the configuration storage center corresponding to the database identifier, and determine the front-end data node corresponding to the data table identifier based on the database configuration information.

[0142] The database configuration information returned by the configuration storage center corresponding to the database identifier includes at least: the cluster topology information of the database, the front-end data node (also called the entry data node) corresponding to the data table identifier, the node address of the front-end data node, and the authentication information corresponding to the front-end data node.

[0143] Step S204: Access the front-end data node corresponding to the data table identifier to obtain the data table attribute information corresponding to the data table identifier.

[0144] Specifically, based on the node address and authentication information of the front-end data node, the system accesses the front-end data node corresponding to the data table identifier to obtain the data table attribute information corresponding to the data table identifier. This data table attribute information is stored on the front-end data node and is used to characterize specific aspects of the data table, such as its structure, field features, and partitioning.

[0145] In other alternative implementations, the cluster topology information of the database can be stored on the front-end data nodes, and database-related content such as cluster topology information can be obtained from the front-end data nodes.

[0146] Step S205: Determine the backend data node corresponding to the data table identifier, and perform a partition adjustment operation on the target data table corresponding to the data table identifier stored in the backend data node.

[0147] The backend data node corresponding to the data table identifier can be determined directly based on the database configuration information corresponding to the database identifier returned by the configuration storage center, or it can be determined based on the cluster topology information stored in the frontend data node. The specific determination method depends on the storage location of the cluster topology information (e.g., it can be stored in the database configuration information or stored in the frontend data node). This application does not limit the specific determination method of the backend data node.

[0148] Since the backend data nodes are the physical nodes that actually store the target data tables, this step involves performing partition adjustment operations on the target data tables stored in the backend data nodes that correspond to the data table identifiers. Specifically, partition adjustment can be implemented as follows: First, obtain the partition adjustment parameters corresponding to the data table operation request. These parameters can be directly included in the data table operation request, or they can be included in the partition adjustment instruction associated with the request. This instruction can be a code instruction or other forms of instruction. Next, determine the initial partition information of the target data table based on the data table attribute information. This initial partition information characterizes the partition state of the target data table before data operations, including the number of partitions, the names of each partition, and other specific details. Finally, adjust the initial partition information of the target data table according to the adjustment type and virtual partition field included in the partition adjustment parameters to obtain the adjusted partition information. The adjustment type can be a partition change type (used for creating and deleting partitions) or a data deletion type (used to delete existing data in a partition for rewriting), etc.

[0149] Step S206: Based on the adjusted partition information of the target data table, perform data operations on the target data table stored in the backend data nodes that corresponds to the data table identifier.

[0150] Data operations are typically write operations. In practice, when there are multiple backend data nodes, a distributed write approach is needed to write data to each of the backend data nodes. Furthermore, during the write process, the format of user input data can be automatically adjusted according to the storage format of the target database. Additionally, the order of fields in the data to be written can be automatically adjusted based on the format of the target data table, ensuring the write process adapts automatically to the table format.

[0151] As can be seen in Example 1, the Catalog plugin enables operations such as reading basic metadata of a data table, partitioning, data deletion, and adaptive metadata writing, simplifying the data writing process and improving efficiency. This method significantly reduces code size, eliminating the need for users to manually input field information, usernames, passwords, etc. The Catalog plugin directly retrieves database configuration information, automatically enabling adaptive data writing. No additional metadata is required. Since manual coding is prone to errors, the more code there is, the greater the potential for error. For example, it's often difficult to distinguish between uppercase and lowercase letters, numbers, and similar letters. Furthermore, SQL code lacks flexible debugging tools; therefore, reducing code size significantly reduces the likelihood of errors.

[0152] Additionally, this example allows for optimization of some technical details in the code implementation process:

[0153] For example, in the specific implementation of this example, the return value of the data manipulation method can be set to values ​​such as ACCEPT_ANY_SCHEMA and V1_BATCH_WRITE. These values ​​are enumeration classes of the Spark interface, representing the capabilities supported by the plugin itself. By setting the enumeration values ​​to the above values, field adaptation and batch data writing functions can be achieved.

[0154] For example, Spark SQL, built upon Hive's database metadata model, uses multi-level custom partitioning. Doris, however, is more like a relational database, providing a fixed two-level partitioning. The second-level partition is a bucket partition, not directly involved in data operations; the first-level partition name needs to be globally unified and defaults to a fixed value. Furthermore, Hive table partitioning is user-defined. For instance, a game business table can be partitioned into three levels based on information such as "date," "game name," and "server," with each partition corresponding to its own fields, such as "logdate = 2023-01-01, game = game A, region = first server." However, Doris tables can only use single-level partitioning. Therefore, this embodiment uses a virtual partition field 'p' to represent the partition name, allowing the virtual partition field 'p' to flexibly correspond to multiple fields according to business needs, such as 'p = "2023-01-01|game A|first server". In other words, the virtual partition field can have multi-level values ​​to flexibly adapt to different partitioning methods and computational frameworks. Therefore, to address the issue arising from the difference in partitioning methods between SparkSQL and Doris databases, this embodiment proposes a virtual partition field. Furthermore, by setting the virtual partition field to a multi-level value, it can accurately point to the target partition under different partitioning methods, thus facilitating partition adjustment operations. Additionally, partitioned tables have certain unique characteristics when deleting data. For example, when deleting data from a partitioned table, the specific location of the partition must be specified. Therefore, in this embodiment, the partitioning condition is set using the following code: `where p = "2023-01-01|Game A|First Server"` to locate the partition position.

[0155] For example, many detailed characteristics need to be considered when streaming data to backend data nodes, such as read / write concurrency and data format control. In practice, an HTTP call with `expect 100` can be used to explicitly maintain a long-lived connection. Furthermore, dual-threading or multi-threading can be employed to improve CPU and I / O utilization. Additionally, parameters in the Doris database's streaming interface that conflict with the HTTP protocol can be discarded to avoid errors. Moreover, when streaming data to backend data nodes, it's necessary to use `Expect:100-continue` to establish a session and perform I / O in a separate thread, thereby increasing overall throughput while consuming the same amount of virtual resources. The specific reason is that after the operating system writes data, it takes time for the data to reach the Doris database via network devices. During this time, data format conversion and other computational tasks can be performed, avoiding CPU idle waiting and thus improving utilization.

[0156] Example 2

[0157] Example 2 provides a data import method implemented through microservices, primarily using an object-relational mapping (ORM) framework based on microservices to implement the target database service mentioned above. In Example 2, the ORM framework is mainly an ORM framework (a software framework used to bridge object-oriented languages ​​and relational databases). The main principle of Example 2 is similar to that of Example 1. This ORM framework also requires the implementation of at least the three types of interfaces mentioned above: information retrieval interface, data manipulation interface, and partitioning operation interface. Furthermore, these three types of interfaces can also be implemented using objects, depending on programming requirements; this application does not limit the specific implementation form.

[0158] The main difference between Example 2 and Example 1 lies in the specific implementation details. In ORM frameworks, to avoid mechanically writing large amounts of repetitive code, the framework provides code generation tools. For example, these code generation tools can be in the form of plugins. For instance, a code generation plugin might be used to read metadata, then generate and format the code.

[0159] In summary, after adopting the solutions in the two examples of this application, the data import-related indicators remained stable, and no issues such as data or partition misalignment or inconsistent query results occurred. Simultaneously, code production efficiency was improved, significantly reducing the workload of manual code writing and increasing write efficiency.

[0160] Figure 3 A schematic diagram of a data manipulation apparatus according to another embodiment of the present disclosure is shown. The data manipulation apparatus includes:

[0161] The identifier acquisition module 31 is adapted to acquire the database identifier and the data table identifier contained in the data table operation request;

[0162] The configuration acquisition module 32 is adapted to acquire database configuration information corresponding to the database identifier, and determine the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information.

[0163] The attribute acquisition module 33 is adapted to acquire data table attribute information corresponding to the data table identifier from the front-end data node;

[0164] The data operation module 34 is adapted to perform data operations on the target data table stored in the backend data node that corresponds to the data table identifier, based on the data table attribute information.

[0165] In one optional implementation, the data operation includes: a write operation; the data operation module is specifically adapted to:

[0166] Obtain the data to be written corresponding to the data table operation request; wherein, the data to be written includes: multiple fields to be written arranged in the order of the first field;

[0167] Based on the data table attribute information, determine the order of the second field corresponding to the target data table;

[0168] If the order of the second field does not match the order of the first field, the order of the multiple fields to be written is adjusted, and the multiple adjusted fields to be written are written into the target data table.

[0169] In one alternative implementation, the data manipulation module is specifically adapted to:

[0170] Based on the database identifier, determine the target data format corresponding to the database identifier;

[0171] Obtain the original operation data corresponding to the data table operation request, and determine the original data format of the original operation data;

[0172] The original operation data is subjected to format conversion processing to obtain the data to be written that matches the target data format.

[0173] In one alternative implementation, the data manipulation module is further adapted to:

[0174] Obtain the partition adjustment parameters corresponding to the data table operation request, and determine the initial partition information of the target data table based on the data table attribute information;

[0175] Based on the adjustment type and virtual partition field included in the partition adjustment parameters, the initial partition information of the target data table is adjusted to obtain the adjusted partition information of the target data table.

[0176] The specific steps of performing data operations on the target data table corresponding to the data table identifier stored in the backend data node include: performing data operations on the target data table according to the adjusted partition information of the target data table.

[0177] In one alternative implementation, the data manipulation module is specifically adapted to:

[0178] Obtain the field value of the virtual partition field, and determine the target partition corresponding to the field value of the virtual partition field based on the partition configuration information contained in the data table attribute information;

[0179] The target partition is adjusted according to the adjustment type to obtain the adjusted partition information; wherein, the adjustment type includes: size adjustment type and / or data deletion type.

[0180] In one alternative implementation, the configuration acquisition module is specifically adapted to:

[0181] Based on the database configuration information, determine the cluster topology corresponding to the database identifier;

[0182] Based on the cluster topology, multiple front-end data nodes and multiple back-end data nodes corresponding to the database identifier are determined;

[0183] The process involves determining a front-end data node corresponding to the data table identifier from among the plurality of front-end data nodes, and determining a back-end data node corresponding to the data table identifier from among the plurality of back-end data nodes.

[0184] In one alternative implementation, the attribute acquisition module is specifically adapted to:

[0185] Based on the cluster topology, obtain the node address and authentication information of the front-end data node corresponding to the data table identifier;

[0186] Based on the node address and authentication information, obtain the data table attribute information corresponding to the data table identifier from the front-end data node; wherein, the authentication information includes: account information and / or password information.

[0187] In one alternative implementation, the apparatus further includes:

[0188] The configuration loading module is adapted to generate the database identifier based on the database type of the target database; configure and load the target database service corresponding to the database identifier;

[0189] The database identifier and data table identifier included in the data table acquisition operation request, as well as the subsequent operations, are executed through the target database service.

[0190] In one optional implementation, the target database is a distributed relational columnar database;

[0191] Furthermore, the target database service includes at least one of the following interfaces: an information acquisition interface, a data operation interface, and a partition operation interface;

[0192] The operation of obtaining database configuration information corresponding to the database identifier, determining the front-end data node and back-end data node corresponding to the data table identifier based on the database configuration information, and obtaining data table attribute information corresponding to the data table identifier from the front-end data node is implemented by calling the information acquisition interface.

[0193] The operation of performing data operations on the target data table corresponding to the data table identifier stored in the backend data node is implemented by calling the data operation interface;

[0194] The operation of adjusting the initial partition information of the target data table is implemented by calling the partition operation interface.

[0195] In one alternative implementation, the target database service includes: a plug-in service based on a batch processing computing framework; or, the target database service includes: an object-relational mapping framework service based on microservices.

[0196] The specific structure and working principle of each of the above modules can be found in the description of the corresponding part of the method embodiment, and will not be repeated here.

[0197] Another embodiment of this application provides a non-volatile computer storage medium storing at least one executable instruction that can execute the object loading method in the virtual scene of any of the above method embodiments. Specifically, the executable instruction can be used to cause the processor to perform the corresponding operations in the above method embodiments.

[0198] Figure 4 The diagram shows a structural schematic of an electronic device according to yet another embodiment of the present disclosure. The specific embodiments of the present disclosure do not limit the specific implementation of the electronic device.

[0199] like Figure 4 As shown, the electronic device may include: a processor 502, a communications interface 506, a memory 504, and a communications bus 508.

[0200] in:

[0201] The processor 502, communication interface 506, and memory 504 communicate with each other via communication bus 508.

[0202] Communication interface 506 is used to communicate with other network elements such as clients or other servers.

[0203] The processor 502 is used to execute program 510, specifically to execute the relevant steps in the above-described video image detection method embodiment.

[0204] Specifically, program 510 may include program code that includes computer operation instructions.

[0205] Processor 502 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present disclosure. The electronic device includes one or more processors, which may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0206] Memory 504 is used to store program 510. Memory 504 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0207] Specifically, program 510 can be used to cause processor 502 to perform the corresponding operations in the above-described video image detection method embodiment.

[0208] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. Various general-purpose devices can also be used in conjunction with the teachings herein. The required structure for constructing such devices is apparent from the above description. Furthermore, this disclosure is not directed to any particular programming language. It should be understood that the contents of this disclosure described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this disclosure.

[0209] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0210] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the disclosure, in the above description of exemplary embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim.

[0211] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose.

[0212] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this disclosure and form different embodiments. For example, any of the claimed embodiments can be used in any combination.

[0213] The various component embodiments of this disclosure can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to embodiments of this disclosure. This disclosure can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such an implementation of this disclosure can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0214] It should be noted that the above embodiments are illustrative of this disclosure and not restrictive, and that alternative embodiments can be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This disclosure can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A data operation method, comprising: obtaining a database identifier and a data table identifier included in a data table operation request; obtaining database configuration information corresponding to the database identifier, and determining a front-end data node and a back-end data node corresponding to the data table identifier according to the database configuration information; obtaining data table attribute information corresponding to the data table identifier from the front-end data node; performing data operation on a target data table corresponding to the data table identifier stored in the back-end data node according to the data table attribute information; the obtaining of the data table attribute information corresponding to the data table identifier from the front-end data node comprises: obtaining a node address and authentication information of the front-end data node corresponding to the data table identifier according to a cluster topology structure corresponding to the database identifier; and obtaining the data table attribute information corresponding to the data table identifier from the front-end data node according to the node address and the authentication information; wherein the authentication information comprises account information and / or password information.

2. The method of claim 1, wherein, the data operation comprises a write operation; and the performing of the data operation on the target data table corresponding to the data table identifier stored in the back-end data node according to the data table attribute information comprises: obtaining to-be-written data corresponding to the data table operation request; wherein the to-be-written data comprises a plurality of to-be-written fields arranged in a first field order; determining a second field order corresponding to the target data table according to the data table attribute information; adjusting the field order of the plurality of to-be-written fields in a case where the second field order does not match the first field order, and writing a plurality of adjusted to-be-written fields into the target data table.

3. The method of claim 2, wherein, the obtaining of the to-be-written data corresponding to the data table operation request comprises: determining a target data format corresponding to the database identifier according to the database identifier; obtaining original operation data corresponding to the data table operation request, and determining an original data format of the original operation data; performing format conversion processing on the original operation data to obtain the to-be-written data matching the target data format.

4. The method of claim 2 or 3, wherein, before the performing of the data operation on the target data table corresponding to the data table identifier stored in the back-end data node, the method further comprises: obtaining partition adjustment parameters corresponding to the data table operation request, and determining initial partition information of the target data table according to the data table attribute information; adjusting the initial partition information of the target data table according to an adjustment type and a virtual partition field included in the partition adjustment parameters to obtain adjusted partition information of the target data table; and the performing of the data operation on the target data table corresponding to the data table identifier stored in the back-end data node comprises: performing the data operation on the target data table according to the adjusted partition information of the target data table.

5. The method of claim 4, wherein, The adjusting the initial partition information of the target data table according to the adjustment type and the virtual partition field in the partition adjustment parameter comprises: obtaining a field value of the virtual partition field, and determining a target partition corresponding to the field value of the virtual partition field according to partition configuration information contained in the data table attribute information; adjusting the target partition according to the adjustment type to obtain the adjusted partition information; wherein the adjustment type comprises a size adjustment type and / or a data deletion type.

6. The method of any one of claims 1-3, wherein, The determining the front-end data node and the back-end data node corresponding to the data table identifier according to the database configuration information comprises: determining a cluster topology structure corresponding to the database identifier according to the database configuration information; determining a plurality of front-end data nodes and a plurality of back-end data nodes corresponding to the database identifier according to the cluster topology structure; determining the front-end data node corresponding to the data table identifier from the plurality of front-end data nodes, and determining the back-end data node corresponding to the data table identifier from the plurality of back-end data nodes.

7. The method of any one of claims 1-3, wherein, Before the method is executed, the method further comprises: generating the database identifier according to a database type of a target database; configuring and loading a target database service corresponding to the database identifier; wherein the steps of obtaining the database identifier and the data table identifier contained in the data table operation request and subsequent steps are executed through the target database service.

8. The method of claim 7, wherein, The target database is a distributed relational columnar database; and the target database service comprises at least one of the following interfaces: an information obtaining interface, a data operation interface, and a partition operation interface; then the obtaining the database configuration information corresponding to the database identifier and determining the front-end data node and the back-end data node corresponding to the data table identifier according to the database configuration information; the step of obtaining the data table attribute information corresponding to the data table identifier from the front-end data node is implemented by calling the information obtaining interface; the step of executing the data operation on the target data table corresponding to the data table identifier stored in the back-end data node is implemented by calling the data operation interface; the step of adjusting the initial partition information of the target data table is implemented by calling the partition operation interface.

9. The method of claim 8, wherein, The target database service comprises a plug-in service based on a batch processing computing framework, or the target database service comprises an object-relational mapping framework service based on a micro-service.

10. A data operation apparatus, the apparatus comprising: an identifier obtaining module adapted to obtain a database identifier and a data table identifier contained in a data table operation request; a configuration obtaining module adapted to obtain database configuration information corresponding to the database identifier, and determine a front-end data node and a back-end data node corresponding to the data table identifier according to the database configuration information; An attribute obtaining module, adapted to obtain data table attribute information corresponding to the data table identifier from the front-end data node; A data operation module, adapted to perform data operation on a target data table corresponding to the data table identifier and stored in the back-end data node according to the data table attribute information; The obtaining of the data table attribute information corresponding to the data table identifier from the front-end data node comprises: obtaining a node address and authentication information of the front-end data node corresponding to the data table identifier according to a cluster topology structure corresponding to the database identifier; obtaining the data table attribute information corresponding to the data table identifier from the front-end data node according to the node address and the authentication information; wherein the authentication information comprises account information and / or password information.

11. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus; The memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the data operation method in any one of claims 1-9.

12. A computer storage medium, the storage medium storing at least one executable instruction, the executable instruction causing a processor to execute the data operation method in any one of claims 1-9.

Images