Database data processing method and device, equipment and storage medium

Abstract

Embodiments of the present application disclose a database data processing method, device and equipment and a storage medium. The method comprises: determining a data row to be processed according to a data processing request; querying the data row to be processed from at least one data node to obtain data correlation information of the data row to be processed; determining at least one target data node associated with the data row to be processed according to field position information in the data correlation information; wherein the field position information is pre-configured and generated in the process of creating a data partition to which the data row to be processed belongs; and calling the target data node to process the corresponding data row to be processed. The embodiments of the present application improve the database data processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a database data processing method, apparatus, device, and storage medium. Background Technology

[0002] In distributed databases, computing nodes typically distribute data to various data nodes based on predefined distribution routing rules. For example, in distributed scenarios, performing data queries, insertions, updates, and deletions often requires distribution routing calculations. The larger the data volume, the longer the distribution routing calculations take, especially for complex routing rules such as multiple distribution keys, function distribution keys, and multi-level distributions, which consume significant resources and reduce the efficiency of database data processing. Summary of the Invention

[0003] This invention provides a database data processing method, apparatus, device, and storage medium to improve database data processing efficiency.

[0004] According to one aspect of the present invention, a database data processing method is provided, the method comprising:

[0005] Based on the data processing request, determine the data rows to be processed;

[0006] Query the data row to be processed from at least one data node to obtain the data association information of the data row to be processed;

[0007] Based on the field location information in the data association information, at least one target data node associated with the data row to be processed is determined; wherein, the field location information is pre-configured and generated during the creation of the data partition to which the data row to be processed belongs;

[0008] The target data node is invoked to process the corresponding data row to be processed.

[0009] According to another aspect of the present invention, a database data processing apparatus is provided, the apparatus comprising:

[0010] The pending data row determination module is used to determine the pending data rows based on the data processing request.

[0011] The data association information determination module is used to query the data row to be processed from at least one data node to obtain the data association information of the data row to be processed;

[0012] The target data node determination module is used to determine at least one target data node associated with the data row to be processed based on the field location information in the data association information; wherein, the field location information is pre-configured and generated during the creation of the data partition to which the data row to be processed belongs;

[0013] The field processing module is used to call the target data node to process the corresponding data row to be processed.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the database data processing method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the database data processing method according to any embodiment of the present invention.

[0019] This invention's solution involves: determining a data row to be processed based on a data processing request; querying the data row from at least one data node to obtain its data association information; determining at least one target data node associated with the data row based on field position information within the data association information; wherein the field position information is pre-configured and generated during the creation of the data partition to which the data row belongs; and calling the target data node to process the corresponding data row. This technical solution, by pre-adding a virtual column for storing position information during the creation of the data partition (physical table), eliminates the need for subsequent data processing request execution. Instead, the field position information of the data row to be processed is directly obtained from the virtual column, reducing the time and resource consumption of distribution and routing calculations and improving database data execution efficiency in certain statement scenarios.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a database data processing method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of a database data processing device according to Embodiment 3 of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the database data processing method of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Example 1

[0028] Figure 1 This is a flowchart of a database data processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of distributed database data routing calculation without distribution. The method can be executed by a database data processing device, which can be implemented in hardware and / or software. The database data processing device can be configured in an electronic device, such as... Figure 1 As shown, the method includes:

[0029] S110. Determine the data rows to be processed based on the data processing request.

[0030] It's important to note that in a distributed database, there can be compute nodes for calculating distribution routing rules and data nodes for storing physical tables. In a distributed scenario, when executing data update statements, for simpler cases, compute nodes can be easily modified to directly forward the statements to data nodes for execution. However, in most cases, data needs to be fetched to compute nodes, which then calculate the data rows that need updating. Based on the data rows and distribution routing rules, they calculate the data node and data partition where the data rows reside. After determining the data node or data partition, the compute node distributes the statements to the corresponding data nodes to update the corresponding data rows. The calculation of distribution routing rules consumes a significant amount of time, substantially reducing execution efficiency.

[0031] Among them, data processing requests can be data update requests or data deletion requests initiated by the client to the distributed database.

[0032] The data rows to be processed can be fields to be updated or deleted. For example, the data processing request can be parsed to obtain the statement to be executed, and then the data rows to be processed can be determined based on the statement to be executed.

[0033] For example, if the data processing request is: UPDATA t1SET a=2WHERE b IN(SELECT b FROM t2), this update statement means: if field b in logical table t1 exists in logical table t2, then field a in logical table t2 will be updated to the value 2. The data rows that satisfy the WHERE condition are the data rows to be processed.

[0034] S120. Query the data row to be processed from at least one data node to obtain the data association information of the data row to be processed.

[0035] In a distributed database, there can be multiple data nodes, and each data node can correspond to at least one data partition for storing different data. A data partition, also known as a physical table, is the actual location area where the data is stored. Data association information can include primary key values ​​and field location information.

[0036] For example, a compute node can query a data row to be processed from at least one data partition of at least one data node and obtain data association information of the data row to be processed.

[0037] For example, suppose a distributed database has three data nodes: data node A, data node B, and data node C. Data node A has data partitions a1 and a2; data node B has data partition b1; and data node C has data partitions c1 and c2. The compute node sequentially queries the field to be queried from data partitions a1 and a2 of data node A, data partition b1 of data node B, and data partitions c1 and c2 of data node C, and obtains the data association information of the data row to be processed when the query is found.

[0038] It should be noted that when compute nodes perform distribution route calculations, they often determine which data node the data row to be processed belongs to, or which data partition within which data node it belongs to, so that the execution statement can be sent to the corresponding data node for execution. To achieve distribution-free route calculation, thereby reducing the amount of computation and improving execution efficiency, when creating data partitions in data nodes, a column can be added to the data partition to describe the logical and / or physical location of the data partition.

[0039] S130. Based on the field location information in the data association information, determine at least one target data node associated with the data row to be processed; wherein, the field location information is pre-configured and generated during the creation of the data partition to which the data row to be processed belongs.

[0040] The field location information can be the location information stored in a virtual location column created within the data partition when the data partition is created.

[0041] In one optional embodiment, the field location information is generated as follows: during the process of creating a data partition within a data node, a virtual location column is created for the data partition; wherein, the virtual location column is used to store the field location information; the field location information is associated with each field within the data partition.

[0042] The virtual location column can be a virtual column created within a data partition. It is a column of data that is not stored on disk, thus avoiding impact on data insertion performance. Furthermore, to avoid affecting query performance, the virtual location column is defined as an invisible column.

[0043] When creating a virtual location column, it's also necessary to define it, specifying its storage content and data type. This directly relates to the logical or physical location of the data partition. For example, in a sharded database architecture, the partition name describing the logical location can be used as the storage content, and the data type can be varchar.

[0044] For example, if there exist data node A and data node B, where data node A includes data partition a1 and data node B includes data partition b1, a virtual location column is created for data partition a1 during its creation to store the field location information of that data partition. For instance, the virtual location column for data partition a1 could store "a1, A", where "a1" represents the partition name and "A" represents the node name of the data node to which the partition belongs. Similarly, for data partition b1, the virtual location column could store "b1, B", where "b1" represents the partition name and "B" represents the node name of the data node to which the partition belongs.

[0045] Accordingly, in one optional embodiment, a data node includes at least one data partition; the data partition contains at least one field; querying the data row to be processed from at least one data node to obtain the data association information of the data row to be processed includes: querying and traversing the fields in each data partition of each data node to determine at least one data partition containing the data row to be processed, and determining the field position information in the virtual position column corresponding to each data partition as the field position information of the data row to be processed, thereby obtaining data association information including the field position information.

[0046] Each data node can include at least one data partition, which means it can include at least one physical table. Each database partition can contain at least one field and its corresponding value.

[0047] For example, after determining the data row to be processed, the computing node performs a query traversal of the fields in each data partition of each data node, and obtains the field position information from the virtual position column in at least one partition where the data row to be processed exists.

[0048] In one specific embodiment, if the client's database data execution requirement is to update field 'a' in logical table t1 to 2, the process involves traversing each data partition of each data node. When field 'a' is found in any data partition, the corresponding virtual location column is retrieved to determine its position. After querying all data nodes and data partitions, the physical location of the data row to be processed—that is, its data partition and data node—can be determined using the retrieved field location information. This process eliminates the need to calculate distribution routing rules, thereby improving the efficiency of determining the physical location of the data row to be processed.

[0049] In one optional embodiment, determining at least one target data node associated with the data row to be processed based on field location information in the data association information includes: determining the target data partition to which the data row to be processed belongs based on the field location information; and determining at least one target data node associated with the data row to be processed based on the target data partition.

[0050] It should be noted that the field location information can include both the partition name of the data partition and the name of the data node to which the data partition belongs; alternatively, the field location information can include only the partition name of the data partition. The data node to which the data partition belongs can also be determined through the partition name.

[0051] For example, the target data partition to which the data row to be processed belongs can be determined based on the field location information. There can be at least one target data partition. Based on each target data partition, the target data node to which each target data partition belongs can be determined, thereby identifying at least one target data node associated with the data row to be processed.

[0052] S140. Call the target data node to process the corresponding data row to be processed.

[0053] For example, after identifying the target data nodes, the corresponding target data nodes can be invoked to process the data rows to be processed. For instance, if the data rows to be processed involve target data nodes A, B, and C, then target data node A can be invoked to process the data rows to be processed in target data node A; target data node B can be invoked to process the data rows to be processed in target data node B; and target data node C can be invoked to process the data rows to be processed in target data node C.

[0054] In an optional embodiment, the data association information further includes a field primary key value; before calling each target data node to process the corresponding data row to be processed, the method further includes: generating at least one field execution instruction corresponding to the data row to be processed based on the field primary key value; calling the target data node to process the corresponding data row to be processed includes: calling the target data node to process the corresponding field execution instruction.

[0055] The key information for the fields in the data row to be processed also includes the primary key value of the field.

[0056] It should be noted that the execution statement initiated by the client may contain execution conditions, such as the "WHERE" statement. When execution conditions exist, the data row to be processed and the primary key values ​​of the data row to be processed are the fields and field values ​​that satisfy the execution conditions.

[0057] For example, a compute node can generate at least one field execution instruction corresponding to the data row to be processed by concatenating execution statements based on the primary key value of the field, and send each field execution instruction to the corresponding target data node for processing.

[0058] In one optional embodiment, determining the data row to be processed based on the data processing request includes: parsing the data processing request to obtain the data row to be processed, the request type, and the field processing conditions; correspondingly, generating at least one field execution instruction corresponding to the data row to be processed based on the field primary key value includes: selecting a target execution module from at least one candidate execution module based on the request type; and invoking the target execution module to generate at least one field execution instruction corresponding to the data row to be processed based on the field primary key value and the field processing conditions.

[0059] The request type can be the type of data processing request sent by the client. For example, the request type can be an update type or a delete type. An update type corresponds to a data update request sent by the client; a delete type corresponds to a data deletion request sent by the client.

[0060] Field processing conditions can be conditions included in the data processing request sent by the client, such as the conditions specified by the "WHERE" statement.

[0061] For example, a compute node can select a target execution module from at least one candidate execution module based on the request type. For instance, if it is a data update request, the target execution module could include a SELECT module and an UPDATE module. If it is a data deletion request, the target execution module could include a SELECT module and a DELETE module.

[0062] The compute node calls the query module to query the data row to be processed, and obtains the primary key value and field position information of the data row to be processed. The query module sends the obtained primary key value and field position information to the update module or the delete module. The update module or the delete module generates at least one field execution instruction corresponding to the data row to be processed based on the primary key value and field processing conditions, and sends each field execution instruction to the corresponding data node for execution.

[0063] It should be noted that the technical solution of this embodiment is applicable not only to distributed databases, but also to non-sharded databases, distributed middleware, and distributed storage.

[0064] This invention's solution involves: determining a data row to be processed based on a data processing request; querying the data row from at least one data node to obtain its data association information; determining at least one target data node associated with the data row based on field position information within the data association information; wherein the field position information is pre-configured and generated during the creation of the data partition to which the data row belongs; and calling the target data node to process the corresponding data row. This technical solution, by pre-adding a virtual column for storing position information during the creation of the data partition (physical table), eliminates the need for subsequent data processing request execution. Instead, the field position information of the data row to be processed is directly obtained from the virtual column, reducing the time and resource consumption of distribution and routing calculations and improving database data execution efficiency in certain statement scenarios.

[0065] Example 2

[0066] This embodiment also provides a preferred embodiment based on the above embodiments, and the execution process of the preferred embodiment is as follows:

[0067] If the client wants to perform a database data update operation.

[0068] S11. The client sends an update request; the execution statement corresponding to the update request is: UPDATE t1 SET a = 2 WHERE b IN (SELECT b FROM t2).

[0069] The meaning of this execution statement is: if data b in logical table t1 exists in logical table t2, then set a in t1 to 2.

[0070] S12. The compute node parses the update request and obtains the field to be updated, a, the update condition "b IN (SELECT b FROM t2)", etc.

[0071] S13. The compute node calls the SELECT module to execute the query statement: SELECT a, GDB FROM t1 WHERE b IN (SELECT b FROM t2);

[0072] GDB is a virtual column pre-created in each data partition (physical table) to store field location information. The compute node calls the SELECT module to query for conditions a and GDB that satisfy the "WHERE" condition.

[0073] S14. The compute node sends the query results to the UPDATE module;

[0074] S15. The UPDATE module infers the data partition and data node where the field row is located based on the field location information contained in the GDB column in the query results. Then, it combines the primary key value of the field to be updated to construct the update statement and sends it to the corresponding data node for execution.

[0075] If the client wants to perform a database data deletion operation.

[0076] S21. The client sends a delete (update) request; the execution statement corresponding to the delete request is: DELETE a FROM t1 WHERE b IN (SELECT b FROM t2).

[0077] The meaning of this execution statement is: if data b in logical table t1 exists in logical table t2, then delete a in t1.

[0078] S22. The compute node parses the delete request and obtains the field to be deleted, a, the update condition "b IN (SELECT b FROM t2)", etc.

[0079] S23. The compute node calls the SELECT module to execute the query statement: SELECT a, GDB FROM t1 WHERE b IN (SELECT b FROM t2);

[0080] GDB is a virtual column pre-created in each data partition (physical table) to store field location information. The compute node calls the SELECT module to query for conditions a and GDB that satisfy the "WHERE" condition.

[0081] S24. The compute node sends the query results to the DELETE module;

[0082] S25. The DELETE module infers the data partition and data node where the field row is located based on the field location information contained in the GDB column in the query results. Then, it combines the primary key value of the field to be updated to construct the DELETE statement and sends it to the corresponding data node for execution.

[0083] Example 3

[0084] Figure 2 This is a schematic diagram of a database data processing device provided in Embodiment 3 of the present invention. The database data processing device provided in this embodiment of the present invention is applicable to distributed database data routing calculations without data distribution. This database data processing device can be implemented in hardware and / or software, such as... Figure 2As shown, the device specifically includes: a data row determination module 201, a data association information determination module 202, a target data node determination module 203, and a field processing module 204. Among them,

[0085] The pending data row determination module 201 is used to determine the pending data row according to the data processing request;

[0086] The data association information determination module 202 is used to query the data row to be processed from at least one data node to obtain the data association information of the data row to be processed;

[0087] The target data node determination module 203 is used to determine at least one target data node associated with the data row to be processed based on the field position information in the data association information; wherein, the field position information is pre-configured and generated during the creation of the data partition to which the data row to be processed belongs;

[0088] The field processing module 204 is used to call the target data node to process the corresponding data row to be processed.

[0089] This invention's solution involves: determining a data row to be processed based on a data processing request; querying the data row from at least one data node to obtain its data association information; determining at least one target data node associated with the data row based on field position information within the data association information; wherein the field position information is pre-configured and generated during the creation of the data partition to which the data row belongs; and calling the target data node to process the corresponding data row. This technical solution, by pre-adding a virtual column for storing position information during the creation of the data partition (physical table), eliminates the need for subsequent data processing request execution. Instead, the field position information of the data row to be processed is directly obtained from the virtual column, reducing the time and resource consumption of distribution and routing calculations and improving database data execution efficiency in certain statement scenarios.

[0090] Optionally, the target data node determination module 203 includes:

[0091] The target data partition determination unit is used to determine the target data partition to which the data row to be processed belongs based on the field position information.

[0092] The target data node determination unit is used to determine at least one target data node associated with the data row to be processed based on the target data partition.

[0093] Optionally, the data association information further includes a primary key value; the device further includes:

[0094] The execution instruction generation module is used to generate at least one field execution instruction corresponding to the data row to be processed based on the field primary key value before the target data nodes are called to process the corresponding data row to be processed.

[0095] The field processing module 204 includes:

[0096] The field processing unit is used to call the target data node to execute instructions on the corresponding fields for processing.

[0097] Optionally, the data row determination module 201 includes:

[0098] The data row to be processed determination unit is used to parse the data processing request to obtain the data row to be processed, the request type, and the field processing conditions;

[0099] Accordingly, the execution instruction generation module includes:

[0100] The target execution module determination unit is configured to select a target execution module from at least one candidate execution module according to the request type.

[0101] The execution instruction generation unit is used to call the target execution module to generate at least one field execution instruction corresponding to the data row to be processed, based on the field primary key value and the field processing conditions.

[0102] Optionally, the field location information is generated in the following way:

[0103] During the creation of a data partition within a data node, a virtual location column is created for that data partition; wherein, the virtual location column is used to store field location information; the field location information is associated with each field within the data partition.

[0104] Optionally, the data node includes at least one data partition; the data partition contains at least one field;

[0105] The data association information determination module 202 includes:

[0106] The data association information determination unit is used to query and traverse the fields in each of the data partitions of each of the data nodes, determine at least one data partition where the data row to be processed exists, and determine the field position information in the virtual position column corresponding to each of the data partitions as the field position information of the data row to be processed, thereby obtaining data association information including the field position information.

[0107] The database data processing apparatus provided in the embodiments of the present invention can execute the database data processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0108] Example 4

[0109] Figure 3 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0110] like Figure 3 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0111] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0112] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as database data processing methods.

[0113] In some embodiments, the database data processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the database data processing method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the database data processing method by any other suitable means (e.g., by means of firmware).

[0114] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0118] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0119] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0120] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0121] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A database data processing method, characterized in that, include: Based on the data processing request, determine the data rows to be processed; Query the data row to be processed from at least one data node to obtain the data association information of the data row to be processed; Based on the field location information in the data association information, at least one target data node associated with the data row to be processed is determined; wherein, the field location information is pre-configured and generated during the creation of the data partition to which the data row to be processed belongs; The target data node is invoked to process the corresponding data row to be processed; The method for generating the field location information is as follows: During the creation of a data partition within a data node, a virtual location column is created for that data partition; wherein, the virtual location column is used to store field location information; the field location information is associated with each field within the data partition.

2. The method according to claim 1, characterized in that, The step of determining at least one target data node associated with the data row to be processed based on the field position information in the data association information includes: Based on the field location information, determine the target data partition to which the data row to be processed belongs; Based on the target data partition, determine at least one target data node associated with the data row to be processed.

3. The method according to claim 1, characterized in that, The data association information also includes a primary key value; before calling each of the target data nodes to process the corresponding data rows to be processed, it also includes: Based on the primary key value of the field, generate at least one field execution instruction corresponding to the data row to be processed; The step of calling the target data node to process the corresponding data row to be processed includes: The target data node is invoked to execute instructions for the corresponding fields.

4. The method according to claim 3, characterized in that, The step of determining the data row to be processed according to the data processing request includes: The data processing request is parsed to obtain the data row to be processed, the request type, and the field processing conditions; Accordingly, generating at least one field execution instruction corresponding to the data row to be processed based on the primary key value of the field includes: Based on the request type, a target execution module is selected from at least one candidate execution module; The target execution module is invoked to generate at least one field execution instruction corresponding to the data row to be processed, based on the primary key value of the field and the field processing conditions.

5. The method according to any one of claims 1-4, characterized in that, The data node includes at least one data partition; the data partition contains at least one field; The step of querying the data row to be processed from at least one data node to obtain the data association information of the data row to be processed includes: The fields in each data partition of each data node are queried and traversed to determine at least one data partition containing the data row to be processed. The field position information in the virtual position column corresponding to each data partition is determined as the field position information of the data row to be processed, thereby obtaining data association information including the field position information.

6. A database data processing device, characterized in that, include: The pending data row determination module is used to determine the pending data rows based on the data processing request. The data association information determination module is used to query the data row to be processed from at least one data node to obtain the data association information of the data row to be processed; The target data node determination module is used to determine at least one target data node associated with the data row to be processed based on the field location information in the data association information; wherein, the field location information is pre-configured and generated during the creation of the data partition to which the data row to be processed belongs; The field processing module is used to call the target data node to process the corresponding data row to be processed; The method for generating the field location information is as follows: During the creation of a data partition within a data node, a virtual location column is created for that data partition; wherein, the virtual location column is used to store field location information; the field location information is associated with each field within the data partition.

7. The apparatus according to claim 6, characterized in that, The target data node determination module includes: The target data partition determination unit is used to determine the target data partition to which the data row to be processed belongs based on the field position information. The target data node determination unit is used to determine at least one target data node associated with the data row to be processed based on the target data partition.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the database data processing method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the database data processing method according to any one of claims 1-5.

Images