Database-based data processing method and device and cluster

Abstract

The invention provides a data processing method and device based on a database and a cluster. The method comprises the steps that an access request is received, the access request is used for requesting to access target data in a data table, and the target data has a target affiliation identifier; when the plurality of affiliation identifiers comprise a target affiliation identifier, identifying a first partition corresponding to the target affiliation identifier in the plurality of partitions; and accessing the target data in the first partition based on the access request. The method can improve the access performance of the data table.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, and in particular to a database-based data processing method and device and cluster. BACKGROUND

[0002] In cloud services, a plurality of tenants share a data platform (referred to as data multi-tenancy) technology is a key technology to realize reasonable reuse of resources. In the data multi-tenancy technology, in order to improve resource utilization, database resources are reused among multiple tenants. Specifically, the data of multiple tenants is stored in the same data table of the database.

[0003] When a tenant accesses the data in the data table, the data table is locked. Since the data table stores the data of multiple tenants, locking the data table affects the access of other tenants to the data in the data table, resulting in a decline in the access performance of the data table. SUMMARY

[0004] The present application provides a database-based data processing method, device and cluster, which can improve the access performance of the data table.

[0005] In a first aspect, a database-based data processing method is provided. The database includes a data table, and the data table includes a plurality of partitions. The plurality of partitions and a plurality of ownership identifiers are in one-to-one correspondence. A partition is used to store data having an ownership identifier corresponding to the partition. The method includes receiving an access request, the access request being used to request access to target data in the data table, the target data having a target ownership identifier; when the plurality of ownership identifiers includes the target ownership identifier, identifying a first partition corresponding to the target ownership identifier among the plurality of partitions; and accessing the target data in the first partition based on the access request. Different partitions are physically isolated. In one example, the ownership identifier can be a tenant identifier, i.e., different ownership identifiers in the plurality of ownership identifiers are different tenant identifiers. In another example, the ownership identifier can be a data type, i.e., different ownership identifiers in the plurality of ownership identifiers correspond to different data types.

[0006] In this method, the data of different ownership identifiers is stored in different partitions in the data table. When accessing data having a certain ownership identifier, the data is accessed within the partition corresponding to the ownership identifier, rather than within the entire scope of the data table, so that the access of the data does not affect the data access of other partitions, improving the concurrency of data access of the data table and improving the access performance of the data table.

[0007] In one possible implementation, the first partition has an index, and the index is created based on the data in the first partition. Accessing the target data in the first partition based on the access request includes querying the target data in the first partition based on the index, and accessing the target data based on the access request.

[0008] The partition is indexed, the target data can be queried based on the index of the partition, and the query efficiency of the target data is improved. The partition of the partition is created based on the data in the partition, and is irrelevant to other partitions, so that inserting data or deleting data in the index only needs to lock the data in the partition, and does not affect other partitions.

[0009] In a possible implementation, the plurality of partitions and the plurality of storage spaces are in one-to-one correspondence, and the storage space corresponding to the partition is used to store data with a home identity corresponding to the partition; in the plurality of partitions, identifying a first partition corresponding to a target home identity comprises: in the plurality of storage spaces, identifying a storage space corresponding to the first partition; and based on the access request, accessing the target data in the first partition comprises: based on the access request, accessing the target data in the storage space corresponding to the first partition.

[0010] The data in different partitions is in different storage spaces, and physical isolation between the data in different partitions is achieved. The access to the data in the partition is performed in the storage space corresponding to the partition, does not affect other partitions, can improve the access concurrency of the data packet where the partition is located, and improves the access performance of the data packet.

[0011] In a possible implementation, the plurality of partitions and the plurality of files are in one-to-one correspondence, and the file corresponding to the partition is used to store data with a home identity corresponding to the partition; in the plurality of partitions, identifying a first partition corresponding to a target home identity comprises: in the plurality of files, identifying a file corresponding to the first partition; and based on the access request, accessing the target data in the first partition comprises: based on the access request, accessing the target data in the file corresponding to the first partition.

[0012] The data in different partitions is in different files, and physical isolation between the data in different partitions is achieved. The access to the data in the partition is performed in the file corresponding to the partition, does not affect other partitions, can improve the access concurrency of the data packet where the partition is located, and improves the access performance of the data packet.

[0013] In a possible implementation, the method further comprises: when the plurality of home identities does not include a target home identity, creating a second partition corresponding to the target home identity in the data table; and based on the access request, accessing the target data in the second partition.

[0014] When data of a new home identity needs to be stored, a partition is created in the data table, and the data is stored in the created partition. In this way, the existing partitions in the data table are not affected.

[0015] In a second aspect, a data processing device based on a database is provided, wherein the database includes a data table, the data table includes multiple partitions, the multiple partitions correspond one-to-one to multiple attribution identifiers, and the partitions are used to store data with attribution identifiers corresponding to the partitions; the device includes: a receiving module for receiving an access request, the access request is used to request access to target data in the data table, the target data having a target attribution identifier; an identification module for identifying a first partition corresponding to the target attribution identifier in multiple partitions when multiple attribution identifiers include a target attribution identifier; and an access module for accessing the target data in the first partition based on the access request.

[0016] In a possible implementation, the first partition has an index, which is created based on the data in the first partition; the access module is used to: query target data in the first partition based on the index; and access the target data based on the access request.

[0017] In one possible implementation, multiple partitions correspond to multiple storage spaces one-to-one, and the storage space corresponding to the partition is used to store data with an ownership identifier corresponding to the partition; the identification module is used to: identify the storage space corresponding to the first partition among multiple storage spaces; and the access module is used to: access target data in the storage space corresponding to the first partition based on an access request.

[0018] In one possible implementation, multiple partitions correspond to multiple files one-to-one, and the files corresponding to the partitions are used to store data with an attribution identifier corresponding to the partitions; the identification module is used to: identify the file corresponding to the first partition among multiple files; and the access module is used to: access target data in the file corresponding to the first partition based on an access request.

[0019] In one possible implementation, the device also includes: a creation module; the creation module is used to: when the multiple attribution identifiers do not include the target attribution identifier, create a second partition corresponding to the target attribution identifier in the data table; the access module is also used to: access the target data in the second partition based on the access request.

[0020] In a possible implementation, different attribution identifiers among the multiple attribution identifiers are different tenant identifiers, or different attribution identifiers among the multiple attribution identifiers correspond to different data types.

[0021] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method provided in the first aspect.

[0022] In a fourth aspect, a computer-readable storage medium is provided, including computer program instructions, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method provided in the first aspect.

[0023] In a fifth aspect, a computer program product including instructions is provided, when the instructions are run by a computing device cluster, the computing device cluster executes the method provided in the first aspect.

[0024] The beneficial effects of the second aspect to the fifth aspect can be referred to the beneficial effects of the first aspect described above, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A schematic diagram of a system architecture provided by an embodiment of the present application is shown in the figure;

[0026] Figure 2 A schematic diagram of a data table provided by an embodiment of the present application is shown in the figure;

[0027] Figure 3 A schematic diagram of a data table provided by an embodiment of the present application is shown in the figure;

[0028] Figure 4 An index creation schematic diagram provided by an embodiment of the present application is shown in the figure;

[0029] Figure 5 An index creation schematic diagram provided by an embodiment of the present application is shown in the figure;

[0030] Figure 6 A flowchart of a data processing method provided by an embodiment of the present application is shown in the figure;

[0031] Figure 7 A schematic diagram of a data processing method provided by an embodiment of the present application is shown in the figure;

[0032] Figure 8 A schematic diagram of a data processing method provided by an embodiment of the present application is shown in the figure;

[0033] Figure 9 A structural schematic diagram of a data processing apparatus provided by an embodiment of the present application is shown in the figure;

[0034] Figure 10 A structural schematic diagram of a computing device provided by an embodiment of the present application is shown in the figure;

[0035] Figure 11 A structural schematic diagram of a computing device cluster provided by an embodiment of the present application is shown in the figure;

[0036] Figure 12 A structural schematic diagram of a computing device cluster provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0037] The solutions provided by the embodiments of the present application will be described below with reference to the drawings. In the embodiments of the present application, “multiple” refers to two or more, and “multiple types” refers to two or more types. “First”, “second”, and the like are only used to distinguish similar objects, and do not necessarily indicate a specific order or number of objects.

[0038] To facilitate understanding of the solutions provided by the embodiments of the present application, the technical terms that may be involved in the embodiments of the present application will be introduced first.

[0039] Data table: a table participating in data storage or management in a database. A data table has a table header and the like. The database performs metadata management and template management in the granularity of a data table. That is, the same data table corresponds to the same metadata and the same template, and different data tables correspond to different metadata and different templates.

[0040] Logical data table: also referred to as a logical table, is a kind of data table in a database. A logical data table is a virtual table and does not store data.

[0041] Physical data table: also referred to as a physical table, is a table that can store data in a database. Data can be recorded in a physical data table to store the data in the physical data table.

[0042] Template: refers to a statement template, including an insertion statement template, a query statement template, and the like. Taking a query statement template as an example, when querying target data, a constant in the query statement template is modified into a variable related to the target data, and a query statement for querying the target data is obtained.

[0043] Metadata of a data table: refers to data describing a data table. The metadata of a data table can include description information of a table header of the data table and the like.

[0044] Ownership identifier: is an identifier used to indicate to which tenant or which type of data the data belongs. The ownership identifier can be a tenant identifier, used to indicate that the data belongs to a tenant corresponding to the tenant identifier. The ownership identifier can be an identifier of a data type, used to indicate that the data is data of the data type. For example, the data type can include write-intensive and read-intensive.

[0045] Flexible field: refers to a table header with a non-fixed meaning in a data table. When a table header of a data table is a flexible field, the meaning of the table header under different tenants can be different.

[0046] The three common modes of cloud services are infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Among them, the infrastructure and application management of SaaS are provided by the cloud service provider, which enables the cloud service provider to reuse resources and solutions among tenants, thereby greatly reducing the operating costs of the provider and the costs of tenants renting cloud services.

[0047] Multi-tenant shared data platform technology, referred to as data multi-tenancy technology, can achieve reasonable reuse of resources, thereby achieving the effect that the more tenants, the lower the cost. For a database management system (DMBS), reasonable reuse of resources includes data table reuse, that is, storing data of multiple tenants in the same data table. The multi-tenant reuse of the data table is implemented through row-level data isolation.

[0048] In order to implement data table reuse, a unique tenant identifier (renter ID) is allocated to each tenant, and the table and field of each tenant are virtual. Whenever a new tenant is added, the database management system does not create a new data table for the tenant in the database (DB), but inserts the data of the tenant and the tenant identifier of the tenant into the existing data table. When the data of a tenant needs to be queried, the tenant identifier of the tenant is used as a filter condition to query the data of the tenant in the data table. Moreover, when a tenant adds a field, the database management system does not add a field in the data table, but establishes a mapping relationship between the field to be added by the tenant and the flexible field of the data table, and the column corresponding to the flexible field to which the field to be added by the tenant is mapped is used to record the data of the field to be added by the tenant. The mapping relationship between the field to be added by the tenant and the flexible field of the data table can be recorded in a metadata engine.

[0049] In related technologies, in order to avoid conflicts caused by multiple tenants accessing data in a data table at the same time, when a tenant accesses data in a data table, the data table is locked, and the lock only allows the current tenant to access the data table and prohibits other tenants from accessing the data table. Therefore, multiple tenants can only access the data table in series, resulting in low access efficiency and poor performance of the data table.

[0050] In particular, in order to guarantee the efficiency of tenant accessing data, tenant-level indexes of data tables are created. Different tenants in the same data table have different requirements for the number of indexes, flexible field types, etc. The first flexible field of tenant 1 is of timestamp type, which requires range condition query. The first flexible field of tenant 2 is of text type, which requires creation of an inverted index suitable for full-text search. The first flexible field of tenant 3 is just a descriptive field and does not require indexing. Therefore, for a multi-tenant shared data table, when creating an index, tenant data needs to be isolated, that is, a tenant-level index only takes effect on data of a specific value of the tenant.

[0051] In one scheme, a tenant-level predicate partial index is created for a data table. Specifically, an equality filter condition on a tenant identification field is specified when creating the index, so that data with the same tenant identification is inserted into the same index. In this scheme, when tenant 1 data needs to be inserted into the data table, all indexes of the data table for all tenants need to be traversed, and it is determined in turn whether the traversed index belongs to tenant 1. The process of traversing all tenant indexes requires locking all tenant indexes and reading the metadata of the indexes. In addition, when tenant 1 data needs to be queried in the data table, all tenant indexes also need to be traversed, and the tenant identification of tenant 1 is compared with the predicates of all partial predicate indexes. When a predicate that identifies the same tenant identification as tenant 1 is obtained, the corresponding index path is generated. This process also requires locking all tenant indexes and reading the metadata of the indexes.

[0052] Locking all tenant indexes causes all tenant indexes to be inaccessible to tenants other than tenant 1, which reduces data access performance. Moreover, as the number of tenants in the data table increases, data access performance will continue to decline. Specifically, locking all tenant indexes requires a large number of lock resources. Traversing all tenant indexes and reading metadata requires caching all tenant indexes and metadata, which is likely to cause index cache and metadata cache to be hit, resulting in a sharp decline in performance.

[0053] In addition, this scheme cannot take advantage of the template general plan of database precompiled statements to optimize query performance. Specifically, a statement precompilation interface is usually provided by a database to improve the execution performance of high-frequency template statements. If the access request of a client hits a compiled statement template, the cached general execution plan can be directly executed, thereby realizing lightweight structured query language (SQL) parsing. However, for a predicate partial index, the general plan cannot determine in the plan generation stage whether the generated index plan is suitable for the actual called tenant identification.

[0054] In addition, the tenant data migration capability is poor in the scheme. Since the tenant data uses function row-level isolation, the tenant data is cross-distributed in different data files. When the tenant data is cleaned and migrated, multiple data files need to be scanned and modified row by row, which is high in cost.

[0055] In another scheme, after the index field of the tenant in the data table is converted into the target data type, indexes of the same target data type of the field are copied into the same perspective table, and indexes of different target data types of the field are copied into different perspective tables. When data in the data table is accessed, such as added, deleted, or modified, the business arrangement layer expands a single data access operation statement into multiple maintenance statements of the data table and the perspective table, and places the multiple maintenance statements in the same transaction block to ensure the atomicity between the data table and the perspective table. When data in the data table is queried, the business arrangement layer parses the query task in the data table into a query statement of the perspective table according to the semantic of the tenant query, and performs data query by using the index.

[0056] The scheme needs the business arrangement layer to expand the statement and parse the task, which is high in complexity. In addition, indexes of different target data types of the field are copied into different perspective tables, the target data type of the field of the index of the tenant is used to identify the perspective table corresponding to the tenant, which is tedious, and different perspective tables need different statement expansion functions and task parsing functions, that is, the business arrangement layer needs to adapt to different perspective tables, which results in high expansion cost and parsing cost. In addition, since the data table can need multiple perspective tables, the number of system objects and templates of the database doubles, which increases the cache pressure and causes the concurrent performance to decline.

[0057] Embodiments of the present application provide a database-based data processing method. The method can set a data table in a database, and divide the data table into multiple partitions, which are physically isolated between different partitions. Each partition is used to store data under the same ownership identifier. When target data is accessed, the partition where the target data is located can be obtained based on the ownership identifier of the target data, and then the target data is accessed in the partition. In this way, data is accessed based on the partition, when data is accessed in a certain partition, other partitions are not affected by the data access, and data access can be performed in parallel between different partitions, thereby improving the concurrency of data access of the data table and improving the access performance of the data table.

[0058] Next, the method provided by the embodiments of the present application will be specifically introduced.

[0059] Figure 1A system architecture that can be used to implement the method is shown. The system architecture includes a server 100 and a client 200. Among them, the server 100 is deployed with a database 300. Among them, the database 300 can include at least one data table, such as data table 310, data table 320, etc. Among them, the database 300 further includes a database management system 330. The database management system 330 can manage the data tables in the database 300. The client 200 can issue an access request, and the database management system 330 can receive the access request and perform relevant data access based on the access request.

[0060] The data tables in the database 300 can be divided into multiple partitions. Referring to Figure 2 Taking the data table 310 as an example, the data table 310 can be divided into partitions 311, 312, etc. Each partition corresponds to an ownership identifier, and the partition is used to store data with the ownership identifier corresponding to the partition. Different partitions of the same data table are physically isolated. For example, different partitions are stored in different storage spaces. For example, different partitions are stored in different files. In this way, access to one partition will not affect another partition, and data access can be performed in parallel between different partitions, thereby improving the access performance of the data table.

[0061] Among them, the data table 310 presented to the tenant, therefore, the data actually accessed by the tenant is in the partition 311 or in the partition 312, the tenant can initiate an access request for the data table 310, without the tenant identifying the partition where the target data of the access request is located. Therefore, the user experience is improved.

[0062] And, the partitions 311, 312, etc. all belong to the data table 310, therefore, the partitions 311, 312, etc. share the metadata and templates of the data table 310, etc., so that it is not necessary to manage the metadata and templates for each partition separately, saving the management overhead of the database.

[0063] Referring to Figure 3The data table 310 is partitioned according to the row. In some embodiments, the server 200 is deployed with an objective query language (OQL) engine. The OQL engine can issue a multi-tenancy system initialization statement, which instructs the database management system 330 to create the data table 310, the data table 320, and the like in the database 300, and instructs the database management system 330 to partition the data table 310, the data table 320, and the like according to the home identity. The data table 310 can be one of a tenant entity table and a tenant extension table, and the data table 320 can be the other of the tenant entity table and the tenant extension table. The tenant entity table refers to a data table with a fixed table header meaning, i.e., the meaning of the table header is the same for different tenants. The tenant extension table refers to a data table with a flexible table header, i.e., the meaning of the table header can be different for different tenants.

[0064] In one example, the data table 310 can be partitioned in a list partition manner. For example, even rows in the data table 310 are mapped to the home identity A1, and odd rows are mapped to the home identity A2. The even rows of the data table 310 form a partition corresponding to the home identity A1, and the odd rows of the data table 310 form a partition corresponding to the home identity A1. In one example, the data table 310 can be partitioned in a range partition manner, e.g., rows 1-100 in the data table 310 are divided into the partition 311, and rows 101-200 in the data table 310 are divided into the partition 312.

[0065] In some embodiments, the data table 310 can be a logical table, and the partition 311 and the partition 312 can be physical tables mapped to the data table 310.

[0066] In some embodiments, the data table 310 can be referred to as a collection of physical tables, and the physical tables in the collection share metadata and templates, and the like. The partition 311 and the partition 312 are physical tables in the collection.

[0067] In some embodiments, an index can be created based on data in a partition. Since different partitions are independent of each other, when creating an index based on data in a partition, only the current partition needs to be locked, and other partitions do not need to be locked. In this way, other partitions are not affected, and the business of other partitions is not blocked.

[0068] In some embodiments, the data table 310 can include a flexible field. When creating an index for the flexible field, the partition in the data table 310 converts the expression index according to the data type of the flexible field in the current partition, without considering the data type of the flexible field in other partitions. The data type corresponding to the flexible field or the table header is the data in the column where the flexible field or the table header is located.

[0069] In one example, it can be set that the table header V1, the table header V2 and the table header V4 of the data table 310 are flexible fields. It can also be set that the data type corresponding to the table header V1 in the partition 311 is int8, the data type of the data corresponding to the table header V2 is timestamp, and the data type corresponding to the table header V4 is numeric. For the partition 311, the indexes 3111 and 3112 as shown in Figure 4 may be generated.

[0070] In one example, it can be set that the table header V1 and the table header V2 of the data table 310 are flexible fields. It can also be set that the data type corresponding to the table header V1 in the partition 312 is timestamp, and the data type of the data corresponding to the table header V2 is varchar. For the partition 312, the index 3121 as shown in Figure 5 may be generated.

[0071] In some embodiments, the ownership identifier can be a data type. Generally, the data type herein includes read-intensive and write-intensive. Wherein, when the ratio of the occurrence frequency of the read operation on the data to the occurrence frequency of the write operation on the data is greater than a preset threshold, the data type of the data is read-intensive. When the ratio of the occurrence frequency of the write operation on the data to the occurrence frequency of the read operation on the data is greater than a preset threshold, the data type of the data is write-intensive.

[0072] An index corresponding to the partition and adaptive to the ownership identifier of the partition can be created. For example, the ownership identifier corresponding to the partition is read-intensive, and a read-friendly index is created for the partition. The read-friendly index refers to an index suitable for read operation, so that the read size, read latency and read operation experience are good. Common read-friendly indexes include indexes in a log structured merge tree (LSM tree) structure. For another example, the ownership identifier corresponding to the partition is write-intensive, and a write-friendly index is created for the partition. The write-friendly index refers to an index suitable for write operation, so that the write size, write latency and write operation experience are good. Common write-friendly indexes include indexes in a B+ tree structure.

[0073] The above examples introduce the system architecture provided by the embodiments of the present application. Next, the data processing method provided by the embodiments of the present application is described in combination with the system architecture.

[0074] The method can be performed by the server 100. For example, the method can be performed by the database management system 330 in the server 100. As described above, the database 300 deployed in the server 100 includes a data table 310, where the data table 310 includes a plurality of partitions 311 and 312, and the plurality of partitions and a plurality of home identifiers are in one-to-one correspondence, and each partition is configured to store data with a home identifier corresponding to the partition. Figure 6 The method includes the following steps.

[0075] At step 601, an access request is received, where the access request is configured to request to access target data in the data table 310, and the target data has a home identifier A1.

[0076] The client 200 can send the access request in response to the operation of the tenant. The access request includes relevant information of the data to be accessed by the access request, where the relevant information includes the home identifier A1, and the home identifier A1 is a home identifier of the data to be accessed by the access request. The data to be accessed by the access request can be referred to as target data.

[0077] As described above, the server 100 presents the data table 310 to the client 200, so that the client 200 can send the access request to the data table 310, that is, send the access request to access the data in the data table 310.

[0078] In some embodiments, the access request can be an insertion request or a write request, configured to insert the target data. In some embodiments, the access request can be a read request, configured to read the target data. In some embodiments, the access request can be a modification request, configured to modify the target data. In some embodiments, the access request can be a deletion request, configured to delete the target data.

[0079] In some embodiments, the home identifier can be a tenant identifier of the tenant, that is, the data with the home identifier refers to the data of the tenant corresponding to the home identifier.

[0080] In some embodiments, the home identifier can be a data type. Generally, the data type here includes read-intensive and write-intensive.

[0081] At step 602, it is determined whether the plurality of home identifiers includes the home identifier A1.

[0082] The multiple home identities are home identities corresponding to the multiple partitions. The home identity A1 is in the multiple home identities, i.e., the multiple home identities include the home identity A1, which indicates that the data table 310 includes the partition corresponding to the home identity A1. When the home identity A1 is not in the multiple home identities, i.e., the multiple home identities do not include the home identity A1, it indicates that the data table 310 does not include the partition corresponding to the home identity A1.

[0083] In some embodiments, when the data table 310 is partitioned based on the home identity, the home identity can be recorded in the list B1, so that the list B1 records the multiple home identities. In step 602, it can be queried whether the home identity A1 is in the list B1. If the home identity A1 is in the list B1, the multiple home identities include the home identity A1. If the home identity A1 is not in the list B1, the multiple home identities do not include the home identity A1.

[0084] When the multiple home identities include the home identity A1, step 603 is performed to identify the partition 311 corresponding to the home identity A1 in the multiple partitions.

[0085] In some embodiments, the correspondence between the partition and the home identity corresponding to the partition can be recorded in the list B2. For example, when the partition 311 is created based on the home identity A1, the correspondence between the home identity A1 and the partition 311 can be recorded in the list B2. In step 603, the partition corresponding to the home identity A1, i.e., the partition 311, can be identified based on the list B2.

[0086] In some embodiments, the data in different partitions of the same data table are respectively stored in different storage spaces, in other words, the multiple partitions in the data table 310 and the multiple storage spaces are in one-to-one correspondence, and the storage space corresponding to the partition is used to store the data having the home identity corresponding to the partition. That is, the data in the partition is actually stored in the storage space corresponding to the partition. In step 603, the partition 311 corresponding to the home identity A1 is identified in the multiple partitions, and then the storage space corresponding to the partition 311 is identified in the multiple storage spaces. Exemplarily, the storage space can be a physical storage space composed of continuous physical storage addresses. Exemplarily, the storage space can be a physical page, a disk partition, a volume, etc.

[0087] In one example of the embodiment, the correspondence between the partition and the storage space can be recorded in the list B3. In step 603, the storage space corresponding to the partition 311 can be identified based on the list B3. The correspondence between the partition and the storage space can be the correspondence between the address of the partition and the address of the storage space. Identifying the storage space can be identifying the address of the storage space.

[0088] In some embodiments, the data in different partitions in the same data table are stored in different files. In other words, the multiple partitions in the data table 310 and the multiple files are one-to-one corresponding, and the file corresponding to a partition is used to store or record the data with the home identity corresponding to the partition. That is, the data in a partition is actually stored in the form of a file, and the data in different partitions are stored as different files. In step 603, the partition 311 corresponding to the home identity A1 is identified in the multiple partitions, and then the file corresponding to the partition 311 is identified in the multiple files.

[0089] In one example of this embodiment, the correspondence between the partition and the file can be recorded in the list B4. In step 603, the file corresponding to the partition 311 can be identified based on the list B4. In one example, the correspondence between the partition and the file can be the correspondence between the path of the partition and the file. Identifying the file can be identifying the path of the file. In one example, the correspondence between the partition and the file can be the correspondence between the handle of the partition and the file. Identifying the file can be identifying the handle of the file.

[0090] After step 603, step 604 can be performed to access the target data in the partition 311 based on the access request.

[0091] The target data is stored in the partition 311, and in step 604, the target data can be accessed in the local range of the data table 310 in the partition 311, without the need to access the target data in the global range of the data table 310. Therefore, only the partition 311 needs to be locked, without the need to lock the entire data table 310, so that the access to the target data does not affect the data access of other partitions. And the access to the target data only needs to cache the data in the partition 311, without the need to cache the data in other partitions, thereby saving the cache resource overhead and avoiding the cache thrashing problem that may occur when the access is frequent. In addition, through the pruning routing of the partition 311, the location of the target data (i.e. the partition 311) can be quickly located, thereby improving the efficiency of data access.

[0092] In some embodiments, as described above, the data with the home identity A1 is stored in the storage space corresponding to the partition 311, that is, the target data is stored in the storage space corresponding to the partition 311. In step 604, the target data can be accessed in the storage space corresponding to the partition 311 based on the access request. For example, according to the access request, the target data is inserted, read, modified or deleted in the storage space corresponding to the partition 311 based on the address of the storage space.

[0093] In some embodiments, as mentioned above, the data with the home identity A1 is stored in the file corresponding to the partition 311, i.e. the target data is stored in the file corresponding to the partition 311. In step 604, the target data can be accessed in the file corresponding to the partition 311 based on the access request. For example, according to the access request, the file corresponding to the partition 311 is located based on the path or handle of the file, and the target data is inserted, read, modified or deleted in the file.

[0094] In some embodiments, as mentioned above, the index can be created for the data in the partition 311. In step 604, the target data can be queried in the partition 311 based on the index of the partition 311. In the querying, only the index scan plan of the partition 311 is generated, thereby ensuring the correctness and efficiency of the data access. In this way, the query efficiency of the target data can be improved. After the target data is queried, the target data can be accessed based on the access request, e.g. the target data is inserted, read, modified or deleted.

[0095] Continuing to refer to Figure 6 When the multiple home identities do not include the home identity A1, step 605 can be performed to create the partition 313 corresponding to the home identity A1 in the data table 310.

[0096] The multiple home identities do not include the home identity A1, which means that there is no partition corresponding to the home identity A1 in the data table 310. In this case, as shown in Figure 7 the partition 313 corresponding to the home identity A1 can be created in the data table 310 to store the data with the home identity A1 in the partition 313.

[0097] In which, the partition 313 can be created according to the data routing result. In this way, the creation operation of the partition 313 does not block the data access of other partitions.

[0098] In some embodiments, the partition 313 can be created based on range division. For example, 1000-1100 rows in the data table 310 are divided into the partition 313. In some embodiments, the data table 310 is a logical table, a new physical table can be created and mapped to the data table 310, thereby obtaining the partition 313. In some embodiments, the data table 310 is a collection of physical tables, a new physical table can be added to the collection, and the new physical table is the partition 313. In some embodiments, the partition corresponds to a storage space, a new storage space can be divided in the memory, and the storage space is mapped to the partition 313, and the storage space is used to store the data corresponding to the partition 313. In some embodiments, the partition corresponds to a file, a new file can be created, and the file is mapped to the partition 313, and the file is used to store the data corresponding to the partition 313.

[0099] In some embodiments, the home identity A1 is a tenant identity. In step 605, a partition corresponding to the home identity A1 is created to store data of a tenant corresponding to the tenant identity in the partition.

[0100] For example, the partition 313 can be inserted in the data table 310 by an insert statement without separately executing a data table creation statement, such as a data definition language (DDL) statement.

[0101] After the partition 313 corresponding to the home identity is created in the data table 310, in step 606, target data can be accessed in the partition 313 based on an access request. The access request can be a write request or an insert request. In step 606, the target data is written or inserted in the partition 313.

[0102] In some embodiments, steps 605 and 606 are performed only when the plurality of home identities does not include the home identity A1 and the access request is a write request or an insert request. If the access request is a request other than a write request or an insert request, such as a read request, a modification request, or a deletion request, an empty result can be directly returned when the plurality of home identities does not include the home identity A1.

[0103] In some embodiments, when data corresponding to a certain home identity needs to be migrated, only data in a partition corresponding to the home identity needs to be migrated. For example, as shown in FIG. 6, the partition 311 can be migrated from the data table 310 to the data table 320, that is, the partition 311 is changed from a partition of the data table 310 to a partition of the data table 320. Since the partition and other partitions are physically isolated, when data is migrated, a row-by-row filtering copy is not required, and only data in the partition needs to be directly copied, so that data migration can be efficiently performed. Figure 8

[0104] In summary, data of different home identities is stored in different partitions of a data table, when data with a certain home identity is accessed, the data is accessed in a range of a partition corresponding to the home identity, rather than in a whole range of the data table, so that access to the data does not affect data access of other partitions, the concurrency of data access of the data table is improved, and the access performance of the data table is improved.

[0105] Embodiments of the present application provide a database-based data processing apparatus. The database includes a data table, and the data table includes a plurality of partitions. The plurality of partitions and a plurality of home identities correspond to each other in a one-to-one manner, and the partitions are used to store data with home identities corresponding to the partitions. Figure 9 As shown in FIG. 7, the apparatus includes: ​

[0106] The receiving module 910 is configured to receive an access request, where the access request is used to request access to target data in the data table, and the target data has a target home identity;

[0107] The identifying module 920 is configured to identify, when the multiple home identities include the target home identity, a first partition corresponding to the target home identity in the multiple partitions.

[0108] The access module 930 is configured to access the target data in the first partition based on the access request.

[0109] In some embodiments, the first partition has an index, and the index is created based on data in the first partition.

[0110] The access module 930 is configured to:

[0111] query the target data in the first partition based on the index;

[0112] access the target data based on the access request.

[0113] In some embodiments, the multiple partitions and multiple storage spaces are in one-to-one correspondence, and a storage space corresponding to the partition is used to store data having a home identity corresponding to the partition.

[0114] The identifying module 920 is configured to identify, in the multiple storage spaces, a storage space corresponding to the first partition.

[0115] The access module 930 is configured to access the target data in the storage space corresponding to the first partition based on the access request.

[0116] In some embodiments, the multiple partitions and multiple files are in one-to-one correspondence, and a file corresponding to the partition is used to store data having a home identity corresponding to the partition.

[0117] The identifying module 920 is configured to identify, in the multiple files, a file corresponding to the first partition.

[0118] The access module 930 is configured to access the target data in the file corresponding to the first partition based on the access request.

[0119] In some embodiments, the apparatus further includes a creating module.

[0120] The creating module is configured to create, when the multiple home identities do not include the target home identity, a second partition corresponding to the target home identity in the data table.

[0121] The access module 930 is further configured to: access the target data in the second partition based on the access request.

[0122] In some embodiments, different attribution identifiers among the multiple attribution identifiers are different tenant identifiers, or different attribution identifiers among the multiple attribution identifiers correspond to different data types.

[0123] The receiving module 910, the identifying module 920, and the accessing module 930 can all be implemented in software or hardware. For example, the implementation of the receiving module 910 will be described below using the receiving module 910 as an example. Similarly, the implementation of the identifying module 920 and the accessing module 930 can refer to the implementation of the receiving module 910.

[0124] As an example of a software functional unit, the receiving module 910 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the receiving module 910 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0125] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0126] As an example of a hardware functional unit, the receiving module 910 can include at least one computing device, such as a server or the like. Alternatively, the receiving module 910 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the like. The PLD can be implemented by a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a gener ic array logic (GAL), or any combination thereof.

[0127] The multiple computing devices included in the receiving module 910 can be distributed in the same region or in different regions. The multiple computing devices included in the receiving module 910 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the receiving module 910 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0128] It should be noted that, in other embodiments, the receiving module 910 can be configured to perform any of the steps of the method shown in Figure 6 The identification module 920 can be configured to perform any of the steps of the method shown in Figure 6 The access module 930 can be configured to perform any of the steps of the method shown in Figure 6 The steps implemented by the receiving module 910, the identification module 920, and the access module 930 can be specified as needed, and the entire function of the data storage apparatus 900 can be implemented by the receiving module 910, the identification module 920, and the access module 930 respectively implementing Figure 6 different steps of the method shown in

[0129] The present application also provides a computing device 1000. As shown in Figure 10 The computing device 1000 includes a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008. The processor 1004, the memory 1006, and the communication interface 1008 communicate with each other through the bus 1002. The computing device 1000 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1000.

[0130] The bus 1002 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 only one bus or one type of bus. The bus 1002 can include a path for transmitting information between the various components of the computing device 1000 (e.g., the memory 1006, the processor 1004, the communication interface 1008).

[0131] The processor 1004 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0132] The processor 1004 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0133] The memory 1006 can include a volatile memory (e.g., a random access memory (RAM)), and can also include a non-volatile memory (e.g., a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD)).

[0134] The memory 1006 stores executable program code, and the processor 1004 executes the executable program code to respectively implement the functions of the foregoing receiving module 910, the identifying module 920, and the accessing module 930, thereby implementing the method shown in Figure 6 The memory 1006 stores executable program code, and the processor 1004 executes the executable program code to respectively implement the functions of the foregoing receiving module 910, the identifying module 920, and the accessing module 930, thereby implementing the method shown in Figure 6 The memory 1006 stores executable program code, and the processor 1004 executes the executable program code to respectively implement the functions of the foregoing receiving module 910, the identifying module 920, and the accessing module 930, thereby implementing the method shown in

[0135] The communication interface 1008 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, etc., to implement the communication between the computing device 1000 and other devices or communication networks.

[0136] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device, such as a desktop computer, a notebook computer, or a smart phone.

[0137] As shown in Figure 11 , the computing device cluster includes at least one computing device 1000. The memory 1006 in one or more computing devices 1000 in the computing device cluster can store the same instructions for performing the method shown in Figure 6 .

[0138] In some possible implementations, the memory 1006 in one or more computing devices 1000 in the computing device cluster can also respectively store partial instructions for performing the method shown in Figure 6 . In other words, the combination of one or more computing devices 1000 can collectively perform the instructions for performing the method shown in Figure 6 .

[0139] It should be noted that the memory 1006 in different computing devices 1000 in the computing device cluster can store different instructions, respectively for performing partial functions of the apparatus 900. That is, the instructions stored in the memory 1006 in different computing devices 1000 can implement the functions of one or more of the receiving module 910, the identifying module 920, and the accessing module 930.

[0140] In some possible implementations, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc. Figure 12 A possible implementation is shown. As shown in Figure 12 , two computing devices 1000A and 1000B are connected through a network. Specifically, the communication interface in each computing device is connected to the network. In this type of possible implementation, the memory 1006 in the computing device 1000A stores instructions for performing the functions of the receiving module 910. Meanwhile, the memory 1006 in the computing device 1000B stores instructions for performing the functions of the identifying module 920 and the accessing module 930.

[0141] It should be understood that Figure 12 the functions of the computing device 1000A shown in may also be accomplished by multiple computing devices 1000. Similarly, the functions of the computing device 1000B can also be accomplished by multiple computing devices 1000.

[0142] The embodiments of the present application also provide another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection relationship between the computing devices in the computing device cluster as described above. Figure 11 and Figure 12 The difference is that the memory 1006 in one or more computing devices 1000 in the computing device cluster can store the same instructions for performing the method as shown in Figure 6 .

[0143] In some possible implementation manners, the memory 1006 in one or more computing devices 1000 in the computing device cluster can also respectively store part of the instructions for performing the method as shown in Figure 6 . In other words, the combination of the one or more computing devices 1000 can collectively perform the instructions for performing the method as shown in Figure 6 .

[0144] The embodiments of the present application also provide a computer program product containing instructions. The computer program product can be a software or program product containing instructions, which can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, the at least one computing device is caused to perform the method as shown in Figure 6 .

[0145] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc. The computer readable storage medium includes instructions, which instruct the computing device to perform the method as shown in Figure 6 .

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A database-based data processing method, characterized by, The database includes a data table, the data table includes multiple partitions, the multiple partitions correspond to multiple attribution identifiers one by one, and the partitions are used to store data with the attribution identifiers corresponding to the partitions; the method includes: receiving an access request, the access request being used to request access to target data in the data table, the target data having a target ownership identifier; When the plurality of attribution identifiers include the target attribution identifier, identifying a first partition corresponding to the target attribution identifier among the plurality of partitions; Based on the access request, the target data is accessed in the first partition.

2. The method of claim 1, wherein, The first partition has an index, and the index is created based on the data in the first partition; Accessing the target data in the first partition based on the access request includes: Based on the index, searching the first partition for the target data; Based on the access request, the target data is accessed.

3. The method according to claim 1 or 2, characterized in that, The multiple partitions correspond to the multiple storage spaces in a one-to-one manner, and the storage spaces corresponding to the partitions are used to store data having the attribution identifiers corresponding to the partitions; The step of identifying, among the plurality of partitions, a first partition corresponding to the target attribution identifier, includes: identifying, among the plurality of storage spaces, a storage space corresponding to the first partition; Accessing the target data in the first partition based on the access request includes: accessing the target data in a storage space corresponding to the first partition based on the access request.

4. The method according to claim 1 or 2, characterized in that, The multiple partitions correspond to the multiple files in a one-to-one manner, and the files corresponding to the partitions are used to store data having the attribution identifiers corresponding to the partitions; The identifying the first partition corresponding to the target attribution identifier includes: identifying, among the multiple files, a file corresponding to the first partition; Accessing the target data in the first partition based on the access request includes: accessing the target data in a file corresponding to the first partition based on the access request.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: When the plurality of attribution identifiers do not include the target attribution identifier, creating a second partition corresponding to the target attribution identifier in the data table; Based on the access request, the target data is accessed in the second partition.

6. The method according to any one of claims 1-5, characterized in that, Different attribution identifiers among the multiple attribution identifiers are identifiers of different tenants, or different attribution identifiers among the multiple attribution identifiers correspond to different data types.

7. A database-based data processing apparatus, characterized by comprising: The database includes a data table, the data table includes multiple partitions, the multiple partitions correspond to multiple attribution identifiers one by one, and the partitions are used to store data with the attribution identifiers corresponding to the partitions; the device includes: A receiving module, configured to receive an access request, wherein the access request is used to request access to target data in the data table, and the target data has a target ownership identifier; an identification module, configured to, when the plurality of attribution identifiers include the target attribution identifier, identify a first partition corresponding to the target attribution identifier among the plurality of partitions; An access module is configured to access the target data in the first partition based on the access request.

8. The apparatus of claim 7, wherein, The first partition has an index, the index is created based on data in the first partition; The access module is configured to: query the target data in the first partition based on the index; access the target data based on the access request.

9. The apparatus of claim 7 or 8, wherein, The plurality of partitions and the plurality of storage spaces are in one-to-one correspondence, and the storage space corresponding to the partition is used to store data having a home identity corresponding to the partition; The identification module is configured to identify, in the plurality of storage spaces, a storage space corresponding to the first partition; The access module is configured to access the target data in the storage space corresponding to the first partition based on the access request.

10. The apparatus of claim 7 or 8, wherein, The plurality of partitions and the plurality of files are in one-to-one correspondence, and the file corresponding to the partition is used to store data having a home identity corresponding to the partition; The identification module is configured to identify, in the plurality of files, a file corresponding to the first partition; The access module is configured to access the target data in the file corresponding to the first partition based on the access request.

11. The apparatus of any one of claims 7-10, wherein, The apparatus further includes a creation module; The creation module is configured to create a second partition corresponding to the target home identity in the data table when the plurality of home identities does not include the target home identity; The access module is further configured to access the target data in the second partition based on the access request.

12. The apparatus of any one of claims 7-11, wherein, Different home identities in the plurality of home identities are different tenant identities, or different home identities in the plurality of home identities correspond to different data types.

13. A cluster of computing devices, characterized in that, The at least one computing device includes a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any one of claims 1-6.

14. A computer-readable storage medium, characterized in that, The computer program instructions, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any one of claims 1-6.

15. A computer program product comprising instructions, characterized in that, The instructions, when executed by a cluster of computer devices, cause the cluster of computer devices to perform the method of any one of claims 1-6.

Images