Data processing method, distributed database system, device and storage medium

Abstract

Embodiments of the present application provide a data processing method, a distributed database system, equipment and a storage medium. The method comprises the following steps: determining a target hotspot database partition in the distributed database system; obtaining a plurality of target access requests for the target hotspot database partition; identifying an access mode corresponding to the target hotspot database partition according to the plurality of target access requests to obtain an identification result; and processing the target hotspot database partition by using a corresponding hotspot processing strategy according to the identification result. The technical solution provided by the embodiments of the present application can realize targeted processing of hotspot problems in the distributed database system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and particularly relates to a data processing method, a distributed database system, equipment and a storage medium. BACKGROUND

[0002] In the distributed database, a database table is decomposed into multiple partitions according to certain rules, and then the multiple partitions are distributed to multiple working nodes.

[0003] At present, the partition mode can be divided into Hash partition and range partition. The Hash partition has the characteristics of avoiding hot spots to a certain extent, but is not conducive to sequential scanning of data. In contrast, the range partition is conducive to sequential scanning of data, but is prone to hot spots. SUMMARY

[0004] Embodiments of the present application provide a data processing method, a distributed database system, equipment and a storage medium, which are used for realizing targeted processing of hot spot problems in a distributed database system.

[0005] Therefore, in an embodiment of the present application, a data processing method is provided, which comprises the following steps:

[0006] determining a target hot spot database partition in the distributed database system;

[0007] obtaining multiple target access requests for the target hot spot database partition;

[0008] According to the multiple target access requests, an access mode corresponding to the target hot spot database partition is identified to obtain an identification result;

[0009] According to the identification result, a corresponding hot spot processing strategy is used to process the target hot spot database partition.

[0010] In another embodiment of the present application, a distributed database system is provided, which comprises a management node and multiple working nodes, and at least one database partition is arranged on each working node.

[0011] The management node is configured to:

[0012] determine a target hot spot database partition in the distributed database system;

[0013] obtain multiple target access requests for the target hot spot database partition;

[0014] According to the multiple target access requests, an access mode corresponding to the target hot spot database partition is identified to obtain an identification result;

[0015] According to the identification result, a corresponding hotspot processing strategy is adopted to process the target hotspot database partition.

[0016] In yet another embodiment of the present application, an electronic device is provided. The electronic device comprises a memory and a processor, wherein,

[0017] The memory is configured to store a program.

[0018] The processor is coupled to the memory and configured to execute the program stored in the memory to implement the data processing method described above.

[0019] In yet another embodiment of the present application, a computer readable storage medium storing a computer program is provided, and the computer program can implement the data processing method described above when executed by a computer.

[0020] In the technical solution provided by the embodiments of the present application, the access mode corresponding to the target hotspot database partition can be identified according to a plurality of target access requests for the target hotspot database partition, and a corresponding hotspot processing strategy is adopted for hotspot processing according to the identification result, so that the pertinence of hotspot problem processing can be improved and the negative influence caused by invalid processing can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1a The structural schematic diagram of the distributed database system provided by an embodiment of the present application;

[0023] Figure 1b The processing flow schematic diagram inside the distributed database system provided by an embodiment of the present application;

[0024] Figure 2 The flow schematic diagram of the data processing method provided by an embodiment of the present application;

[0025] Figure 3 The sampling flow schematic diagram provided by an embodiment of the present application;

[0026] Figure 4 The determination flow schematic diagram of the candidate set provided by an embodiment of the present application;

[0027] Figure 5 The bucket sorting result schematic diagram provided by an embodiment of the present application;

[0028] Figure 6 A structural block diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0029] In a distributed database, data needs to be distributed on multiple machines according to certain rules. At present, there are two methods of Hash sharding (or partitioning) and range sharding. Hash sharding naturally has the characteristic of avoiding hot spots but is not conducive to sequential scanning of data. In contrast, the design of range sharding is conducive to sequential scanning of data but is prone to hot spots. In a distributed database, each database region is only responsible for the storage service and read-write service of a part of data. When the access traffic of a small number of database regions is significantly higher than that of other database regions, it is called a hot region.

[0030] Taking range sharding as an example, in a distributed database system, the data of a table can be split into multiple database regions according to ranges, and each database region is responsible for the read-write service of the corresponding data range. At the same time, database regions can be migrated between machines at low cost to achieve load balancing between machines. When the system has hot spots, it often manifests as one or more hot database regions.

[0031] At present, for a distributed database system, there is a scheme of partition splitting to solve the problem of hot regions. In the scheme, the service process corresponding to each database region determines whether the read request number of the database region exceeds a preset traffic threshold. If the preset traffic threshold is exceeded, the database region is determined to be a hot region. When a certain database region is determined to be a hot region, the scheme of partition splitting is used to perform hot processing on the hot region.

[0032] The applicant has found through research that some database regions cannot solve the hot spot problem by the scheme of partition splitting, because no matter how the database region is split, the access traffic is still concentrated on a certain sub-database region. That is, for such a database region, using the scheme of partition splitting for processing is ineffective processing, and constantly splitting not only increases the consumption of computing resources of the entire distributed database, but also has a negative impact on the access of terminal users and increases the maintenance difficulty of the database regions in the entire distributed database.

[0033] In order to solve or partially solve the above problems, an embodiment of the present application proposes a data processing method, which can identify the access mode corresponding to the target hot database region according to a plurality of target access requests for the target hot database region, and perform hot processing by using a corresponding hot processing strategy according to the identification result, thereby improving the pertinence of hot problem processing and reducing the negative impact of ineffective processing.

[0034] For the convenience of understanding, the terms related to the embodiments of the present application are explained as follows:

[0035] Hotspot: Specifically refers to a scenario in a distributed system where access requests or pressure are concentrated on one or several machines in the cluster

[0036] Database partition region: A concept in HBase database, responsible for reading and writing services for a part of data, which can be migrated between machines, and at the same time, can only provide services on one machine

[0037] Range partition: A common method of splitting data in a distributed system, which splits data after sorting to make each shard contain a continuous piece of data

[0038] Throttling: Limiting specific request access through software and hardware means to protect the system itself

[0039] Isolation: Providing services for a specific region using separate hardware resources, which is often a hotspot region or has an exception, and the impact is reduced through hardware isolation to ensure that the remaining regions can provide normal services

[0040] Request sampling: A statistical probability-based method for sampling all requests for services

[0041] Automated governance: Without human intervention, automatically trigger execution through program logic

[0042] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described in detail below based on the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0043] In addition, in some of the processes described in the specification, claims, and above-mentioned drawings, a plurality of operations appearing in a specific order include operations that can not be executed in the order they appear in this document or in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. described herein are used to distinguish different messages, devices, modules, etc., and do not represent the order of sequence, nor do "first" and "second" represent different types.

[0044] Before introducing the data processing method provided in the present application, the system architecture related to the data processing method proposed in the present application is introduced. Figure 1a A schematic diagram of a distributed database system provided by an embodiment of the present application is shown. As shown in the figure, the system comprises a management node 10 and a plurality of worker nodes 11; the worker nodes 11 are configured to provide related services of at least one database partition; the management node 10 is configured to: Figure 1a

[0045] determine a target hotspot database partition in the distributed database system;

[0046] obtain a plurality of target access requests for the target hotspot database partition;

[0047] identify an access mode corresponding to the target hotspot database partition according to the plurality of target access requests, to obtain an identification result;

[0048] adopt a corresponding hotspot processing strategy to process the target hotspot database partition according to the identification result.

[0049] In the distributed database system, at least one database table can be included; the range partitioning manner can be adopted to obtain at least one database partition corresponding to the database table, and the at least one database partition corresponding to the database table can be allocated to different worker nodes 11 in the system, and each worker node 11 provides storage, read / write and other services for the at least one database partition allocated thereto. In addition, the database partition can be migrated between different worker nodes 11, so as to quickly recover the related services when a hardware fault occurs in a certain worker node 11.

[0050] Each worker node 11 can periodically send running information of the at least one database partition allocated thereto to the management node 10 through the network, so as to be counted by the management node 10. The running information includes but is not limited to: read request frequency, write request frequency, total network throughput generated by read requests, total network throughput generated by write requests, average network delay of read requests, and average network delay of write requests.

[0051] In the technical scheme provided by the embodiment of the present application, the access mode corresponding to the target hotspot database partition can be identified according to the plurality of target access requests for the target hotspot database partition, and the hotspot processing strategy is adopted for hotspot processing according to the identification result, so that the pertinence of hotspot problem processing can be improved, and the negative influence caused by invalid processing can be reduced.

[0052] Specifically, a plurality of functional modules can be run on the management node 10, such as Figure 1b ​As shown, the management node 10 can run: a hotspot detection module 101, an access pattern analysis module 102, and a processing module 103. The working node 11 can run a sample storage module 111.

[0053] The processing procedures of the units in the system and the interaction procedures between the units will be described in detail in the following embodiments, and thus will not be repeated here.

[0054] Figure 2 A flowchart of a data processing method provided by an embodiment of the present application is shown. The execution subject of the method can be a management node in the distributed database system. As shown, the method includes: Figure 2

[0055] 201. Determine a target hotspot database partition in the distributed database system.

[0056] 202. Obtain a plurality of target access requests for the target hotspot database partition.

[0057] 203. According to the plurality of target access requests, identify an access pattern corresponding to the target hotspot database partition to obtain an identification result.

[0058] 204. According to the identification result, adopt a corresponding hotspot processing strategy to process the target hotspot database partition.

[0059] In 201, the distributed database system includes a plurality of database partitions. According to the running information corresponding to each of the plurality of database partitions, the target hotspot database partition can be determined from the plurality of database partitions.

[0060] In 202, the target access request can include a write request and / or a read request. In an example, in order to simultaneously consider the hotspot detection of write traffic and read traffic, the target access request can include a write request and a read request.

[0061] The plurality of target access requests can be implemented in one or more of the following ways:

[0062] Way one, obtain all access requests for the target hotspot database partition in a first preset time period as the plurality of target access requests.

[0063] Way two, obtain a plurality of sample access requests obtained by sampling the access requests for the target hotspot database partition in a first preset time period as the plurality of target access requests.

[0064] ​In actual application, if full amount storage is performed on each database partition access request, a large amount of storage resources needs to be occupied. Therefore, in actual application, a preset sampling ratio can be used to sample each database partition access request, and the sampled sample access requests are stored, so that the consumption of storage resources can be effectively reduced; and by controlling the sampling ratio, it can be ensured that the sample access requests have a certain representativeness, so that the recognition accuracy of the database partition corresponding access mode according to the sample access requests is also guaranteed. Therefore, in actual application, mode two can be used to implement the above step 102. By controlling the sampling ratio, a good balance between storage overhead cost and recognition accuracy can be achieved. The above plurality of target access requests can be stored in memory or persistent storage.

[0065] In an implementable scheme, the "obtaining a plurality of target access requests for the target hotspot database partition" in 102 can be implemented by the following steps:

[0066] 1021. A data acquisition request is sent to the working node where the target hotspot database partition is located, so that the working node obtains the plurality of target access requests within a first preset time period according to the data acquisition request.

[0067] 1022. The working node sends the plurality of target access requests.

[0068] The working node samples the access request data for the target hotspot database partition within the first preset time period to obtain the plurality of target access requests, and performs persistent storage processing on the plurality of target access requests. The working node samples the plurality of sample access requests for the target hotspot database partition according to a preset sampling ratio within the first preset time period. As shown in the figure, the access requests (including write requests and read requests) for the database partition are sampled and stored. Figure 3

[0069] Using persistent storage, the storage duration and storage capacity can be effectively extended, which helps to provide more reference data for subsequent access mode recognition and improve the recognition accuracy of the access mode. In addition, the persistent storage access request can also provide necessary materials for subsequent product continuous iteration and evolution.

[0070] The first preset time period can be a recent first preset time period, for example, the last 7 minutes. The first preset time period can be set according to actual needs, and the embodiments of the present application do not make specific limitations thereto.

[0071] ​In 203, the access mode is related to whether the target hotspot database partition can disperse access traffic through partition splitting. The access rule corresponding to the plurality of target access requests can be determined according to the primary keys accessed by each target access request in the plurality of target access requests and the access time. The access mode corresponding to the target hotspot database partition can be determined according to the access rule.

[0072] In actual application, at least one candidate access mode can be set in advance, and the candidate access mode refers to a discovered access mode. The at least one candidate access mode can include a discovered access mode that can disperse access traffic through partition splitting and / or a discovered access mode that cannot disperse access traffic through partition splitting. Whether the target hotspot database partition belongs to one of the at least one candidate access mode can be determined according to the plurality of target access requests. Specifically, the access rule corresponding to the plurality of target access requests can be determined according to the primary keys accessed by each target access request in the plurality of target access requests and the access time. Whether the target hotspot database partition belongs to one of the at least one candidate access mode can be determined according to the access rule.

[0073] As time goes on, more and more new access modes can be discovered. Therefore, in the embodiments of the present application, the number of candidate access modes can be increased continuously. That is, the number of at least one candidate access mode can gradually increase over time. In this way, the recognition coverage and recognition accuracy of the access mode can be improved, and thus the effectiveness of hotspot processing can be improved.

[0074] In 204, the corresponding hotspot processing strategy is determined according to the recognition result, and the target hotspot database partition is processed by using the corresponding hotspot processing strategy. Different recognition results can be processed by using different hotspot processing strategies.

[0075] In an implementable scheme, for each access mode, a hotspot processing strategy that is adapted to the access mode can be configured, and an association between the access mode and the hotspot processing strategy that is adapted to the access mode can be established. In this way, the corresponding hotspot processing strategy can be obtained according to the access mode corresponding to the target hotspot database partition in the recognition result.

[0076] In the technical scheme provided by the embodiments of the present application, the access mode corresponding to the target hotspot database partition can be recognized according to the plurality of target access requests for the target hotspot database partition, and the corresponding hotspot processing strategy is used for hotspot processing according to the recognition result. The pertinence of hotspot problem processing can be improved, and the negative influence caused by invalid processing can be reduced.

[0077] In an example, 201 can be implemented by the hotspot detection module 101 in the hotspot detection device 100, and 202 and 203 can be implemented by the hotspot processing module 102 in the hotspot detection device 100. Figure 1b Figure 1b ​the access pattern analysis module 102 in the database cluster management system 100, wherein the storage process of the plurality of target access requests can be performed by Figure 1b the sample storage module 111 in the database cluster management system 100; and the step 204 can be performed by Figure 1b the access pattern analysis module 102 and the processing module 103 in the database cluster management system 100.

[0078] In an implementable solution, the step 203 of "identifying the access pattern corresponding to the target hot database partition according to the plurality of target access requests to obtain an identification result" can be implemented by the following steps:

[0079] 2031. determining whether the access pattern corresponding to the target hot database partition belongs to one of at least one candidate access pattern according to the plurality of target access requests.

[0080] The candidate access pattern refers to a discovered access pattern that cannot disperse access traffic by partition splitting.

[0081] In this embodiment, each of the at least one candidate access pattern is a discovered access pattern that cannot disperse access traffic by partition splitting.

[0082] To improve the identification accuracy, a corresponding pattern recognition algorithm can be configured for each candidate access pattern. Subsequently, the pattern recognition algorithm corresponding to each candidate access pattern can be used to determine whether the access pattern corresponding to the target hot database partition belongs to each candidate access pattern according to the plurality of target access requests. Specifically, the at least one candidate access pattern includes a first candidate access pattern; and the pattern recognition algorithm corresponding to the first candidate access pattern is used to determine whether the access pattern corresponding to the target hot database partition belongs to the first candidate access pattern according to the plurality of target access request data.

[0083] If the identification result indicates that the access pattern corresponding to the target hot database partition belongs to one of the at least one candidate access pattern, it means that the target hot database partition cannot disperse access traffic by partition splitting, and the partition isolation strategy or the partition flow limiting strategy can be used to process the target hot database partition. If the identification result indicates that the access pattern corresponding to the target hot database partition does not belong to any of the at least one candidate access pattern, the partition splitting strategy can be used to process the target hot database partition. The partition isolation strategy refers to a strategy of migrating the partition to an isolated node; the partition flow limiting strategy refers to a strategy of limiting the access traffic of the partition; and the partition splitting strategy refers to a strategy of splitting the partition.

[0084] The applicant finds two access modes that cannot disperse traffic by partition splitting, i.e., a centralized access mode and a sequential access mode. The centralized access mode refers to a mode in which a point access request accesses a same record in a database partition or a range access request accesses a same record range in a database partition. The sequential access mode refers to a mode in which records in a database partition are sequentially accessed.

[0085] Optionally, the centralized access mode is included in the at least one alternative access mode. The step of determining, according to the multiple target access requests, whether the access mode corresponding to the target hotspot database partition belongs to one of the at least one alternative access mode in 2031 can be implemented as follows:

[0086] S11, determining at least one point access request from the multiple target access requests.

[0087] S12, determining a first primary key from at least one primary key accessed by the at least one point access request.

[0088] S13, determining a first proportion of point access requests that access the first primary key in the multiple target access requests.

[0089] S14, if the first proportion is greater than or equal to a first preset proportion, determining that the access mode corresponding to the target hotspot database partition belongs to the centralized access mode.

[0090] In S11, generally, access requests can be divided into two types, i.e., a point access request and a range access request. The point access request carries a single primary key, and the range access request carries a start primary key and an end primary key used to represent a primary key range. Therefore, the type of a target access request can be determined according to the form of a primary key carried in the target access request.

[0091] In S12, the at least one primary key can be all primary keys accessed by the at least one point access request.

[0092] In S13, the first proportion can be a ratio between a number of point access requests that access the first primary key in the multiple target access requests and a total number of the multiple target access requests.

[0093] In S14, the first preset proportion can be set according to actual needs, and embodiments of the present application do not make specific limitations thereon. In an example, the first preset proportion can be greater than 50%, for example, 80%.

[0094] In an example, the first primary key can be determined from the at least one primary key in a traversal manner, and then the step S13 is performed; if the first proportion is greater than or equal to the first preset proportion, it is determined that the access mode corresponding to the target hotspot database partition belongs to the concentrated access mode; if the first proportion is less than the first preset proportion, the traversal step is returned to continue, until the first proportion greater than or equal to the first preset proportion is determined or until all the primary keys in the at least one primary key are traversed.

[0095] At the end of the traversal, if the first proportion determined last time is still less than the first preset proportion, it is determined that the access mode corresponding to the target hotspot database partition does not belong to the concentrated access mode.

[0096] In another example, according to at least one point access request, the access times of each primary key in the at least one primary key are counted; the primary key with the maximum access times is taken as the first primary key, and then the step S13 is performed to obtain the first proportion; if the first proportion is greater than or equal to the first preset proportion, it is determined that the access mode corresponding to the target hotspot database partition belongs to the concentrated access mode; if the first proportion is less than the first preset proportion, it is determined that the access mode corresponding to the target hotspot database partition does not belong to the concentrated access mode.

[0097] Optionally, the step of "determining whether the access mode corresponding to the target hotspot database partition belongs to one of the at least one alternative access mode according to the plurality of target access requests" in 2031 can further include the following steps:

[0098] S15, determining at least one range access request from the plurality of target access requests.

[0099] S16, determining a first primary key range from at least one primary key range accessed by the at least one range access request.

[0100] S17, determining a second proportion of range access requests in the plurality of target access requests that access the first primary key range.

[0101] S18, if the second proportion is greater than or equal to a second preset proportion, it is determined that the access mode corresponding to the target hotspot database partition belongs to the concentrated access mode.

[0102] In S16, the at least one primary key range can be all the primary key ranges accessed by the at least one range access request.

[0103] In S17, the ratio between the number of range access requests in the plurality of target access requests that access the first primary key range and the total number of the plurality of target access requests can be taken as the second proportion.

[0104] The second preset proportion can be set according to actual needs, and embodiments of the present application do not make specific limitations thereon. In an example, the second preset proportion can be greater than 50%, for example, 80%. The first preset proportion and the second preset proportion can be the same or different.

[0105] In an example, the first primary key range can be determined from the at least one primary key range in a traversal manner, and then the step S17 is performed; if the second proportion is greater than or equal to the second preset proportion, it is determined that the access mode corresponding to the target hot database partition belongs to the concentrated access mode; if the second proportion is less than the second preset proportion, the step of the above traversal is returned to continue, until it is determined that the second proportion is greater than or equal to the second preset proportion or until all the primary key ranges in the at least one primary key range are traversed.

[0106] At the end of the traversal, if the second proportion determined last time is still less than the second preset proportion, it is determined that the access mode corresponding to the target hot database partition does not belong to the concentrated access mode.

[0107] In another example, according to the at least one range access request, the number of accesses of each primary key range in the at least one primary key range is counted; the primary key range with the maximum number of accesses is taken as the first primary key range, and then the step S13 is performed to obtain the second proportion; if the second proportion is greater than or equal to the second preset proportion, it is determined that the access mode corresponding to the target hot database partition belongs to the concentrated access mode; if the second proportion is less than the second preset proportion, it is determined that the access mode corresponding to the target hot database partition does not belong to the concentrated access mode.

[0108] In actual application, the steps S11 to S13 can be performed first; if the first proportion is less than the first preset proportion, the steps S15 to S18 are performed; if the first proportion is greater than or equal to the first preset proportion, it is determined that the access mode corresponding to the target hot database partition belongs to the concentrated access mode, and the steps S15 to S18 can be omitted.

[0109] Of course, the steps S15 to S17 can be performed first; if the second proportion is less than the second preset proportion, the steps S11 to S14 are performed; if the second proportion is greater than or equal to the second preset proportion, it is determined that the access mode corresponding to the target hot database partition belongs to the concentrated access mode, and the steps S11 to S14 can be omitted.

[0110] Optionally, the at least one alternative access mode includes a sequential access mode. The step of determining whether the access mode corresponding to the target hot database partition belongs to one of the at least one alternative access mode according to the plurality of target access requests in the above 2031 can be implemented by the following steps:

[0111] S21, sort the primary keys accessed by the plurality of target access requests according to access time corresponding to the plurality of target access requests to obtain a primary key sequence.

[0112] S22, determine a longest subsequence of the primary key sequence.

[0113] The longest subsequence is one of a longest increasing subsequence of the primary key sequence and a longest decreasing subsequence of the primary key sequence.

[0114] S23, if a ratio of a number of primary keys of the longest subsequence to a number of primary keys of the primary key sequence is greater than or equal to a preset ratio threshold, determine that the access mode corresponding to the target hotspot database partition belongs to the sequential access mode.

[0115] In the above S21, the primary key accessed by the target access request with earlier access time is sorted earlier in the primary key sequence; or the primary key accessed by the target access request with earlier access time is sorted later in the primary key sequence.

[0116] In the above S22, in an example, the primary key sequence can be input into a longest increasing subsequence algorithm to obtain a longest increasing subsequence of the primary key sequence.

[0117] In another example, the primary key sequence can also be reversed to obtain a reversed primary key sequence; the reversed primary key sequence is input into the longest increasing subsequence algorithm to obtain a longest increasing subsequence of the reversed primary key sequence; and the longest increasing subsequence of the reversed primary key sequence is taken as a longest decreasing subsequence of the primary key sequence.

[0118] In actual application, whether the access mode corresponding to the target hotspot database partition belongs to the sequential access mode can be determined based on the longest increasing subsequence of the primary key sequence first; if the access mode corresponding to the target hotspot database partition is not determined to belong to the sequential access mode based on the longest increasing subsequence of the primary key sequence, whether the access mode corresponding to the target hotspot database partition belongs to the sequential access mode is determined according to the longest decreasing subsequence of the primary key sequence; and if the access mode corresponding to the target hotspot database partition is determined to belong to the sequential access mode based on the longest increasing subsequence of the primary key sequence, the longest decreasing subsequence of the primary key sequence does not need to be determined.

[0119] Of course, whether the access mode corresponding to the target hotspot database partition belongs to the sequential access mode can also be determined based on the longest decreasing subsequence of the primary key sequence first. Then it is decided whether the longest increasing subsequence of the primary key sequence needs to be determined.

[0120] In actual application, the longest increasing subsequence and the longest decreasing subsequence of the primary key sequence can be determined first; then the longest sequence of the longest increasing subsequence and the longest decreasing subsequence is taken as the longest subsequence. In this way, if the ratio of the number of primary keys in the longest subsequence to the number of primary keys in the primary key sequence is greater than or equal to a preset ratio threshold, it is determined that the access mode corresponding to the target hotspot database partition belongs to the sequential access mode; otherwise, it is determined that the access mode corresponding to the target hotspot database partition does not belong to the sequential access mode.

[0121] Optionally, the step of "adopting a corresponding hotspot processing strategy to process the target hotspot database partition according to the identification result" in 204 can include one or more of the following steps:

[0122] 2041. When the identification result indicates that the access mode of the target hotspot database partition belongs to the identified mode that cannot disperse access traffic by partition splitting, a partition isolation strategy or a partition flow limiting strategy is adopted to process the target hotspot database partition.

[0123] 2042. When the identification result indicates that the access mode of the target hotspot database partition does not belong to the identified mode that cannot disperse access traffic by partition splitting, a partition splitting strategy is adopted to split the target hotspot database partition.

[0124] In 2041, the partition isolation strategy is adopted to process the target hotspot database partition. Specifically, a migration instruction is sent to the working node where the target hotspot database partition is located, so that the target hotspot database partition is migrated to an isolated working node in the distributed database system by the working node where the target hotspot database partition is located. The isolated working node can be set in advance, and the isolated working node is specially used to receive the hotspot database partition that cannot disperse access traffic by partition splitting.

[0125] The partition flow limiting strategy is adopted to process the target hotspot database partition. Specifically, a flow limiting instruction is sent to the working node where the target hotspot database partition is located, so that the working node where the target hotspot database partition is located performs flow limiting operation on the target hotspot database partition. The flow limiting instruction can carry a threshold of the number of access requests per unit time; and the working node is configured to perform packet loss processing when the number of access requests per unit time for the target hotspot database partition is greater than or equal to the threshold of the number of access requests.

[0126] In order to reduce the cost of operation and maintenance personnel intervention and troubleshooting, the method can further include:

[0127] 205、when the identification result indicates that the access mode of the target hotspot database partition belongs to an identified mode that cannot disperse access traffic through partition splitting, sending, to a user, a reason why the target hotspot database partition cannot disperse access traffic through partition splitting according to the access mode corresponding to the target hotspot database partition.

[0128] Specifically, the access mode corresponding to the target hotspot database partition can be directly sent to the user as the reason. It can be seen that the technical solution provided by the embodiments of the present application has certain automatic attribution capability.

[0129] In the above 2042, considering that access traffic on a database partition is not uniformly distributed, appropriate splitting positions must be used for splitting, so as to achieve traffic dispersion, otherwise the sub-database partitions after splitting will become hotspot database partitions.

[0130] The target hotspot database partition is split by using a partition splitting strategy. Specifically, the following steps can be used to achieve this:

[0131] S31, finding a target splitting position for the target hotspot database partition according to the plurality of target access requests.

[0132] S32, after finding the target splitting position, splitting the target hotspot database partition at the target splitting position to obtain two sub-database partitions.

[0133] S33, assigning the two sub-database partitions to different target worker nodes in the distributed database system.

[0134] In an implementable scheme, in the above S31, a plurality of candidate primary keys as splitting positions are determined according to the plurality of target access requests; a splitting index of each candidate primary key as a splitting position is determined among the plurality of candidate primary keys as splitting positions according to the plurality of target access requests; a target candidate primary key is determined from the plurality of candidate primary keys as splitting positions according to the splitting index of each candidate primary key as a splitting position among the plurality of candidate primary keys as splitting positions; and the target candidate primary key is taken as the target splitting position. The splitting index is related to the degree of appropriateness of the corresponding candidate primary key as a splitting position.

[0135] The plurality of candidate primary keys as splitting positions are determined according to the plurality of target access requests. Specifically, as Figure 4According to the plurality of target access requests, a first candidate primary key is determined as a split indicator of the split position, and specifically, the first candidate primary key is determined as a first split indicator of the split position according to the plurality of target access requests. The first split indicator is used to indicate an unbalanced degree of access traffic after splitting. In an example, a first number of requests accessing a first candidate sub-database partition and a second number of requests accessing a second candidate sub-database partition in the plurality of target access requests can be determined. The first candidate sub-database partition and the second candidate sub-database partition are obtained by splitting the target hotspot database partition at the first candidate primary key as the split position. An absolute value of a difference between the first number of requests and the second number of requests and a ratio of the absolute value to a sum of the first number of requests and the second number of requests are taken as the first split indicator.

[0136] According to the plurality of target access requests, a first candidate primary key is determined as a split indicator of the split position, and specifically, the first candidate primary key is determined as a first split indicator of the split position according to the plurality of target access requests. The first split indicator is used to indicate an unbalanced degree of access traffic after splitting. In an example, a first number of requests accessing a first candidate sub-database partition and a second number of requests accessing a second candidate sub-database partition in the plurality of target access requests can be determined. The first candidate sub-database partition and the second candidate sub-database partition are obtained by splitting the target hotspot database partition at the first candidate primary key as the split position. An absolute value of a difference between the first number of requests and the second number of requests and a ratio of the absolute value to a sum of the first number of requests and the second number of requests are taken as the first split indicator.

[0137] According to the plurality of target access requests, a first candidate primary key is determined as a split indicator of the split position, and specifically, the first candidate primary key is determined as a first split indicator of the split position according to the plurality of target access requests. The first split indicator is used to indicate an unbalanced degree of access traffic after splitting. In an example, a first number of requests accessing a first candidate sub-database partition and a second number of requests accessing a second candidate sub-database partition in the plurality of target access requests can be determined. The first candidate sub-database partition and the second candidate sub-database partition are obtained by splitting the target hotspot database partition at the first candidate primary key as the split position. An absolute value of a difference between the first number of requests and the second number of requests and a ratio of the absolute value to a sum of the first number of requests and the second number of requests are taken as the first split indicator.

[0138] In actual application, according to the plurality of target access requests, a first candidate primary key is determined as a split indicator of the split position, and specifically, the first candidate primary key is determined as a first split indicator of the split position according to the plurality of target access requests. The first split indicator is used to indicate an unbalanced degree of access traffic after splitting. In an example, a first number of requests accessing a first candidate sub-database partition and a second number of requests accessing a second candidate sub-database partition in the plurality of target access requests can be determined. The first candidate sub-database partition and the second candidate sub-database partition are obtained by splitting the target hotspot database partition at the first candidate primary key as the split position. An absolute value of a difference between the first number of requests and the second number of requests and a ratio of the absolute value to a sum of the first number of requests and the second number of requests are taken as the first split indicator.

[0139] The first split indicator is determined as the first candidate primary key as the split indicator of the split position.

[0140] The second split indicator is determined as the first candidate primary key as the split indicator of the split position.

[0141] The sum of the first split indicator and the second split indicator is determined as the first candidate primary key as the split indicator of the split position.

[0142] The target candidate primary key is determined from the plurality of candidate primary keys as the split position according to the split indicators of the candidate primary keys as the split position. In an example, the candidate primary key with the minimum split indicator is taken as the target candidate primary key. In another example, in order to ensure the effectiveness of the split, the first split indicator and the second split indicator corresponding to the candidate primary key with the minimum split indicator are obtained according to the third manner above; if the first split indicator corresponding to the candidate primary key with the minimum split indicator is less than or equal to the first preset threshold and the second split indicator corresponding to the candidate primary key with the minimum split indicator is less than or equal to the second preset threshold, the candidate primary key with the minimum split indicator is taken as the target candidate primary key; if the first split indicator corresponding to the candidate primary key with the minimum split indicator is greater than the first preset threshold or the second split indicator corresponding to the candidate primary key with the minimum split indicator is greater than the second preset threshold, it is determined that no suitable target split position is found.

[0143] If no suitable target split position is found, the user is sent prompt information about the target hotspot database partition, so that the user performs access pattern analysis according to the access requests (for example, sample access requests obtained by sampling) stored in advance for the target hotspot database partition, to find a new access pattern that cannot disperse traffic by partition splitting. In an example, an access pattern input interface is provided; the new alternative access pattern input by the user in the input interface is received and saved, for use in subsequent access pattern identification. In order to facilitate the user to reproduce the scene, the sample access requests obtained by sampling for the target hotspot database partition are persistently stored, so as to provide necessary materials for continuous iteration and evolution.

[0144] In the actual application, the target split position is one, so that the target hotspot database partition is split at the target split position to obtain two sub-database partitions. The target candidate primary key at the target split position is located in one of the two sub-database partitions.

[0145] In the above S33, the two sub-database partitions are allocated to different target worker nodes in the distributed database system. The selection process of the target worker node can refer to the prior art, which will not be described here.

[0146] It needs to be explained that for the database partition of the centralized access mode, the target split position finding method provided by the embodiment of the present application cannot find a suitable split position; for the database partition of the sequential access mode, the target split position finding method provided by the embodiment of the present application can find a suitable split position, but through further observation, it is found that in the sampling time window (i.e. the first preset time period described above), the access requests in the first half are concentrated above the split point, and the access requests in the second half are concentrated below the split point, so it can be seen that if the split point determined is used for splitting, the future access traffic will be concentrated on the sub-database partition responsible for the first half of the database partition and the sub-database partition responsible for the second half of the database partition in turn. That is, it does not play a role in dispersing traffic.

[0147] In the present example, one database partition is split into two sub-database partitions, and the two sub-database partitions are load balanced to different worker nodes, which can effectively disperse access traffic and fully utilize the hardware resources of the cluster.

[0148] In order to reduce the negative impact of always executing incorrect governance on scenarios that cannot be correctly handled, the above method can further include:

[0149] 206, determining the actual distribution balance degree of the access traffic between the two sub-database partitions.

[0150] Among them, the two sub-database partitions are obtained after the target hotspot database partition is subjected to the above splitting processing.

[0151] 207, determining whether the splitting processing is effective according to the actual distribution balance degree.

[0152] 208, when it is determined that the splitting processing is not effective, obtaining the invalid split number of the database table to which the target hotspot database partition belongs.

[0153] 209, when the invalid split number is greater than or equal to a preset number, suspending the processing based on the partition splitting strategy for the database table.

[0154] In the above 206, the first actual request number of the first sub-database partition and the second actual request number of the second sub-database partition in a unit of time are obtained; the actual distribution balance degree can be determined according to the ratio of the absolute value of the difference between the first actual request number and the second actual request number to the sum of the first actual request number and the second actual request number. The larger the ratio is, the smaller the actual distribution balance degree is.

[0155] In 207, if the actual distribution balance degree is greater than or equal to the preset balance degree value, it is determined that the splitting processing is valid; otherwise, it is determined that the splitting processing is invalid; and the invalid splitting times of the database table to which the target hotspot database partition belongs are updated, for example, increased by 1.

[0156] In 209, the preset number of times can be set according to actual needs, which is not limited in the present application, for example, 3 times. When the invalid splitting times are greater than or equal to the preset number of times, it indicates that the access rule or traffic characteristics of the database table cannot be accurately captured at present, and therefore, the processing based on the partition splitting strategy for the database table needs to be suspended. In addition, a splitting invalidation notification can be sent to a user (for example, an operation and maintenance personnel) to inform the user to intervene and manually analyze the traffic characteristics to find a new access mode, so as to improve the identification ability of the access mode.

[0157] Optionally, the distributed database system includes a database table, and the database table corresponds to a plurality of database partitions. In 201, the target hotspot database partition in the distributed database system can be determined by the following steps:

[0158] 2011. At every preset time interval, the plurality of database partitions are sorted according to running information of the plurality of database partitions in a second preset time period.

[0159] 2012. According to the sorting result, the hotspot database partition corresponding to the database table is determined.

[0160] 2013. According to the hotspot database partition corresponding to the database table, the target hotspot database partition in the distributed database system is determined.

[0161] In 2011, the running information can include an access request frequency, a total network throughput generated by an access request, and an average response delay of an access request. The access request can include a read request and a write request. The second preset time period can be the current preset time interval. The length of the second preset time period can be less than the length of the first preset time period. In an example, the second preset time period can be 7 minutes, and the first preset time period can be 1 minute.

[0162] In 2011, the plurality of database partitions can be sorted according to the running information of the plurality of database partitions in the second preset time period.

[0163] In an implementable solution, a bucket sorting algorithm can be used to sort multiple database partitions. Specifically, the multiple database partitions can be sorted by bucket according to their target access request frequencies in a first preset time period. The target access request frequency can be an access request frequency or an adjusted access request frequency, which is obtained by adjusting the access request frequency according to the total network throughput generated by the access request and / or the average response delay of the access request. Specifically, an adjustment coefficient can be determined according to the average network throughput generated by the access request and the average response delay of the access request. The larger the average network throughput, the larger the adjustment coefficient, and the larger the average response delay of the access request, the larger the adjustment coefficient.

[0164] As shown in Figure 5 , database partitions with similar target access request frequencies are concentrated in a bucket, and multiple database partitions of a database table are finally divided into N buckets. When N is greater than or equal to 2, the average access request frequency (operation per second, referred to as ops) corresponding to each bucket is determined according to the target access request frequency of at least one database partition included in each bucket; and the N buckets are sorted according to the average access request frequency of each bucket. At least one database partition included in the bucket with the highest average access request frequency is determined as the hotspot database partition corresponding to the database table.

[0165] In the above 2013, considering that the access traffic of each database partition is constantly changing, some database partitions can only exhibit short hot spot effects, and once identified as a hotspot database partition, over-treatment is often caused by performing governance, which is undesirable. Therefore, in actual application, only stable hotspot database partitions can be governed.

[0166] It is determined whether there is a stable hotspot database partition in at least one hotspot database partition corresponding to a database table. The stable hotspot database partition is continuously determined as a hotspot database partition corresponding to the database table for n times. The size of n can be set according to actual needs, and the present application embodiment does not make specific limitation thereto.

[0167] In an example, the stable hotspot database partition corresponding to the database table can be used as a target hotspot database partition in the distributed database system.

[0168] In another example, considering that there are a large number of database tables in the distributed database system, in order to control the frequency of automatic governance in the cluster range and avoid instability caused by excessive governance, a plurality of hotspot database partitions can be selected as target hotspot database partitions from the respective stable hotspot database partitions of a plurality of database tables according to a preset governance proportion. For example, the preset governance proportion is 10%, and assuming that the total number of stable hotspot database partitions corresponding to a plurality of database tables is 100, 10 of the 100 are randomly selected as target hotspot database partitions.

[0169] In summary, in the technical scheme provided by the embodiments of the present application, read and write traffic are all included in the running information for determining hotspot database partitions, and read and write traffic is all sampled for analyzing traffic characteristics and access patterns, so that hotspot database partitions generated by read traffic, write traffic, and mixed read and write traffic can be processed. In addition, since the target hotspot database partitions are identified by the management node, the order of magnitude of read and write traffic of all database partitions in the cluster dimension can be known, and through the relative relationship, database partitions that are significantly higher than the average level can be automatically identified. Compared with manually specifying a hotspot traffic threshold, this is more flexible and accurate, reduces parameter tuning and human intervention, and can also control the frequency of governance in the cluster range to avoid instability caused by excessive governance. For hotspot database partitions that cannot be governed by splitting, such as hotspot database partitions with centralized access patterns and sequential access patterns, the present scheme has two processing means, namely isolation and flow limiting, so that the influence of hotspot database partitions that cannot be governed by splitting on the operation of other database partitions can be effectively avoided. In addition, for hotspot database partitions that cannot be governed by splitting, the present scheme can automatically determine whether they meet the centralized access pattern and the sequential access pattern according to the request sampling modeling analysis, directly feed back the reason why they cannot be governed by splitting to the database operation and maintenance personnel, reduce the cost of intervention and troubleshooting of the operation and maintenance personnel, have a certain automatic attribution ability, and are friendly to the operation and maintenance personnel. The request sampling of the database partition is stored in a persistent manner, so that if the current situation cannot be handled or is not handled properly, the scene can be reproduced by querying the sampled sample access requests of the corresponding time period in the future, providing necessary materials for continuous iteration and evolution. After splitting the hotspot database partition, the traffic distribution of the sub-database partition is continuously observed, and the effectiveness of the splitting operation and maintenance action is automatically determined, so that incorrect governance on scenarios that cannot be correctly handled can be avoided.

[0170] The following will be described by taking an online shopping scenario as an example:

[0171] On a certain online shopping platform, a merchant launches a promotion activity for a hot commodity, the starting time of the promotion activity is 20:00 on April 6, 2022, and the ending time is 21:00 on April 6, 2022, and the inventory quantity of the hot commodity is stored as a record in database partition A in the distributed database system. The database partition A also records the inventory quantities of other commodities. After 20:00 on April 6, 2022, a large number of customers purchase the hot commodity on the online shopping platform, and when each customer purchases the hot commodity, the terminal of the customer will send an access request to the database partition A to the distributed database system, and the access request is used to update the inventory quantity of the hot commodity, for example: minus 1. The management node in the distributed database system finds that the database partition A is a stable hot database partition from 20:00 on April 6, 2022 to 20:07 on April 6, 2022, so it needs to be managed. The management node obtains a plurality of sample access requests sampled from the access requests to the database partition A between 20:00 on April 6, 2022 and 20:07 on April 6, 2022, and according to the plurality of sample access requests, it is found that the plurality of sample access requests are mostly concentrated on the record of the inventory quantity of the above-mentioned hot commodity, therefore, it is determined that the database partition A belongs to the concentrated access mode, in order to avoid the influence of the database partition A on other database partitions, the database partition A can be migrated to an isolated working node, and the isolated working node is used to provide storage and access services for the database partition A.

[0172] Figure 6 A structure schematic diagram of an electronic device provided by an embodiment of the application is shown. As shown in the figure, Figure 6 The electronic device includes a memory 1101 and a processor 1102. The memory 1101 can be configured to store various data to support operations on the electronic device. Examples of these data include instructions for operating any application or method on the electronic device. The memory 1101 can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0173] The memory 1101 is configured to store programs;

[0174] The processor 1102 is coupled to the memory 1101 and is configured to execute the programs stored in the memory 1101 to implement the data processing method provided by the above-mentioned method embodiments.

[0175] Further, asFigure 6 As shown, the electronic device also includes a communication component 1103, a display 1104, a power component 1105, an audio component 1106, and other components. Figure 6 Some components are only shown schematically, and it does not mean that the electronic device only includes Figure 6 the components shown.

[0176] Correspondingly, the embodiment of the present application further provides a computer readable storage medium storing a computer program, and the computer program is capable of realizing the steps or functions of the data processing method provided by each method embodiment when executed by a computer.

[0177] The device embodiments described above are only schematic, and the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme. Those skilled in the art can understand and implement it without creative labor.

[0178] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary universal hardware platforms, and of course, it can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiment.

[0179] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, wherein, The method comprises the following steps: determining a target hotspot database partition in a distributed database system; obtaining a plurality of target access requests for the target hotspot database partition; identifying an access mode corresponding to the target hotspot database partition according to the plurality of target access requests to obtain an identification result; processing the target hotspot database partition by using a corresponding hotspot processing strategy according to the identification result, including: when the access mode belongs to an alternative access mode that cannot disperse access traffic by partition splitting, migrating the target hotspot database partition to an isolated worker node or performing a flow limiting operation on the target hotspot database partition; the alternative access mode includes a sequential access mode; when the access mode belongs to the sequential access mode, a ratio of a number of primary keys in a longest subsequence of a primary key sequence to a number of primary keys in the primary key sequence is greater than or equal to a preset ratio threshold value, the primary key sequence is obtained by sorting primary keys accessed by the plurality of target access requests according to access times corresponding to the plurality of target access requests, and the longest subsequence is a longest increasing subsequence of the primary key sequence or a longest decreasing subsequence of the primary key sequence.

2. The method of claim 1, wherein, The method comprises the following steps: determining whether the access mode corresponding to the target hotspot database partition belongs to one of at least one alternative access mode according to the plurality of target access requests; the alternative access mode refers to a discovered access mode that cannot disperse access traffic by partition splitting.

3. The method of claim 2, wherein, The at least one alternative access mode includes a concentrated access mode. The method comprises the following steps: determining at least one point access request from the plurality of target access requests; determining a first primary key from at least one primary key accessed by the at least one point access request; determining a first proportion of point access requests in the plurality of target access requests that access the first primary key; if the first proportion is greater than or equal to a first preset proportion, determining that the access mode corresponding to the target hotspot database partition belongs to the concentrated access mode.

4. The method of claim 3, wherein, The method further comprises the following steps: determining at least one range access request from the plurality of target access requests; determining a first primary key range from at least one primary key range accessed by the at least one range access request; determining a second proportion of range access requests in the plurality of target access requests that access the first primary key range; if the second proportion is greater than or equal to a second preset proportion, determining that the access mode corresponding to the target hotspot database partition belongs to the concentrated access mode.

5. The method of claim 2, wherein, The at least one alternative access mode includes a sequential access mode. According to the plurality of target access requests, it is determined whether the access mode corresponding to the target hotspot database partition belongs to one of the at least one alternative access mode, comprising: According to the access time corresponding to the plurality of target access requests, the primary keys accessed by the plurality of target access requests are sorted to obtain a primary key sequence; Determine the longest subsequence of the primary key sequence; the longest subsequence is one of the longest increasing subsequence of the primary key sequence and the longest decreasing subsequence of the primary key sequence; If the ratio of the number of primary keys of the longest subsequence to the number of primary keys of the primary key sequence is greater than or equal to a preset ratio threshold, it is determined that the access mode corresponding to the target hotspot database partition belongs to the sequential access mode.

6. The method of any one of claims 1 to 5, wherein, Also includes: When the identification result shows that the access mode of the target hotspot database partition belongs to the alternative access mode that cannot disperse access traffic through partition splitting, according to the access mode corresponding to the target hotspot database partition, the reason why the target hotspot database partition cannot disperse access traffic through partition splitting is sent to the user.

7. The method of any one of claims 1 to 5, wherein, According to the identification result, the target hotspot database partition is processed by using a corresponding hotspot processing strategy, comprising: When the identification result shows that the access mode of the target hotspot database partition does not belong to the alternative access mode that cannot disperse access traffic through partition splitting, the target hotspot database partition is processed by using a partition splitting strategy.

8. The method of claim 7, wherein, Also includes: Determine the distribution balance degree of access traffic between two sub-database partitions; the two sub-database partitions are obtained after the target hotspot database partition is processed by the splitting processing; According to the distribution balance degree, it is determined whether the splitting processing is effective; When it is determined that the splitting processing is not effective, the invalid splitting number of the database table to which the target hotspot database partition belongs is obtained; When the invalid splitting number is greater than or equal to a preset number, the processing based on the partition splitting strategy for the database table is suspended.

9. The method of any one of claims 1 to 5, wherein, Obtain a plurality of target access requests for the target hotspot database partition, comprising: Send a data acquisition request to the working node where the target hotspot database partition is located, so that the working node acquires the plurality of target access requests within a first preset time period according to the data acquisition request; Receive the plurality of target access requests sent by the working node; Wherein, the working node samples the access request data for the target hotspot database partition within the first preset time period to obtain the plurality of target access requests, and performs persistent storage processing on the plurality of target access requests.

10. The method of any one of claims 1 to 5, wherein, The distributed database system includes a database table; the database table corresponds to a plurality of database partitions; Determine a target hotspot database partition in a distributed database system, comprising: Every preset time interval, according to the running information of the plurality of database partitions within a second preset time period, the plurality of database partitions are sorted; According to the sorting result, it is determined that the hotspot database partition corresponding to the database table; According to a hotspot database partition corresponding to the database table, a target hotspot database partition in the distributed database system is determined.

11. A distributed database system, wherein, Comprise: A management node and a plurality of worker nodes; at least one database partition is arranged on the worker node; The management node is used for: Determining a target hotspot database partition in a distributed database system; Obtaining a plurality of target access requests for the target hotspot database partition; According to the plurality of target access requests, an access mode corresponding to the target hotspot database partition is identified to obtain an identification result; According to the identification result, a corresponding hotspot processing strategy is used to process the target hotspot database partition, including: when the access mode belongs to a sequential access mode, migrating the target hotspot database partition to an isolated worker node or performing a flow limiting operation on the target hotspot database partition; When the access mode belongs to a sequential access mode, the ratio of the number of primary keys in the longest subsequence of the primary key sequence to the number of primary keys in the primary key sequence is greater than or equal to a preset ratio threshold value, the primary key sequence is obtained by sorting the primary keys accessed by the plurality of target access requests according to the access time corresponding to the plurality of target access requests, and the longest subsequence is the longest increasing subsequence of the primary key sequence or the longest decreasing subsequence of the primary key sequence.

12. An electronic device, comprising: Comprise: Memory and processor, wherein, The memory is used for storing programs; The processor is coupled with the memory, and is used for executing the programs stored in the memory to realize the data processing method in any one of claims 1 to 10.

13. A computer readable storage medium storing a computer program, wherein, The computer program can realize the data processing method in any one of claims 1 to 10 when executed by a computer. The computer program can realize the data processing method in any one of claims 1 to 10 when executed by a computer.

Images