Data processing method and electronic device

Abstract

The application provides a data processing method and an electronic device, which can be applied to the technical field of data storage. The method comprises the following steps: in response to a first data site in a data processing system receiving a data processing request sent by a client, analyzing the data processing request, and determining a target data shard required for operation of the data processing request; in a case where it is determined that the first data site has a locking permission for the target data shard, adding a shard lock to the target data shard; based on the data processing request, processing the target data shard to obtain a data processing result; in a case where it is determined that the first data site does not have the locking permission, determining a second data site having the locking permission for the target data shard according to a permission attribution relationship between a plurality of data shards and a plurality of data sites; and forwarding the data processing request to the second data site, so that the second data site processes the target data shard to obtain the data processing result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, in particular to a data processing method and an electronic device. BACKGROUND

[0002] In a distributed data processing scenario, a dual-active storage architecture refers to two or more data sites simultaneously performing read and write operations, and any site failure will not cause business interruption. The architecture can achieve disaster recovery through multiple copy storage of data. In a traditional dual-active storage architecture, there is a master node, and all instructions and requests to the storage architecture need to be responded by the master node, which leads to a high load of the master node, while a large amount of resources of other slave nodes are in idle state, resulting in an unbalanced load of the dual-active storage architecture.

[0003] To solve the above problems, multiple data sites in the dual-active dual-write architecture can respond to instructions or requests. However, when multiple data sites simultaneously receive processing requests for the same data, data inconsistency may be caused due to concurrent operations, thereby affecting the authenticity of the data. SUMMARY

[0004] In view of the above problems, the present application provides a data processing method and an electronic device.

[0005] According to a first aspect of the present application, a data processing method is provided, comprising: in response to a first data site in a data processing system receiving a data processing request sent by a client, parsing the data processing request, and determining a target data shard required for operation of executing the data processing request, wherein the data processing system comprises a plurality of data sites, and the plurality of data sites each store a plurality of data shards obtained by block processing of a data volume of the data processing system; in a case where it is determined that the first data site has a locking permission for the target data shard, adding a shard lock to the target data shard, so that no other data processing request for the target data shard is responded to before data processing of the target data shard based on the data processing request is completed; processing the target data shard stored by the first data site based on the data processing request to obtain a data processing result; in a case where it is determined that the first data site does not have the locking permission for the target data shard, determining a second data site having the locking permission for the target data shard according to a permission attribution relationship between the plurality of data shards and the plurality of data sites; and forwarding the data processing request to the second data site, so that the second data site adds a shard lock to the target data shard based on the data processing request, and processes the target data shard stored by the second data site to obtain a data processing result.

[0006] The second aspect of the present application provides a data processing apparatus, comprising: a request analysis module, configured to analyze a data processing request received by a first data site in a data processing system, and determine a target data shard required for operation of the data processing request, wherein the data processing system comprises a plurality of data sites, and each of the plurality of data sites stores a plurality of data shards processed by a data volume of the data processing system; a shard locking module, configured to lock the target data shard in a case where the first data site has a locking right to the target data shard, so that no other data processing request for the target data shard is responded to before data processing of the target data shard based on the data processing request is completed; a data processing module, configured to process the target data shard stored by the first data site based on the data processing request, and obtain a data processing result; a site determination module, configured to determine a second data site having a locking right to the target data shard according to a right attribution relationship between the plurality of data shards and the plurality of data sites in a case where the first data site does not have the locking right to the target data shard; and a request forwarding module, configured to forward the data processing request to the second data site, so that the second data site locks the target data shard based on the data processing request, and processes the target data shard stored by the second data site to obtain the data processing result.

[0007] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0008] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0009] The fifth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the method. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above content of the present application and other purposes, features and advantages will be more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0011] Figure 1 An application scenario diagram of the data processing method and the electronic device according to the embodiments of the present application is shown.

[0012] Figure 2 A flowchart of the data processing method according to the embodiments of the present application is shown.

[0013] Figure 3 A flow chart illustrating a process of adjusting permissions of a data slice according to the data processing method of an embodiment of the present application is shown.

[0014] Figure 4A An interaction diagram illustrating the data processing method according to an embodiment of the present application is shown.

[0015] Figure 4B An interaction diagram illustrating the data processing method according to another embodiment of the present application is shown.

[0016] Figure 5 A structure block diagram of a data processing apparatus according to an embodiment of the present application is shown.

[0017] Figure 6 A block diagram of an electronic device suitable for implementing the data processing method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present application, and is not intended to limit the scope of the present application. In the following detailed description of the embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0019] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological expressions thereof, such as "including," "includes," "include," "contains," "containing," and so forth, shall not be taken to exclude

[0020] All terms used herein including technical and scientific terms have the same meanings as commonly understood by one of ordinary skill in the art unless otherwise defined herein. It should be noted that the terms used herein are merely specific examples for the described embodiments and should not be limiting, and should also be interpreted not to be idealized or overly formalistic.

[0021] In the case where expressions similar to "at least one of A, B, and C, and the like" are used, it should be generally construed in the meaning that the expression is commonly understood by one of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, and the like).

[0022] Embodiments of the present application provide a data processing method, comprising: in response to a first data site in a data processing system receiving a data processing request sent by a client, parsing the data processing request, and determining a target data shard required for operation of the data processing request, wherein the data processing system comprises a plurality of data sites, and each of the plurality of data sites stores a plurality of data shards processed by a data volume of the data processing system; in a case where it is determined that the first data site has a locking right to the target data shard, adding a shard lock to the target data shard, so that no other data processing request for the target data shard is responded to before data processing of the target data shard based on the data processing request is completed; processing the target data shard stored by the first data site based on the data processing request to obtain a data processing result; in a case where it is determined that the first data site does not have the locking right to the target data shard, determining a second data site having the locking right to the target data shard according to a right attribution relationship between the plurality of data shards and the plurality of data sites; and forwarding the data processing request to the second data site, so that the second data site adds a shard lock to the target data shard based on the data processing request, and processes the target data shard stored by the second data site to obtain the data processing result.

[0023] Figure 1 An application scenario diagram of the data processing method and the electronic device according to embodiments of the present application is shown.

[0024] As shown in Figure 1 , the application scenario 100 according to the embodiment can include a client 110, a network 120, and a data processing system 130, wherein the data processing system 130 includes a data site one 131, a data site two 132, and a data site three 133. The network 120 is a medium for providing a communication link between the client 110 and the data processing system 130. The network 120 can include various connection types, such as wired, wireless communication links, or fiber optic cables, and the like.

[0025] A user can use the client 110 to interact with the data processing system 130 through the network 120 to receive or send messages, and the like. Various communication client applications can be installed on the client 110, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, and the like (only as examples).

[0026] The client 110 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0027] The data processing system 130 can be a data system providing data support for providing various services, for example, a data system providing support for a website browsed by a user using the client 110. The data system can analyze and process data such as data processing requests submitted by a user, and feed back the processing result to a terminal device. A plurality of data sites included in the data processing system, for example, a data site one 131, a data site two 132, and a data site three 133, all store all data of the data processing system 130, and the data site one 131, the data site two 132, and the data site three 133 will process the data stored respectively according to the data processing request, to ensure the data consistency between the data sites, so as to recover the data by using the data in other data sites in the case of failure of some data sites.

[0028] The following will be based on Figure 1 The scenario described, the data processing method of the application embodiment is described in detail. Figure 2 、 Figure 3 、 Figure 4A 、 Figure 4B The data processing method of the application embodiment is described in detail.

[0029] Figure 2 The flow chart of the data processing method according to the application embodiment is shown.

[0030] As Figure 2 The data processing method of the embodiment includes operation S210~operation S250, and the data processing method can be executed by the first data site.

[0031] In operation S210, in response to the first data site in the data processing system receiving the data processing request sent by the client, the data processing request is parsed, and the target data slice required for the operation of executing the data processing request is determined.

[0032] According to the embodiment of the application, the data processing system includes a plurality of data sites, and the plurality of data sites all store a plurality of data slices obtained by block processing of a data volume of the data processing system.

[0033] According to the embodiment of the application, the data processing request is used to request processing of one of the plurality of data slices, and the above processing can include at least one of a plurality of processing such as reading and writing. By parsing the data processing request, the data slice to which the data processing request is directed can be determined, so that when the data processing request is executed, the data slice needs to be read and written, and the data slice can be determined as the target data slice.

[0034] According to an embodiment of the present application, the first data site refers to a data site that first receives a data processing request in the data processing system. In different data processing processes, for different data processing requests, even if the data site that first receives the data processing request in the data processing system is different, the different data sites are respectively regarded as the first data site in the data processing process.

[0035] In operation S220, in a case where it is determined that the first data site has the locking permission for the target data shard, a shard lock is added to the target data shard, so that no other data processing request for the target data shard is responded to before the data processing of the target data shard based on the data processing request is completed.

[0036] According to an embodiment of the present application, the plurality of data sites also respectively store a plurality of data shards and a permission attribution relationship between the plurality of data shards and the plurality of data sites, that is, which data site can add a shard lock to each data shard. For each data shard, only the data site that has the locking permission can add a shard lock to the data shard and initiates an operation on the data shard first after the shard lock is added. Other data sites need to respond to an indication sent by the data site that has the locking permission to perform an operation on the data shard again, so as to ensure the data consistency between the plurality of data sites.

[0037] According to an embodiment of the present application, the first data site can determine the data site that can add a shard lock to the target data shard according to the permission attribution relationship, and if the data site is the first data site, it is determined that the first data site has the locking permission for the target data shard.

[0038] According to an embodiment of the present application, in a case where it is determined that the first data site has the locking permission for the target data shard, the first data site adds a shard lock to the target data shard, and the shard lock can be released after the first data site determines that the plurality of data sites in the data processing system have completed the operation on the target data shard, so as to ensure that the target data shard is continuously in the state of being added with the shard lock during the operation execution. In a case where the target data shard is added with the shard lock and the shard lock is not released, other data processing requests for the target data shard will not be responded to, so that the consistency of data in the target data shard can be ensured, and the occurrence of a situation that affects the accuracy of data, such as dirty reading, can be prevented.

[0039] In operation S230, based on the data processing request, the target data shard stored by the first data site is processed to obtain a data processing result.

[0040] According to an embodiment of the present application, in response to the data processing request, a processing manner of the target data shard indicated by the data processing request is determined, and the target data shard is processed according to the processing manner to obtain a data processing result.

[0041] In operation S240, in a case where it is determined that the first data site does not have the locking permission for the target data shard, a second data site having the locking permission for the target data shard is determined according to the permission attribution relationship between the plurality of data shards and the plurality of data sites.

[0042] According to an embodiment of the present application, in a case where it is determined that the first data site does not have the locking permission for the target data shard, the data site capable of locking the target data shard determined by the permission attribution relationship is determined as the second data site.

[0043] In operation S250, the data processing request is forwarded to the second data site, so that the second data site locks the target data shard based on the data processing request and processes the target data shard stored in the second data site to obtain a data processing result.

[0044] According to an embodiment of the present application, since the first data site does not have the locking permission for the target data shard, the second data site has the locking permission for the target data shard but does not receive the data processing request, so that the data processing request can be forwarded by the first data site to the second data site for subsequent processing.

[0045] Specifically, after receiving the data processing request, the second data site is consistent with the operation of the first data site in locally responding to the data processing request, that is, locking the target data shard, and then processing the target data shard stored in the second data site to obtain a data processing result.

[0046] According to an embodiment of the present application, by analyzing the data processing request to determine the target data shard, judging the main data site responding to the data processing request according to the locking permission, ensuring that the same data shard is locked during processing, and avoiding the problem of data inconsistency caused by concurrent operation. In a case where the first data shard does not have the locking permission, the data processing request is forwarded in time to the data site having the locking permission determined by the permission attribution relationship, so as to ensure the orderliness and accuracy of data processing, and improve the stability of the distributed data processing system.

[0047] According to an embodiment of the present application, the data processing method further comprises: performing block processing on the data volume according to a predetermined block size to obtain a plurality of data shards; and distributing the locking permission of each of the plurality of data shards to the plurality of data sites according to the number of sites of the plurality of data sites and the number of shards of the plurality of data shards.

[0048] According to an embodiment of the present application, all data in the data processing system is stored in a data volume, and the data volume can be divided into multiple data shards of the same size according to a predetermined block size, preferably, the predetermined block size can be 8M.

[0049] According to an embodiment of the present application, the locking permissions of the multiple data shards are evenly distributed according to the number of sites and the number of shards, so that the number of locking permissions allocated to each data site is the same or similar. In the case where the number of shards is an integer multiple of the number of sites, each data site can be allocated the same number of locking permissions, and in the case where the number of shards is not an integer multiple of the number of sites, the number of locking permissions allocated to each data site differs by no more than 1.

[0050] According to an embodiment of the present application, the data volume is divided into blocks according to a predetermined block size, and the locking permissions are allocated based on the number of sites and the number of shards, which realizes reasonable allocation of data shards and processing permissions. This structured allocation method ensures the standardization of permission management, avoids permission confusion, lays a foundation for subsequent orderly data processing, and improves the manageability of the system.

[0051] According to an embodiment of the present application, the locking permissions of the multiple data shards are allocated to the multiple data sites according to the number of sites and the number of shards, including: sequentially allocating the locking permissions of the multiple data shards to the multiple data sites according to the number of sites.

[0052] According to an embodiment of the present application, all data shards are sorted according to the sequence number, and the round-robin method is used to allocate according to the number of sites. That is, starting from the first shard, each site is sequentially assigned according to the sequence number, and after one round of allocation, the first site is recycled for further allocation until the locking permissions of the multiple data shards are all allocated.

[0053] According to an embodiment of the present application, the locking permissions of the data shards are sequentially allocated according to the number of sites, which makes the permission allocation process simple and efficient, and facilitates the system to quickly complete the initialization configuration. The ordered allocation method ensures the preliminary balance of the load of each data site, reduces the single-point pressure caused by the concentration of permissions, and improves the initial running efficiency of the system.

[0054] According to an embodiment of the present application, the data processing method further comprises: analyzing, according to a preset time period, the at least one historical processing record of each of the plurality of data shards to obtain a first processing number of the plurality of data shards being processed at the first data site with a shard lock and a second processing number of the plurality of data shards being processed at the second data site with a shard lock; comparing the first processing number and the second processing number of each of the plurality of data shards to determine whether there is a to-be-transferred data shard in the plurality of data shards that needs to transfer the locking permission, wherein, in a case where the first processing number of the data shard is less than the second processing number of the data shard in a continuous preset number of periods, the data shard in which the above case occurs is determined as the to-be-transferred data shard; and in a case where there is a to-be-transferred data shard, in response to a permission transfer instruction sent by the data processing system, updating the permission attribution relationship to transfer the locking permission of the to-be-transferred data shard to a target data site, wherein the target data site is a data site in the data processing system that receives a data processing request number for operating the to-be-transferred data shard that meets a predetermined condition.

[0055] According to an embodiment of the present application, since the first data site is the data site that first receives the data processing request in the data processing system, if the first data site has the locking permission of the target data shard, subsequent data processing can be directly performed at the first data site, and if the first data site does not have the locking permission of the target data shard, the data processing request needs to be forwarded to the second data site for subsequent data processing by the second data site.

[0056] Since the processes of the first data site and the second data site for subsequent data processing are consistent, the difference between the above two cases is whether to perform a data processing request forwarding once. Compared with the way of forwarding a data processing request once and then performing subsequent processing, the way of directly performing data processing at the first data site without forwarding has less communication load and higher processing efficiency.

[0057] Therefore, the historical processing records of each data shard can be analyzed according to a preset time period to determine whether the number of data processing at the first data site (the first processing number) in the historical processing record of each data shard is more or the number of data processing after forwarding a data processing request (the second processing number) is more. If it is found that the first processing number is less than the second processing number of the data shard in a continuous preset number of periods, it can be determined that the communication load of the processing process of the data shard is high.

[0058] Specifically, a migration count with an initial value of 0 can be set, and for a data shard, the migration count is incremented by 1 when the first processing time is less than the second processing time, and the migration count is set to 0 when the first processing time is greater than or equal to the second processing time. In the case where the migration count reaches a preset number, it can be determined that the data shard has the condition that the first processing time is less than the second processing time of the data shard in the last preset number of periods.

[0059] To address the problem of high communication load in the data processing process of the data shard, various methods can be used for correction. For example, the request sending logic of the upstream application can be adjusted so that the data processing request for the data shard is accurately allocated to the data site that has the locking permission of the data shard. The ownership of the locking permission of the data shard can also be changed to allocate the locking permission to the data site that first receives the data processing request, so that in the subsequent data processing process, after the data site receives the data processing request for the data shard again, the data site can directly add a shard lock to the data shard without forwarding the data processing request.

[0060] Among the above correction methods, the complexity of modifying the request sending logic of the upstream application is higher, and the implementation is more difficult. Therefore, the method of changing the ownership of the permission can be used to reduce the communication load in the data processing process.

[0061] Specifically, the data shard whose locking permission ownership needs to be changed can be regarded as a data shard to be transferred, and the data site that first receives the data processing request for the data shard to be transferred in the historical processing record and meets the predetermined condition can be determined as a target data site. The predetermined condition can be set as the number of times of receiving the data processing request.

[0062] The data processing system sends a permission transfer instruction for the data shard to be transferred, and the permission transfer instruction includes the mark of the data shard to be transferred, the data site to which the locking permission of the data shard to be transferred currently belongs, and the target data site. The permission transfer instruction is broadcast to multiple data sites in the data processing system.

[0063] Taking the first data site as an example, after receiving the permission transfer instruction, the first data site updates the saved permission ownership relationship, so that the locking permission of the data shard to be transferred is transferred to the target data site. The operations of other data sites in the data processing system after receiving the permission transfer instruction are the same as those of the first data site, and will not be described here.

[0064] According to the embodiments of the present application, the data shard to be transferred is determined by analyzing the historical processing records, and the locking permission is transferred according to the request number, so as to realize dynamic optimization of the permission. The permission adjustment based on the actual processing demand makes the locking permission tilt to the site that processes the corresponding data shard more frequently, reduces the delay caused by request forwarding, and improves the overall efficiency of data processing.

[0065] According to the embodiments of the present application, since the first processing number and the second processing number of each data shard need to be counted and compared periodically, in the data processing process, the first data site is determined to have the locking permission for the target data shard, and the first processing number of the data shard is incremented by 1 after the locking is completed by the first data site. The first data site is determined not to have the locking permission for the target data shard, and the second processing number of the data shard is incremented by 1 after the locking is completed by the second data site.

[0066] According to the above manner of recording the first processing number and the second processing number, after reaching the preset time period, the first processing number and the second processing number of each data shard can be obtained directly by reading the corresponding counters of the multiple data shards without analyzing the at least one historical processing record of each of the multiple data shards, thereby reducing the periodic data processing amount, improving the judgment efficiency of the permission transfer demand, and reducing the system load.

[0067] According to the embodiments of the present application, the locking permission of the data shard to be transferred is transferred to the target data site, including: in the case that there is an ongoing data processing request for the data shard to be transferred in the data processing system, placing a new data processing request for the data shard to be transferred received by the data processing system in a to-be-processed queue; and in the case that it is determined that the ongoing data processing request is executed or there is no executing data processing request in the data processing system, transferring the locking permission of the data shard to be transferred to the target data site.

[0068] According to the embodiments of the present application, before the locking permission transfer, the data processing request for the data shard to be transferred in the data processing system is judged. In the case that it is determined that there is an ongoing data processing request for the data shard to be transferred, the locking permission transfer operation is not performed first, and after the currently executing data processing request is executed, the locking permission transfer is performed. In this process, the new data processing request for the data shard to be transferred received by the data processing system is not executed temporarily, and the new data processing request is placed in the to-be-processed queue according to the receiving order, so as to avoid the delay of the locking permission transfer caused by waiting for the new data processing request to be executed.

[0069] According to an embodiment of the present application, the lock permission of the data shard to be transferred is transferred to the target data site after it is determined that the data processing request being executed is executed.

[0070] According to another embodiment of the present application, in the case that it is determined that there is no data processing request being executed for the data shard to be transferred by judging the data processing request for the data shard to be transferred in the data processing system, the lock permission of the data shard to be transferred can be directly transferred to the target data site.

[0071] In the above-mentioned execution of the lock permission transfer, the newly added data processing request received in the process can also be put into the to-be-processed queue.

[0072] According to an embodiment of the present application, when the lock permission transfer is needed, if there is a data processing request being executed in the data processing system, the newly added data processing request is put into the to-be-processed queue and is not processed temporarily. After the data processing request is executed, the permission transfer is performed, which ensures the continuity and integrity of data processing in the permission transfer process. The request interruption or data conflict caused by the permission transfer is avoided, and the stability of the system in the dynamic adjustment process is ensured.

[0073] According to an embodiment of the present application, the data processing method further comprises: in the case that there is at least one newly added data processing request in the to-be-processed queue, responding to the at least one newly added data processing request in turn according to the order of the at least one newly added data processing request entering the queue.

[0074] According to an embodiment of the present application, after the transfer of the lock permission is completed, the to-be-processed queue can be traversed to determine whether there is a newly added data processing request waiting for execution, and if there is a newly added data processing request, the newly added data processing request in the to-be-processed queue is responded to first, and then the newly added data processing request received in real time is responded to.

[0075] The processing order of the newly added data processing request can be ensured to be correct and accurate by responding to the at least one newly added data processing request in turn according to the order of the at least one newly added data processing request entering the queue.

[0076] According to an embodiment of the present application, the newly added request in the to-be-processed queue is responded to according to the order of entering, which follows the principle of first come first served and ensures the fairness of request processing. Through the ordered request response mode, the request congestion or priority confusion is avoided, and the predictability of the system response by the user is improved.

[0077] Figure 3 A flowchart of the permission adjustment of a data shard by the data processing method according to an embodiment of the present application is shown.

[0078] As Figure 3As shown, the flow includes operation S301 to operation S313.

[0079] In operation S301, it is judged whether a preset time period has been reached. If the preset time period has been reached, operation S302 is executed; if the preset time period has not been reached, operation S301 is returned to.

[0080] In operation S302, a first processing number and a second processing number are acquired. The first processing number and the second processing number can be acquired in the manner of the above-mentioned embodiments, which is not limited herein.

[0081] In operation S303, it is judged whether the first processing number is less than the second processing number. If yes, operation S305 is executed; if no, operation S304 is executed.

[0082] In operation S304, the migration count is cleared. Operation S301 is returned to.

[0083] In operation S305, the migration count is added by 1.

[0084] In operation S306, it is judged whether the migration count reaches a preset number. If yes, operation S307 is executed; if no, operation S301 is returned to.

[0085] In operation S307, a target data site is determined.

[0086] In operation S308, it is judged whether there is an executing data processing request. If yes, operation S309 is executed; if no, operation S311 is executed.

[0087] In operation S309, the data processing request is waited for execution, and the new data request in this process is stored in a to-be-processed queue.

[0088] In operation S310, it is judged whether the waiting duration satisfies a waiting period. If yes, operation S308 is returned to; if no, operation S310 is returned to.

[0089] In operation S311, the locking permission is transferred to the target data site.

[0090] In operation S312, it is judged whether there is an unprocessed new data processing request in the to-be-processed queue. If yes, operation S313 is executed; if no, operation S301 is returned to.

[0091] In operation S313, the new data processing request in the to-be-processed queue is responded to in sequence. Operation S301 is returned to.

[0092] According to the embodiments of the present application, for a data shard, when the first processing number is greater than the second processing number, but the second processing number is relatively high, it can be determined that although data processing requests for the data shard are mostly assigned to data sites with locking authority for the data shard, a relatively large number of data processing requests are assigned to data sites without locking authority for the data shard. In this case, the data processing system still has a relatively high communication load.

[0093] Therefore, by counting the first processing number and the second processing number of each data shard, when the first processing number is greater than the second processing number and the second processing number reaches a threshold value, the number of times that data processing requests corresponding to the data shard are assigned to different data sites can be determined, the data site without locking authority for the data shard and with the highest assignment number can be determined as a shared data site, and the data processing system can issue an instruction to authorize the shared data site to also have locking authority for the data shard.

[0094] In this case, there are two data sites with locking authority for the data shard in the data processing system, and when the two data sites receive data processing requests for the data shard, it can be determined whether the other party is performing data processing for the data shard by the locking status of the data shard. If the other party is performing data processing for the data shard, the current party waits until the other party finishes processing, and then starts to perform. If the other party is not performing data processing for the data shard, a shard lock is added to the data shard, and the current received data processing request is started to be performed.

[0095] When a data site other than the two data sites receives a data processing request for the data shard, it can be determined whether the load of the two data sites with locking authority is relatively low, and the data processing request is forwarded to the data site with relatively low load for processing.

[0096] According to the embodiments of the present application, by setting a data site with shared locking authority, the data site that receives a large number of data processing requests for the data shard can be authorized while ensuring that the original data site still has locking authority, so that the two data sites can both initiate processing of the data shard. By adding a shard lock mechanism, it is ensured that the two data sites do not concurrently process different data processing requests, and the consistency of the data is ensured. By determining the load and then forwarding and assigning the request, the resources of each data site can be fully utilized, and the efficiency of data processing and resource utilization can be improved.

[0097] Since in the dual-active storage architecture, multiple data sites need to respond to data processing instructions to ensure the data consistency of multiple data sites. Therefore, after the data site with the locking permission for the target data shard locks the target data shard, the data site can send an instruction to other data sites in the data processing system to instruct the other data sites to perform the operation corresponding to the data processing instruction.

[0098] According to an embodiment of the present application, in the case where the first data site completes locking the target data shard, a remote processing instruction is generated based on the processing logic indicated by the data processing request; the remote processing instruction is sent to other data sites in the multiple data sites except the first data site, so that the other data sites process the target data shard stored in the other data sites according to the remote processing instruction, and send a remote processing completion message to the first data site in the case where the processing is completed; and in response to the first data site receiving the remote processing completion message sent by the other data sites, the shard lock is released, it is determined that the data processing request is executed and an execution completion message is generated.

[0099] According to an embodiment of the present application, after the first data site completes locking the target shard, a remote processing instruction is generated based on the processing logic indicated by the data processing request, wherein the remote processing instruction is used to instruct other data sites except the first data site to execute the processing logic, so as to ensure the consistency of data in multiple data sites.

[0100] The remote processing instruction is sent to multiple other data sites, so that after the other data sites receive the remote processing instruction, the processing logic indicated by the data processing request is determined, and the target data shard stored in each data site is processed according to the processing logic, after the processing is completed, a remote processing completion message is generated, and the message is returned to the first data site in a point-to-point manner.

[0101] The first data site can determine that the other data site sending the remote processing completion message has completed data processing after receiving one remote processing completion message sent by the other data site, in the case where the first data site determines that multiple other data sites have completed data processing and the first data site has also completed data processing, it can be determined that multiple data sites in the data processing system have completed data processing, therefore, the shard lock for the target data shard can be released, and it is determined that the current data processing request has been executed.

[0102] According to an embodiment of the present application, the first data site sends a remote processing instruction to other sites after locking, synchronously processes the target data shards of each site, and releases the lock after receiving the completion message, which realizes the consistency of data processing in a distributed environment. Multi-site cooperative processing ensures the synchronous update of system data, and the release of the lock mechanism timely releases the restriction on the data shard, improving the availability of data.

[0103] According to another embodiment of the present application, in the case that the first data site does not have the locking permission for the target data shard and forwards the data processing instruction to the second data site, in response to receiving the remote processing instruction generated by the second data site based on the processing logic indicated by the data processing request, the target data shard stored by the first data site is processed to obtain a data processing result; a remote processing completion message is sent to the second data site, so that after the second data site receives the remote processing completion message, the shard lock is released and an execution completion message is generated; and in response to receiving the execution completion message sent by the second data site, it is determined that the data request execution is completed.

[0104] According to an embodiment of the present application, in the case that the first data site does not have the locking permission for the target data shard, the first data site can forward the data processing instruction to the second data site, and the second data site can initiate the data processing of the target data shard by locking the target data shard.

[0105] The process of locking and data processing of the target data shard by the second data site is the same as that of the first data site, and will not be described here. After the first data site receives the remote processing instruction sent by the second data site, the target data shard stored by the first data site is processed according to the processing logic corresponding to the remote processing instruction, and a remote processing completion message is sent to the second data site after the processing is completed, so that the second data site can determine that the data processing of other data sites including the first data site has been completed according to the remote processing completion message.

[0106] According to an embodiment of the present application, since the first data site is the data site that receives the data processing request sent by the client, the first data site can also return information to the client to ensure that the feedback content can be successfully received by the client through the channel between the first data site and the client.

[0107] Therefore, after the second data site determines that the data processing request has been executed, the shard lock for the target data shard can be released, and an execution completion message can be generated and sent to the first data site, and the first data site can determine that the data processing request has been executed after receiving the execution completion message.

[0108] According to an embodiment of the present application, the first data site responds to the remote processing instruction of the second data site and feeds back the completion message, cooperates with the second data site to complete the lock release and result generation, and forms a complete cross-site cooperative processing flow. This cooperative mechanism ensures that even if the processing request is forwarded, the system data can still be consistent, and the cooperative efficiency of the distributed system is improved.

[0109] According to an embodiment of the present application, the data processing method further comprises: in a case where it is determined that the data processing request is executed completely, returning an execution completion message and a data processing result to the client.

[0110] According to an embodiment of the present application, in a case where the first data site determines that the data processing request is executed completely through any of the above-mentioned manners, an execution completion message can be sent to the client, and a data processing result is returned to the client, wherein, since the data saved in the plurality of data sites in the data processing system are consistent, the data processing result can be directly obtained from the target data shard stored in the first data site.

[0111] According to an embodiment of the present application, the result and the execution completion message are returned to the client after processing, the processing state and the result are timely fed back to the user, and the user experience is improved. The complete result feedback mechanism ensures that the user can know the request processing condition in time, and the interactivity and reliability of the system are enhanced.

[0112] In the data processing process, there are some differences between the interaction process of directly locking the target data shard by the first data site and the interaction process of locking the target data shard by the second data site. The following will describe the above-mentioned two interaction processes respectively. Figure 4A and Figure 4B Both of them show the case of two data sites. Figure 4A and Figure 4B Both of them show the case of two data sites.

[0113] Figure 4A An interaction diagram of the data processing method according to an embodiment of the present application is shown.

[0114] As shown in Figure 4A , the interaction process comprises operation S401 to operation S409.

[0115] In operation S401, the client sends a data processing request to the first data site.

[0116] In operation S402, the first data site parses the data processing request and determines a target data shard.

[0117] In operation S403, after analysis, it is determined that the first data site can add a shard lock to the target data shard, and then the first data site completes the locking operation on the target data shard.

[0118] In operation S404, the first data site processes the target data shard and generates a remote processing instruction.

[0119] In operation S405, the first data site sends the remote processing instruction to the second data site.

[0120] At operation S406, the second data site processes the target data shard according to the remote processing instruction.

[0121] At operation S407, the second data site sends a remote processing completion message to the first data site.

[0122] At operation S408, the first data site determines that the local and other data sites have completed processing of the target data shard.

[0123] At operation S409, the first data site sends an execution completion message and a data processing result to the client.

[0124] Figure 4B An interaction diagram of a data processing method according to another embodiment of the application is shown.

[0125] As Figure 4B shown, the interaction process includes operation S401, operation S402, and operations S410-S418.

[0126] At operation S401, the client sends a data processing request to the first data site.

[0127] At operation S402, the first data site parses the data processing request and determines a target data shard. Operation S410 is then performed.

[0128] At operation S410, the first data site forwards the processing request to the second data site.

[0129] At operation S411, the second data site performs a locking operation on the target data shard.

[0130] At operation S412, the second data shard processes the target data shard and generates a remote processing instruction.

[0131] At operation S414, the second data site sends the remote processing instruction to the first data site.

[0132] At operation S414, the first data site processes the target data shard according to the remote processing instruction.

[0133] At operation S415, the first data site sends a remote processing completion message to the second data site.

[0134] At operation S416, the second data site determines that the local and other data sites have completed processing of the target data shard.

[0135] At operation S417, the second data site sends an execution completion message to the first data site.

[0136] At operation S418, the first data site forwards the execution completion message to the client and sends the data processing result to the client.

[0137] Based on the above data processing method, the application further provides a data processing device. The following will be combined with the Figure 5 The device is described in detail.

[0138] Figure 5 The structure block diagram of the data processing device according to the embodiment of the application is shown.

[0139] As Figure 5 shown, the data processing device 500 of the embodiment includes a request parsing module 510, a shard locking module 520, a data processing module 530, a site determining module 540 and a request forwarding module 550.

[0140] The request parsing module 510 is configured to, in response to the first data site in the data processing system receiving a data processing request sent by a client, parse the data processing request, and determine a target data shard required for operation of executing the data processing request, wherein the data processing system includes a plurality of data sites, and each of the plurality of data sites stores a plurality of data shards processed by a data volume of the data processing system in blocks. In an embodiment, the request parsing module 510 can be configured to perform operation S210 described above, and details are not repeated here.

[0141] The shard locking module 520 is configured to, in a case where it is determined that the first data site has a locking permission for the target data shard, lock the target data shard, so that no other data processing request for the target data shard is responded to before the data processing of the target data shard based on the data processing request is completed. In an embodiment, the shard locking module 520 can be configured to perform operation S220 described above, and details are not repeated here.

[0142] The data processing module 530 is configured to process the target data shard stored by the first data site based on the data processing request, to obtain a data processing result. In an embodiment, the data processing module 530 can be configured to perform operation S230 described above, and details are not repeated here.

[0143] The site determining module 540 is configured to, in a case where it is determined that the first data site does not have a locking permission for the target data shard, determine a second data site having a locking permission for the target data shard according to a permission attribution relationship between the plurality of data shards and the plurality of data sites. In an embodiment, the site determining module 540 can be configured to perform operation S240 described above, and details are not repeated here.

[0144] The request forwarding module 550 is configured to forward the data processing request to the second data site, so that the second data site acquires a shard lock on the target data shard based on the data processing request, and processes the target data shard stored in the second data site to obtain a data processing result. In an embodiment, the request forwarding module 550 can be configured to perform the operation S250 described above, and details are not described herein again.

[0145] According to an embodiment of the present application, the data processing apparatus 500 further comprises a data chunking module and a permission assigning module.

[0146] The data chunking module is configured to chunk the data volume according to a predetermined chunk size to obtain a plurality of data shards.

[0147] The permission assigning module is configured to assign the locking permission of each of the plurality of data shards to the plurality of data sites according to a number of sites of the plurality of data sites and a number of shards of the plurality of data shards.

[0148] According to an embodiment of the present application, the permission assigning module comprises a permission assigning sub-module.

[0149] The permission assigning sub-module is configured to assign the locking permission of each of the plurality of data shards to the plurality of data sites in sequence according to the number of sites.

[0150] According to an embodiment of the present application, the data processing apparatus 500 further comprises a record analyzing module, a shard determining module and a permission updating module.

[0151] The record analyzing module is configured to analyze at least one historical processing record of each of the plurality of data shards according to a predetermined time period to obtain a first processing number of each of the plurality of data shards being processed with a shard lock at the first data site and a second processing number of each of the plurality of data shards being processed with a shard lock at the second data site.

[0152] The shard determining module is configured to compare the first processing number and the second processing number of each of the plurality of data shards to determine whether there is a to-be-transferred data shard in the plurality of data shards that needs to transfer the locking permission, wherein, in a case where the first processing number of the data shard is less than the second processing number of the data shard in a continuous predetermined number of periods, the data shard is determined as the to-be-transferred data shard.

[0153] The permission updating module is configured to, in a case where there is the to-be-transferred data shard, update a permission ownership relationship in response to a permission transfer instruction sent by the data processing system, so that the locking permission of the to-be-transferred data shard is transferred to a target data site, wherein the target data site is a data site in the data processing system that receives a data processing request number for operating the to-be-transferred data shard that meets a predetermined condition.

[0154] According to an embodiment of the present application, the permission updating module comprises a request temporary storage submodule and a permission updating submodule.

[0155] The request temporary storage submodule is configured to, in a case where there is an ongoing data processing request for the data shard to be transferred in the data processing system, place a new data processing request for the data shard to be transferred received by the data processing system in the to-be-processed queue.

[0156] The permission updating submodule is configured to, in a case where it is determined that the ongoing data processing request is executed or there is no data processing request in execution in the data processing system, transfer the locking permission of the data shard to be transferred to the target data site.

[0157] According to an embodiment of the present application, the data processing apparatus 500 further comprises a request response module.

[0158] The request response module is configured to, in a case where there is at least one new data processing request in the to-be-processed queue, respond to the at least one new data processing request in turn according to the order in which the at least one new data processing request is queued.

[0159] According to an embodiment of the present application, the data processing apparatus 500 further comprises an instruction generation module, an instruction sending module and a first execution completion module.

[0160] The instruction generation module is configured to, in a case where the locking of the target data shard by the first data site is completed, generate a remote processing instruction based on the processing logic indicated by the data processing request.

[0161] The instruction sending module is configured to send the remote processing instruction to data sites other than the first data site among the plurality of data sites, so that the other data sites process the target data shard stored in the other data sites according to the remote processing instruction, and send a remote processing completion message to the first data site in a case where the processing is completed.

[0162] The first execution completion module is configured to, in response to the first data site receiving the remote processing completion message sent by the other data site, release the shard lock, determine that the data processing request is executed, and generate an execution completion message.

[0163] According to an embodiment of the present application, the data processing apparatus 500 further comprises a shard processing module, a message sending module and a second execution completion module.

[0164] The shard processing module is configured to, in response to receiving a remote processing instruction generated by the second data site based on the processing logic indicated by the data processing request, process the target data shard stored in the first data site to obtain a data processing result.

[0165] The message sending module is configured to send a remote processing completion message to the second data site, so that the second data site releases the shard lock and generates an execution completion message after receiving the remote processing completion message.

[0166] The second execution completion module is configured to determine that the data processing request is completed in response to receiving the execution completion message sent by the second data site.

[0167] According to an embodiment of the present application, the data processing apparatus 500 further comprises a result returning module.

[0168] The result returning module is configured to return the execution completion message and the data processing result to the client in the case that it is determined that the data processing request is completed.

[0169] According to an embodiment of the present application, any multiple modules of the request parsing module 510, the shard locking module 520, the data processing module 530, the site determining module 540 and the request forwarding module 550 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of the modules can be combined with at least part of the functions of other modules, and implemented in one module. According to an embodiment of the present application, at least one of the request parsing module 510, the shard locking module 520, the data processing module 530, the site determining module 540 and the request forwarding module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware that can be integrated or packaged with a circuit, or implemented in any one of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the request parsing module 510, the shard locking module 520, the data processing module 530, the site determining module 540 and the request forwarding module 550 can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0170] Figure 6 A block diagram of an electronic device suitable for implementing the data processing method according to an embodiment of the present application is shown.

[0171] As Figure 6As shown, the electronic device 600 according to embodiments of the present application includes a processor 601 which can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. The processor 601 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chip set, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 601 can also include an on-board memory for cache use. The processor 601 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present application.

[0172] In the RAM 603, various programs and data required for the operation of the electronic device 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method processes according to embodiments of the present application by executing the programs in the ROM 602 and / or the RAM 603. Note that the programs can also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 can also perform various operations of the method processes according to embodiments of the present application by executing the programs stored in the one or more memories.

[0173] According to embodiments of the present application, the electronic device 600 can also include an input / output (I / O) interface 605 which is also connected to the bus 604. The electronic device 600 can also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as necessary. A removable medium 611 such as a magnetic disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 610 as necessary, so that a computer program read out therefrom is installed in the storage section 608 as necessary.

[0174] The application further provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist independently without being assembled into the device / apparatus / system. The computer readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the application.

[0175] According to the embodiments of the application, the computer readable storage medium can be a non-volatile computer readable storage medium, which can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In this application, a computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to the embodiments of the application, the computer readable storage medium can include one or more of the above-described ROM 602 and / or RAM 603 and / or one or more memories other than the ROM 602 and the RAM 603.

[0176] The embodiments of the application also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the methods provided by the embodiments of the application.

[0177] The above-described functions defined in the system / apparatus of the embodiments of the application are performed when the computer program is executed by the processor 601. According to the embodiments of the application, the above-described system, apparatus, module, unit, etc. can be implemented by computer program modules.

[0178] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium, and be downloaded and installed through the communication part 609 and / or installed from the detachable medium 611. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0179] In such embodiments, the computer program can be downloaded and installed from the network via the communication section 609, and / or installed from the removable media 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiments of the present application are performed. According to the embodiments of the present application, the system, device, apparatus, module, unit, and the like described above can be realized by the computer program module.

[0180] According to the embodiments of the present application, the program code for executing the computer program provided by the embodiments of the present application can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming language, and / or assembly / machine language. The programming language includes, but is not limited to, such as Java, C++, python, "C" language, or similar programming language. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0181] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the figures. For example, two blocks that are shown in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams or flowcharts, and combinations of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0182] Those skilled in the art can understand that the features described in various embodiments of the present application can be combined and / or integrated in various combinations, even if such combinations are not explicitly described in the present application. In particular, the features described in various embodiments of the present application can be combined and / or integrated in various combinations without departing from the spirit and teachings of the present application. All such combinations and / or integrations fall within the scope of the present application.

[0183] The embodiments of the application have been described. However, these embodiments are merely for illustration and are not intended to limit the scope of the application. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Various alternatives and modifications to the embodiments described herein will be apparent to those skilled in the art in view of the foregoing description. Such alternatives and modifications are intended to fall within the scope of the application.

Claims

1. A data processing method, characterized in that, The method includes: In response to a data processing request sent by a client received by a first data station in the data processing system, the data processing request is parsed to determine the target data fragment for the operation required to perform the data processing request. The data processing system includes multiple data stations, and each of the multiple data stations stores multiple data fragments obtained by block processing of the data volume of the data processing system. If it is determined that the first data station has the locking permission for the target data fragment, a fragment lock is acquired on the target data fragment so that no other data processing requests for the target data fragment are responded to before the data processing of the target data fragment is completed based on the data processing request; based on the data processing request, the target data fragment stored by the first data station is processed to obtain the data processing result; Once the first data station has completed locking the target data fragments, a remote processing instruction is generated based on the processing logic indicated by the data processing request. The remote processing instruction is sent to other data stations besides the first data station among multiple data stations, so that the other data stations can process the target data fragments stored on the other data stations according to the remote processing instruction, and send a remote processing completion message to the first data station upon completion of processing; and In response to the first data station receiving the remote processing completion message sent by the other data stations, the fragment lock is released, the data processing request is determined to be completed, and an execution completion message is generated; If it is determined that the first data station does not have the permission to lock the target data shard, a second data station with the permission to lock the target data shard is determined according to the permission ownership relationship between the multiple data shards and the multiple data stations; the data processing request is forwarded to the second data station, so that the second data station can lock the target data shard based on the data processing request, and process the target data shard stored by the second data station to obtain the data processing result.

2. The method according to claim 1, characterized in that, The method further includes: The data volume is divided into blocks according to the predetermined block size to obtain multiple data shards; Based on the number of data sites and the number of data shards, the locking permissions of each of the data shards are assigned to the data sites.

3. The method according to claim 2, characterized in that, The step of allocating locking permissions for each of the multiple data shards to the multiple data sites based on the number of data sites and the number of data shards includes: Based on the number of sites, the locking permissions of multiple data shards are sequentially assigned to multiple data sites.

4. The method according to claim 1, characterized in that, The method further includes: According to a preset time period, at least one historical processing record of each of the multiple data shards is analyzed to obtain the first number of times the multiple data shards are locked and processed at the first data station and the second number of times they are locked and processed at the second data station. The first processing count and the second processing count of each of the multiple data shards are compared to determine whether there is a data shard among the multiple data shards that needs to have its locking permissions transferred. Specifically, if a data shard's first processing count is less than its second processing count within a consecutive preset number of periods, the data shard exhibiting this condition is identified as the data shard to be transferred. In the presence of the data fragment to be transferred, in response to the permission transfer instruction sent by the data processing system, the permission ownership relationship is updated so that the locking permission of the data fragment to be transferred is transferred to the target data site, wherein the target data site is the data site in which the number of data processing requests received by the data processing system for operating on the data fragment to be transferred meets a predetermined condition.

5. The method according to claim 4, characterized in that, The step of transferring the locking permissions of the data fragment to be transferred to the target data site includes: If there is an ongoing data processing request for the data fragment to be transferred in the data processing system, the newly received data processing request for the data fragment to be transferred is placed in the processing queue; and If it is determined that the data processing request being executed has been completed, or that there is no data processing request being executed in the data processing system, the locking permission of the data fragment to be transferred is transferred to the target data site.

6. The method according to claim 5, characterized in that, The method further includes: If there is at least one new data processing request in the queue to be processed, at least one new data processing request shall be responded to in the order in which it was enqueued.

7. The method according to claim 1, characterized in that, The method further includes: In response to receiving a remote processing instruction generated by the processing logic of the second data station based on the data processing request indication, the target data fragments stored by the first data station are processed to obtain the data processing result; Send a remote processing completion message to the second data station so that the second data station releases the fragment lock and generates an execution completion message after receiving the remote processing completion message; Upon receiving the execution completion message sent by the second data station, it is determined that the data processing request has been completed.

8. The method according to claim 7, characterized in that, The method further includes: Upon determining that the data processing request has been completed, the execution completion message and the data processing result are returned to the client.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images