Request processing method, device, electronic device, storage medium and program product

Abstract

The present invention provides a request processing method, apparatus, electronic device, storage medium, and program product, which can be applied in the field of computer technology. The request processing method is applied to a first storage cluster in a storage system, the storage system including the first storage cluster and the second storage cluster for parallel data processing. The method comprises: in response to receiving a target data processing request, obtaining a first processing permission of the first storage cluster, and sending a permission acquisition request to the second storage cluster, wherein the permission acquisition request is for obtaining a second processing permission of the second storage cluster; in response to failure to obtain the second processing permission, releasing the first processing permission, and reprocessing the target data processing request after waiting for a first time period configured by the first storage cluster, wherein the second storage cluster is configured with a second time period that is different from the first time period.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, in particular to the field of storage software technology, and specifically to a request processing method, device, electronic device, storage medium and program product. Background Art

[0002] In the computer storage field, to ensure storage system reliability and data security, two storage devices can be used to form an active-active storage system. Both storage devices are in operation, providing redundancy, and the data on both devices is completely consistent. If one storage device fails, the other can automatically and seamlessly take over the business. Currently, in the actual application of active-active storage systems, data stored in the same location may be received simultaneously, causing data storage conflicts and affecting the storage system's processing efficiency. Summary of the Invention

[0003] In view of the above problems, the present invention provides a request processing method, device, electronic device, storage medium and program product for resolving data write conflicts and improving the processing efficiency of a storage system.

[0004] One aspect of the present invention provides a request processing method, which is applied to a first storage cluster in a storage system, wherein the storage system includes a first storage cluster and a second storage cluster for processing data in parallel; the method includes: in response to receiving a target data processing request, obtaining a first processing permission of the first storage cluster, and sending a permission acquisition request to the second storage cluster, wherein the permission acquisition request is used to obtain a second processing permission of the second storage cluster; in response to a failure to obtain the second processing permission, releasing the first processing permission, and reprocessing the target data processing request after waiting for a first time period configured for the first storage cluster, wherein the second storage cluster is configured with a second time period, which is different from the first time period.

[0005] Another aspect of the present invention provides a request processing device, which is applied to a first storage cluster in a storage system, wherein the storage system includes a first storage cluster and a second storage cluster for processing data in parallel; the device includes: a permission acquisition module, which is used to obtain a first processing permission of the first storage cluster in response to receiving a target data processing request, and send a permission acquisition request to the second storage cluster, wherein the permission acquisition request is used to obtain a second processing permission of the second storage cluster; a permission release module, which is used to release the first processing permission in response to a failure to obtain the second processing permission, and reprocess the target data processing request after waiting for a first time period configured by the first storage cluster, wherein the second storage cluster is configured with a second time period, which is different from the first time period.

[0006] Another aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above-mentioned request processing method.

[0007] Another aspect of the present invention further provides a computer-readable storage medium having a computer program or instruction stored thereon, which implements the steps of the above-mentioned request processing method when the computer program or instruction is executed by a processor.

[0008] Another aspect of the present invention further provides a computer program product, comprising a computer program or instructions, which implements the steps of the above request processing method when executed by a processor.

[0009] According to an embodiment of the present invention, in response to a target data processing request, the first processing authority of the first storage cluster is obtained, and a permission acquisition request is sent to the second storage cluster; in response to a failure to obtain the second processing authority, the first processing authority is released, and after waiting for a first time period, the target data processing request is reprocessed. Since, during the request processing process, if the remote second processing authority fails to be obtained, the local first processing authority is released, and the target data processing request is reprocessed after waiting for the first time period, the problem of storage conflicts is resolved; and by setting differentiated first and second time periods, the probability of storage conflicts occurring again when reprocessing the target data processing request can be reduced, thereby improving the processing efficiency of the storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0011] Figure 1A An architecture diagram of a storage active-active system with a master-slave architecture is shown in a related example;

[0012] Figure 1B An architecture diagram of a storage active-active system with asymmetric read and write in a related example is shown;

[0013] Figure 1C An architecture diagram of a storage active-active system with symmetrical read and write in a related example is shown;

[0014] Figure 1D A schematic diagram showing conflicts in the architecture of the related example;

[0015] Figure 2 An application scenario diagram of a request processing method according to an embodiment of the present invention is shown;

[0016] Figure 3A flowchart of a request processing method according to an embodiment of the present invention is shown;

[0017] Figure 4 An architecture diagram of a storage active-active system according to an embodiment of the present invention is shown;

[0018] Figure 5A A schematic diagram illustrating detecting overlapping areas of logical block addresses according to an embodiment of the present invention is shown;

[0019] Figure 5B A schematic diagram illustrating detecting overlapping areas of logical block addresses according to another embodiment of the present invention is shown;

[0020] Figure 6 A framework diagram showing a first mapping relationship according to an embodiment of the present invention is shown;

[0021] Figure 7 A schematic diagram showing a request processing method according to an embodiment of the present invention is shown;

[0022] Figure 8 It shows a structural block diagram of a request processing device according to an embodiment of the present invention;

[0023] Figure 9 A block diagram of an electronic device suitable for implementing a request processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] In the storage field, various disaster recovery and backup technologies have emerged to ensure the reliability of storage systems and the security of stored data. These technologies not only allow for data backup using snapshot and remote replication technologies, but also allow for the use of two storage devices to form an active-active storage system for data storage.

[0029] Active-active storage systems in related technologies generally adopt an Active-Passive architecture, an asymmetric Active-Active architecture, and a symmetric Active-Active architecture.

[0030] Figure 1A to Figure 1C An example architecture of a storage active-active system in a related example is shown.

[0031] Figure 1A The architecture diagram of the storage active-active system of the master-slave architecture in the relevant example is shown. Figure 1A The illustrated architecture may include a first storage device 103 and a second storage device 104. First storage device 103 is the master node, responsible for processing all requests, such as read and write requests. Second storage device 104 is a slave node, serving as a backup. It does not process service requests but synchronizes data with the master node. For example, first host 101 and second host 102 send requests to first storage device 103. After processing the requests, first storage device 103 copies the data to second storage device 104.

[0032] When the master node, ie, the first storage device 103, fails, the slave node, ie, the second storage device 104, will automatically or manually take over and become the new master node, ensuring the continuous availability of the storage system.

[0033] Figure 1B The architecture diagram of the storage active-active system with asymmetric read and write in the relevant example is shown. Figure 1B In the illustrated architecture, a first host 101 can send a data request to a first storage device 103, and a second host 102 can send a data request to a second storage device 104. The second storage device 104 can forward the received data request to the first storage device 103, which processes the data and then synchronously copies the data to the second storage device 104.

[0034] Figure 1C The architecture diagram of the storage active-active system with symmetrical read and write in the relevant example is shown. Figure 1C In the illustrated architecture, the first host 101 can send a data request to the first storage device 103, and the second host 102 can send a processing data request to the second storage device 104. After the first storage device 103 processes the data request sent by the first host 101, the obtained data can be synchronously mirrored to the second storage device 104. After the second storage device 104 processes the data request sent by the second host 102, the obtained data can be synchronously mirrored to the first storage device 103. Figure 1C The architecture shown does not distinguish between master and slave storage devices.

[0035] based on Figure 1A to Figure 1C The architecture of the active-active storage system shown, whether a master-slave active-active system or an asymmetric read-write active-active storage system, consists of two storage devices: a master and a slave. After receiving a write request from the host, the slave forwards it to the master. The master sorts the write requests it receives and those forwarded from the slaves before synchronizing the data with the slaves. This introduces additional link processing delays, impacting storage system performance.

[0036] In a storage active-active architecture with symmetrical read and write, the first storage device 103 can establish an active-active relationship with the second storage device 104. The storage of the first storage device 103 will be synchronized with the second storage device 104; the storage of the second storage device 104 will be synchronized with the first storage device 103. In this way, when the first storage device 103 fails, the second storage device 104 still has complete data for users to use; when the second storage device 104 fails, the first storage device 103 also has complete data for users to use. However, in a storage active-active architecture with symmetrical read and write, it is possible that data is stored from two storage devices at the same time. If this storage conflict problem is not handled, it may lead to inconsistent data on the two storage devices, such as Figure 1D shown.

[0037] Figure 1D A diagram showing conflicts in the architecture of the related example.

[0038] like Figure 1DAs shown, the host cluster where the first host and the second host are located can simultaneously issue data processing requests to the first storage device 103 and the second storage device 104. After the first storage device 103 receives data processing request 1, it needs to obtain write processing permission before it can start processing the data processing request; similarly, after the second storage device 104 receives data processing request 2, it also needs to obtain write processing permission before it can start processing the data processing request. Since both the first storage device 103 and the second storage device 104 need to obtain write permission before they can start processing the data processing request, and in the case where the data processing request 1 received by the first storage device 103 is synchronized to the second storage device 104, it is also necessary to obtain write permission on the second storage device 104 first. In the case where the data processing request 2 received by the second storage device 104 is synchronized to the first storage device 103, it is also necessary to obtain write permission on the first storage device 103 first. Therefore, when the first storage device 103 and the second storage device 104 receive write requests for the same location at the same time, it may happen that the data processing request 1 has already obtained the first processing permission (such as lock permission, lock permission) on the first storage device 103. 第一存储设备 ), try to obtain the second processing permission (such as lock permission, lock 第二存储设备 ); and data processing request 2 has obtained lock on the second storage device 104 第二存储设备 , try to obtain a lock on the first storage device 103 第一存储设备 In this case, a deadlock problem caused by request conflicts will occur, reducing the processing efficiency of the storage system.

[0039] In view of this, an embodiment of the present invention provides a request processing method for resolving the problem of storage conflict and improving the storage efficiency of a storage system.

[0040] Figure 2 An application scenario diagram of the request processing method according to an embodiment of the present invention is shown.

[0041] like Figure 2 As shown, the application scenario 200 according to this embodiment may include a first terminal device 201, a second terminal device 202, a third terminal device 203, a network 204, a server 205, and a storage system 206. The network 204 is used to provide communication links between the first terminal device 201, the second terminal device 202, the third terminal device 203 and the server 205, and between the server 205 and the storage system 206. The network 204 may include various connection types, such as wired or wireless communication links or optical fiber cables, etc.

[0042] A user may use a first terminal device 201, a second terminal device 202, or a third terminal device 203 to interact with a server 205 via a network 204 to receive or send messages, such as a request to process target data. Various communication client applications may be installed on the first terminal device 201, the second terminal device 202, or the third terminal device 203, such as storage applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0043] The first terminal device 201 , the second terminal device 202 , and the third terminal device 203 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0044] Server 205 may be a server that provides various services, such as a backend management server (for example only) that processes target data processing requests sent by users using first terminal device 201, second terminal device 202, and third terminal device 203 and stores the processed data in storage system 206. The backend management server may analyze and process the received requests and other data, and feed back the processing results (e.g., stored results, web pages, information, or data obtained or generated in accordance with the requests) to the terminal devices.

[0045] The storage system 206 may be a system for storing data obtained based on a data processing request. The storage system 206 may include multiple storage clusters that process data in parallel. The data obtained based on the target data processing request may be stored in the storage clusters.

[0046] It should be noted that the request processing method provided in the embodiment of the present invention can generally be executed by the server 205. Accordingly, the request processing apparatus provided in the embodiment of the present invention can generally be set in the server 205. The request processing method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 205 and can communicate with the first terminal device 201, the second terminal device 202, the third terminal device 203, the storage system 206 and / or the server 205. Accordingly, the request processing apparatus provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 205 and can communicate with the first terminal device 201, the second terminal device 202, the third terminal device 203, the storage system 206 and / or the server 205.

[0047] It should be understood that Figure 2The number of terminal devices, networks, servers, storage systems, and storage clusters in the storage system is merely illustrative. Any number of terminal devices, networks, servers, storage systems, and storage clusters may be provided as required.

[0048] The following will be based on Figure 2 The scene described by Figures 3 to 6 The request processing method according to the embodiment of the present invention is described in detail.

[0049] Figure 3 A flowchart of a request processing method according to an embodiment of the present invention is shown.

[0050] like Figure 3 As shown, the request processing method of this embodiment includes operations S310 to S320.

[0051] In operation S310 , in response to receiving a target data processing request, a first processing permission of a first storage cluster is obtained, and a permission acquisition request is sent to a second storage cluster, wherein the permission acquisition request is used to acquire a second processing permission of the second storage cluster.

[0052] In operation S320, in response to a failure to obtain the second processing authority, the first processing authority is released, and the target data processing request is reprocessed after waiting for a first time period configured for the first storage cluster, wherein the second storage cluster is configured with a second time period that is different from the first time period.

[0053] In some embodiments, the request processing method of the embodiment of the present invention may be applied to a first storage cluster in a storage system, where the storage system includes a first storage cluster and a second storage cluster for processing data in parallel.

[0054] Figure 4 The diagram shows an architecture diagram of a storage active-active system according to an embodiment of the present invention.

[0055] like Figure 4 As shown, the architecture may include a first storage cluster 403 and a second storage cluster 404 .

[0056] The first storage cluster 403 and the second storage cluster 404 can communicate with each other via a Fibre Channel (FC) or a Remote Direct Memory Access (RDMA) over Converged Ethernet (ROCE) link. RDMA is the abbreviation for Remote Direct Memory Access.

[0057] In some embodiments, a first data volume 401 can be created on a first storage cluster 403, and a second data volume 402 can be created on a second storage cluster 404. The first and second data volumes 401, 402 can form an active-active relationship and be mapped to the first and second hosts 101, 102 as virtual active-active volumes. The first data volume 401 can serve as the active-active master volume, and the second data volume 402 can automatically serve as a slave volume. The first and second data volumes 401, 402 can have different data processing policies for processing target data requests.

[0058] In some embodiments, the target data processing request may be a request for processing and storing target data. The target data processing request may include a request to write data related to updating or deleting data, or a request to read data related to querying data. The target data processing request may include specific data information to be updated, deleted, or queried.

[0059] In some embodiments, the first processing permission may include an operation lock of the first storage cluster on its own resources, such as a local lock permission, and the second processing permission may include an operation lock of the second storage cluster on its own resources, such as a remote lock permission.

[0060] In some embodiments, if the first processing permission is successfully acquired or the second processing permission is successfully acquired, it can be indicated that the resources of the first storage cluster or the second storage cluster have been occupied, and the target data processing request can be processed in this state. If the first processing permission fails to be acquired or the second processing permission fails to be acquired, it can be indicated that the resources of the first storage cluster or the second storage cluster have not been occupied, and the target data processing request can be not processed in this state, otherwise it may cause inconsistent data stored in the first storage cluster and the second storage cluster.

[0061] In some embodiments, when the acquisition of the second processing authority fails, it can be characterized as a failure to occupy the resources of the second storage cluster, and the resources of the second storage cluster are occupied by storage clusters other than the first storage cluster. In this case, the first processing authority can be released, that is, the target data processing request can be not processed to avoid the problem of inconsistency between the data stored on the first storage cluster and the second storage cluster.

[0062] In some embodiments, the first duration may be a retry wait duration of the first storage cluster. While the first storage cluster waits for the first duration, it may wake up the data processing request, reprocess the target data processing request, i.e., reacquire the first processing permission, and send a permission acquisition request to the second storage cluster to acquire the second processing permission of the second storage cluster.

[0063] In other embodiments, the second storage cluster may also be used to process the target data processing request. The second duration may be the duration that the second storage cluster waits to obtain the second processing permission. In response to receiving the target data processing request, the second storage cluster obtains the second processing permission of the second storage cluster and sends a permission acquisition request to the first storage cluster to obtain the first storage permission of the first storage cluster. If the acquisition of the first storage permission fails, the second processing permission may be released and the target data processing request may be reprocessed after waiting for the second duration.

[0064] In some embodiments, the first duration and the second duration are different. For example, the first duration can be much smaller than the second duration, for example, the first duration is 10ms and the second duration is 50ms. By setting different first and second durations, the probability of synchronization conflicts when reacquiring processing permissions can be reduced, thereby improving the orderliness of storage system processing.

[0065] According to an embodiment of the present invention, in response to a target data processing request, the first processing authority of the first storage cluster is obtained, and a permission acquisition request is sent to the second storage cluster; in response to a failure to obtain the second processing authority, the first processing authority is released, and after waiting for a first time period, the target data processing request is reprocessed. Since, during the request processing process, if the remote second processing authority fails to be obtained, the local first processing authority is released, and the target data processing request is reprocessed after waiting for the first time period, the problem of storage conflicts is resolved; and by setting differentiated first and second time periods, the probability of storage conflicts occurring again when reprocessing the target data processing request can be reduced, thereby improving storage efficiency and ensuring the reliability and security of the storage system.

[0066] In some embodiments, a first data volume in a first storage cluster and a second data volume in a second storage cluster can simultaneously receive and process data processing requests. This may result in data being written to the same logical block address (LBA) simultaneously from the first data volume in the first storage cluster and the second data volume in the second storage cluster. After receiving a write data processing request from an external host and before obtaining first processing authority, the first data volume can first check for local write conflicts on local resources. If a conflict exists, the data processing request enters a waiting state on the first storage cluster to avoid causing anomalies in the first storage cluster by forcibly obtaining first processing authority. After the conflict is resolved, the data processing request is awakened by another data processing request other than the data processing request. If no conflict exists, the first processing authority of the first storage cluster is obtained.

[0067] Based on this, the above-mentioned request processing method can also include the following operations: performing a first data block conflict detection on the target data processing request and the first data processing request to obtain a first detection result; in response to the first detection result indicating a data block conflict, suspending processing of the target data processing request; in response to the conflict being resolved, obtaining the first processing permission of the first storage cluster.

[0068] In some embodiments, the first data processing request may be a data processing request other than the target data processing request among the data processing requests for the first storage cluster. For example, if the data processing requests for the first storage cluster include data processing request a1, data processing request a2, and data processing request a3, and data processing request a3 is the target data processing request, then data processing request a1 and data processing request a2 may be the first data processing requests.

[0069] In some embodiments, data block conflict detection may be used to detect whether a conflict exists between a target data block to be written into a target data processing request and a first data block to be written into a first data processing request. For example, data block conflict detection may be used to detect whether a conflict exists between a target data block to be written into data processing request a3 and first data blocks to be written into data processing requests a1 and a2.

[0070] In some embodiments, if the target data block conflicts with the first data block, processing of the target data processing request is suspended. If the target data block does not conflict with the first data block, or the conflict is resolved, a first processing permission of the first storage cluster may be obtained. In some embodiments, the first processing permission may be permission to occupy resources of the target data block.

[0071] According to an embodiment of the present invention, before obtaining the first processing permission, a determination is made as to whether there is a local write conflict in the first storage cluster. If a conflict exists, the system enters a waiting state and proceeds after the conflict is resolved. This avoids data inconsistencies or system anomalies caused by forcibly obtaining the first processing permission, thereby improving operational security. Furthermore, after the conflict is resolved, the target data processing request is automatically reprocessed without manual intervention, ensuring the continuity of the data processing process.

[0072] In some embodiments, the target data processing request may include a target logical block address of data to be written into the target processing request, and the first data processing request may include a first logical block address of data to be written into the first data processing request.

[0073] In some embodiments, the process of performing the first data block conflict detection on the target data processing request and the first data processing request described above may include the following operations: performing overlapping area detection on the target logical block address and the first logical block address to obtain a first overlapping area detection result; in response to the first overlapping area detection result indicating that there is no overlapping area, determining that the first detection result is that the data blocks do not conflict; in response to the first overlapping area detection result indicating that there is an overlapping area, determining that the first detection result is a data block conflict.

[0074] In some embodiments, when the first end position of the first logical block address is located at or after the target start position of the target logical block address, or the first start position of the first logical block address is located at or before the target end position of the target logical block address, it can be considered that there is an overlapping area between the target logical block address and the first logical block address.

[0075] Figure 5A A schematic diagram illustrating detecting overlapping areas of logical block addresses according to an embodiment of the present invention is shown; Figure 5B A schematic diagram of detecting overlapping areas of logical block addresses according to another embodiment of the present invention is shown.

[0076] like Figure 5A As shown, the target logical block address may include a target start position (e.g., start LBA2) and a target end position (e.g., end LBA2). The first logical block address may include a first start position (e.g., start LBA1) and a first end position (e.g., end LBA1). Figure 5A The start LBA2 of the target logical block address is smaller than the end LBA1 of the first logical block address. Therefore, it is determined that there is a conflict between the target logical address and the first logical address.

[0077] like Figure 5B As shown, the target logical block address may include a target start position (e.g., start LBA2) and a target end position (e.g., end LBA2). The first logical block address may include a first start position (e.g., start LBA1) and a first end position (e.g., end LBA1). Figure 5B The end LBA2 of the target logical block address is greater than the start LBA1 of the first logical block address. Therefore, it is determined that there is a conflict between the target logical address and the first logical address.

[0078] In some embodiments, the first overlapping area detection result may represent an overlapping situation between the target logical block address and the first logical block address, for example, whether an overlapping area exists or does not exist between the target logical block address and the first logical block address.

[0079] In some embodiments, when there is an overlapping area between the target logical block address and the first logical block address, a first detection result indicating a data block conflict may be obtained. When there is no overlapping area between the target logical block address and the first logical block address, a first detection result indicating no data block conflict may be obtained.

[0080] According to an embodiment of the present invention, by comparing the target logical block address with the first logical block address, accurate judgment of data block conflicts can be achieved. A conflict is determined only when the target logical block address overlaps with the first logical block address, thereby reducing unnecessary pause and retry operations and improving the processing efficiency of target data processing requests.

[0081] In some embodiments, a first mapping relationship may be configured in the first storage cluster, and the first mapping relationship may represent a mapping relationship between a first request identifier for a first data processing request and a first logical block address.

[0082] In some embodiments, the first mapping relationship between the first request identifier and the first logical block address may be a key-value pair relationship. The first request identifier may be the key, and the first logical block address may be the value. One first mapping relationship may correspond to one first data processing request.

[0083] In some embodiments, before performing overlapping area detection on the target logical block address and the first logical block address, the first logical block address included in the first data processing request may be obtained according to the first mapping relationship.

[0084] In some embodiments, a first data processing request may correspond to a first mapping relationship, so before performing overlapping area detection on the target logical block address and the first logical block address, the first logical block address occupied by the first storage cluster when processing the first data processing request may be obtained based on the first mapping relationship.

[0085] According to an embodiment of the present invention, by obtaining the first logical block address according to the first mapping relationship before detecting the overlapping area of ​​the target logical block address and the first logical block address, not only the efficiency of obtaining the first logical block address can be improved, but also the efficiency of detecting the overlapping area of ​​the target logical block address and the first logical block address can be improved.

[0086] In some embodiments, the above-mentioned target data processing request may also include a logical unit number (LUN); the above-mentioned method may also include the following operations: determining a target request identifier for the target data processing request based on the logical unit number and the target logical block address; and updating the first mapping relationship based on the target request identifier and the target logical block address.

[0087] In some embodiments, the process of obtaining the request identifier may be as shown in formula (1).

[0088] Request ID = (Lun Id << 48) | (LBA / Data Management Unit) (1);

[0089] Wherein, Lun Id can be a logical unit number identifier, LBA can be a logical block address. The data management unit can be 32KB, 256KB, etc., which can be determined according to the storage system. Lun Id<<48 indicates that Lun Id is offset 48 bits to the left.

[0090] In some embodiments, the target request identifier and the target logical block address obtained by the above operation may be added to the first mapping relationship to update the first mapping relationship.

[0091] In some embodiments, by determining the target request identifier based on the logical unit number and the target logical block address, and updating the first mapping relationship based on the target request identifier and the target logical block address, real-time updating of the first mapping relationship can be achieved, which not only facilitates determining the association status between the data processing request and the data storage location, but also facilitates obtaining updated logical block address information when processing data requests that are later than the processing time of the target data processing request.

[0092] In some embodiments, the first mapping relationship described above may include multiple first mapping relationships, and the multiple first mapping relationships may be divided into at least one mapping relationship group. On this basis, the above method may further include the following operation: determining a target mapping relationship group in the at least one mapping relationship group based on the target logical block address included in the target data processing request, wherein the target mapping relationship group includes at least one target first mapping relationship.

[0093] In some embodiments, the first mapping relationship may be stored in a hash table structure.

[0094] Figure 6 A structural diagram of a first mapping relationship according to an embodiment of the present invention is shown.

[0095] In some embodiments, the first mapping relationship may include: Figure 6 The hash table framework shown.

[0096] exist Figure 6 In the example, the total capacity of the hash table is 256 buckets. Figure 6Only buckets 0 through 7 are shown in the figure. Buckets are the basic unit of hash table storage. Under each bucket, there can be at least one linked list hash (referred to as an entry list). Entry 1 through entry 4 represent different linked list hashes. The permissions for the logical block locations under each entry list are represented by locks, with lock 1 through lock 6 representing different permissions for logical block locations. Figure 6 The push in the hash table indicates that the entry is placed in the hash table, and the dotted line indicates where the connection is placed.

[0097] Continue to refer to Figure 6 Taking Bucket 5 as an example, in the entry list of Bucket 5, each entry corresponds to a different key, namely the request identifier. The request identifier can be determined by the formula shown in formula (1). Each data processing request will generate a corresponding key based on Lun Id and LBA. For example, for the target data processing request, a target request identifier key is generated, and then the target request identifier key is compared with the key of the entry in the Bucket list. If the target request identifier key is different from the key of the entry in the Bucket list, it means that no other data processing request except the target data processing request is accessing the logical block corresponding to the target request identifier key.

[0098] If the key of an entry in the bucket linked list matches the target request's identifier key, the target data processing request accesses the same logical block as other data processing requests. The value is then retrieved based on the key to determine if there is overlap in the locations of the same logical block. For example, in bucket 5, entry 4 with locks 2, 3, and 4 indicates that these three data processing requests access the same logical block, but at different locations. Locks 3, 5, and 6 indicate that these three data processing requests access the same location. These two linked lists, horizontal and vertical, allow for the management and tracking of all conflicting and non-conflicting data processing requests.

[0099] In some embodiments, the process described above of determining the target mapping relationship group in at least one mapping relationship group based on the target logical block address included in the target data processing request may include the following operations: processing the logical unit number and the target logical block address to obtain a processing result; and determining the target mapping relationship group in at least one mapping relationship group based on the processing result.

[0100] In some embodiments, determining the target mapping relationship group in the at least one mapping relationship group may be determining a target data bucket number (Bucket Id) in the at least one data bucket.

[0101] In some embodiments, the process of obtaining the target mapping relationship group may be as shown in formula (2).

[0102] Bucket Id = ((Lun Id<<48)|(LBA / data processing unit))% hash table length (2);

[0103] The hash table length is the number of data buckets in the hash table, << means shifting to the left, and % means taking the remainder.

[0104] Based on formula (2), the process of processing the logical unit number and the target logical block address to obtain the processing result can include processing the Lun Id, for example, shifting it left by 48 bits (i.e., Lun Id<<48), to obtain the processing result of the logical unit number shifted left by 48 bits. Then, the logical block address (e.g., LBA) is processed, for example, by determining the ratio of LBA to data processing unit, and the obtained ratio can be used as the processing result. Then, the logical unit number after shifting left by 48 bits and the ratio of LBA to data processing unit are concatenated, and divided by the number of data buckets. The remainder obtained is the number of the target data bucket, thereby obtaining the Bucket Id.

[0105] According to an embodiment of the present invention, by determining the target mapping relationship group based on the logical unit number and the logical block address, and mapping the logical block address to a unique target mapping relationship group, the uneven distribution problem caused by storing hash data in a single dimension (for example, only based on the logical unit number or only based on the logical block address) can be avoided, thereby improving the load balancing of the hash table.

[0106] In some embodiments, the target mapping relationship group may include at least one target first mapping relationship. The process of obtaining the first logical block address included in the first data processing request based on the first mapping relationship may include the following operations: obtaining the first logical block address included in the first data processing request based on the target first mapping relationship. The target first mapping relationship indicates a mapping relationship between the first request identifier of the first data processing request and the first logical block address. Based on the target first mapping relationship, the first logical block address associated with the first data processing request can be obtained.

[0107] In some embodiments, the target first mapping relationship in the target mapping group can be traversed based on the target mapping group obtained according to the logical unit number and the target logical block address, and the first logical block address indicated by the target first mapping relationship and the target logical block address can be used for detection, thereby eliminating the need to traverse all mapping relationship groups on the first storage cluster, reducing the number of invalid comparisons of the first logical block address and the target logical block address, and improving the efficiency of conflict detection.

[0108] In some embodiments, in the first storage cluster, after determining the location of the target mapping group (e.g., bucket identifier (Bucket Id)) according to the target data processing request, the target first mapping relationship in the target mapping group can be traversed, and the corresponding value, i.e., the target logical block address, can be obtained according to the key key composed of the logical unit number and the target logical block in the target data processing request. The starting position and the ending position of the detailed information in the target logical block address, such as start LBA2 and end LBA2, are used to determine whether there is a conflict with the first logical block address, such as start LBA1 and end LBA1. If start LBA2 ≤end LBA1 or end LBA2 ≥start LBA1), it is determined that there is a conflict between the target logical block address and the first logical block address.

[0109] In some embodiments, the above method may further include the following operations: when the first detection result indicates a data block conflict, writing the mapping relationship for the target data processing request into a hash table, and marking the state of the target data processing request as the first state; in response to the resolution of the data block conflict, waking up the target data processing request, and updating the state of the target data processing request to the second state.

[0110] In some embodiments, if there is no conflict between the target logical block address and the first logical block address, the target data processing request is granted the first processing permission in the first storage cluster, and the mapping information between the target request identifier generated by the target processing request and the target logical block address is written into the hash table of the first storage cluster for management.

[0111] In some embodiments, if a target logical block address conflicts with a first logical block address, the target data processing request is marked as being in a first state in the first storage cluster (e.g., a state of waiting for the first processing permission), and a mapping relationship between a target request identifier generated by the target processing request and the target logical block address is written into a hash table in the first storage cluster. In response to the conflict between the target logical block address and the first logical block address being resolved, that is, after the first data processing request that conflicts with the target data processing request releases the first processing permission, the target data processing request is reactivated, and the first state is updated to a second state (e.g., a state of acquiring the first processing permission) to grant the target data processing request the first processing permission in the first storage cluster.

[0112] According to embodiments of the present invention, by marking a target data processing request as either the first or second state, conflicts between the target logical block address and the first logical block address can be clearly distinguished, facilitating system scheduling based on priority or timing, and avoiding omissions or confusion in data processing requests. Furthermore, the second state can serve as a trigger condition for automatically reprocessing the target data processing request after the conflict is resolved, reducing manual intervention and improving the efficiency and intelligence of processing target data processing requests.

[0113] In some embodiments, if the target data processing request has obtained the first processing right on the first storage cluster, the information and data in the target data processing request may be sent to a remote second storage cluster to obtain the second processing right of the second storage cluster.

[0114] In some embodiments, the permission acquisition request may include: the target logical block address in the target data processing request; the process of obtaining the second processing permission of the second storage cluster may include the following operations: in response to the second detection result indicating that the data blocks do not conflict, granting the first storage cluster the second processing permission, wherein the second detection result is obtained by the second storage cluster performing a second data block conflict detection on the permission acquisition request and the second data processing request based on the target logical block address, and the second data processing request is a data processing request for the second storage cluster.

[0115] In some embodiments, after receiving the target data processing request sent by the first storage cluster, the second storage cluster may refer to the aforementioned conflict detection process between the target logical block address and the first logical block address and perform a second data block conflict detection on the permission acquisition request and the second data processing request based on the address of the target logical block. The second data processing request may be any other data processing request processed by the second storage cluster other than the target data processing request.

[0116] According to an embodiment of the present invention, by granting a second processing permission to the target data processing request when the target logical block address and the second logical block address do not conflict, it is possible to prevent the operation of writing data in the first storage cluster from conflicting with the existing operations of the second storage cluster, thereby ensuring data consistency and system stability in the active-active dual-write scenario.

[0117] In some embodiments, the process of performing a second data block conflict detection on the permission acquisition request and the second data processing request described above may include the following operations: determining the second logical block address for the second data processing request based on the second mapping relationship; performing an overlapping area detection on the target logical block address and the second logical block address to obtain a second overlapping area detection result; in response to the second overlapping area detection result indicating that there is no overlapping area, determining that the second detection result is that the data blocks do not conflict; in response to the second overlapping area detection result indicating that there is an overlapping area, determining that the second detection result is a data block conflict.

[0118] In some embodiments, the process of performing a second data block conflict detection on a permission acquisition request and a second data processing request can refer to the process of performing a first data block conflict detection on a target data processing request and a first data processing request described above, that is, the process of performing a conflict detection on a target logical block address and a first logical block address.

[0119] In some embodiments, the second storage cluster may be configured with a second mapping relationship, and the second mapping relationship may represent a mapping relationship between a second request identifier and a second logical block address of the second data processing request. The process of obtaining the second request identifier and the process of obtaining the first request identifier described above may refer to the process of obtaining the target request identifier described above, and may refer to formula (1), that is, the respective first request identifier or second request identifier may be obtained based on the logical unit number and logical block address associated with the first data processing request or the second data processing request, respectively.

[0120] According to the second mapping relationship, the second logical block address for the second data processing request can be determined. The process of detecting the overlapping area between the target logical block address and the second logical block address can refer to the above Figure 5A and Figure 5B The second detection result may include a second data block conflict or a second data block non-conflict.

[0121] For example, the second logical block address may include a second start position (e.g., start LBA3) and a second end position (e.g., end LBA3). When start LBA3 ≤ end LBA1 or end LBA3 ≥ start LBA1, it can be determined that there is an overlapping area between the target logical address and the second logical address, and then the second detection result can be determined as a data block conflict. When start LBA3 > end LBA1 or end LBA3 < start LBA1, it can be determined that there is no overlapping area between the target logical address and the second logical address, and then the second detection result can be determined as no data block conflict.

[0122] In some embodiments, after the second storage cluster receives a target data processing request sent by the peer first storage cluster or a permission acquisition request associated with the target data processing request, it can adopt the above conflict detection method. By traversing the hash table on the second storage cluster (the hash table structure on the second storage cluster can be the same as the hash table structure on the first storage cluster, refer to Figure 6 ), detect whether there is a conflict between the target data processing request sent from the first storage cluster or the permission acquisition request associated with the target data processing request and the second data processing request sent from the host to the second storage cluster. If there is no conflict, grant the second processing permission to the target data processing request sent from the first storage cluster or the permission acquisition request associated with the target data processing request.

[0123] According to the embodiments of the present invention, by performing conflict detection on the second logical block address and the target logical block address, an accurate determination of data block conflicts can be achieved. Only when there is an overlapping area between the second logical block address and the target logical block address, the second detection result is determined as a data block conflict, which can reduce unnecessary pauses and retries of the second storage cluster. On the other hand, by using the second hash table to map the second data processing requests for the second storage cluster to the corresponding mapping relationship groups according to the second request identifier, only the second target logical block and the target logical block within the mapping relationship group are subjected to conflict detection, without traversing the logical block addresses of all mapping relationship groups of the second storage cluster, reducing invalid comparison operations, reducing the consumption of computing resources and storage resources of the system, and improving the system operation efficiency.

[0124] In some embodiments, the above permission acquisition request may include the target processing data in the target data processing request. The above method may further include the following operations: in response to successful acquisition of the second processing permission, write the target processing data to the second data volume for the second storage cluster; in response to the completion of writing the target processing data to the second data volume, write the target processing data to the first data volume for the first storage cluster.

[0125] In some embodiments, if the second processing permission is successfully obtained, the target processing data can be directly written to the second data volume of the second storage cluster. When the writing is complete, the second processing permission of the second storage cluster can be released, and the write result in the second storage cluster (for example, the result of writing the target processing data to the second data volume of the second storage cluster) is returned to the first storage cluster. The first storage cluster can then write the target processing data in the target data processing request to the first data volume of the first storage cluster, thereby completing the double write operation on the target data processing request.

[0126] According to an embodiment of the present invention, after obtaining the second processing permission, data is stored first on the second data volume, and then on the first data volume. Remote writes are performed only after successfully obtaining the second processing permission. This ensures that write operations have legitimate permissions, prevents unauthorized access, and improves system stability. Furthermore, writing data to the second data volume of the second storage cluster before writing to the first data volume reduces the number of data exchanges and improves processing efficiency.

[0127] In some embodiments, the above method may further include the following operations: determining a write order of target processing data based on resource status information of the first data volume and resource status information of the second data volume; and writing the target processing data to the first data volume and the second data volume according to the write order.

[0128] In some embodiments, the resource status information of the first data volume and the resource status information of the second data volume may include information such as storage capacity, central processing unit (CPU), and network traffic, etc. The writing order of the target data may be determined based on this status information.

[0129] For example, when the storage capacity of the first data volume is greater than the storage capacity of the second data volume, the target data can be written to the first data volume first and then to the second data volume. For another example, the storage capacity, CPU, and network traffic can be configured with respective weight information, and the writing order of the target processing data can be determined based on the product sum of the storage capacity, CPU, and network traffic and their respective weights. For example, a first weight can be assigned to the storage capacity to reflect the degree of influence of the storage capacity on the first data volume or the second data volume. A second weight can be assigned to the CPU utilization to indicate the degree of influence of the CPU computing resources on the first data volume or the second data volume. A third weight can be assigned to the network traffic to reflect the degree of influence of the network bandwidth or delay on the first data volume or the second data volume. The collected storage capacity, CPU utilization, and network traffic of the first data volume and the second data volume can be standardized to obtain standardized storage capacity, CPU utilization, and network traffic. The standardized storage capacity, CPU utilization and network traffic of the first data volume and the second data volume are multiplied and added with the first weight, the second weight and the third weight to obtain the load assessment values ​​corresponding to the first data volume and the second data volume respectively. If the load assessment value of the first data volume is higher than the load assessment value of the second data volume, the data is written to the second data volume first; if the load assessment value of the first data volume is lower than the load assessment value of the second data volume, the data is written to the first data volume first.

[0130] Different normalization methods can be used for storage capacity, CPU utilization, and network traffic. For example, storage capacity normalization can be achieved by using a logarithmic transformation to uniformly convert the data to the same units. Another example is CPU utilization normalization, which can be achieved by using Z-score normalization to convert the CPU utilization to the range [0, 100]. Another example is network traffic normalization, which can be achieved by using a time window normalization, for example, slicing data into 5-minute intervals. According to embodiments of the present invention, by intelligently adjusting the write order based on the resource status information of the first or second data volume, unilateral overload can be avoided and automatic load balancing between storage clusters can be achieved.

[0131] In some embodiments, when a response message indicating that the second storage cluster is busy is received, it may be determined that the acquisition of the second processing permission has failed.

[0132] In some embodiments, the above description is of an embodiment in which the second processing permission is successfully obtained. However, the second processing permission may also fail to be obtained. For example, if the second storage cluster detects that a target data processing request or a permission acquisition request associated with the target data processing request, sent from the first storage cluster, conflicts with the second data processing request, a permission busy response message may be directly returned to the first storage cluster to avoid deadlock.

[0133] According to an embodiment of the present invention, by determining that the second permission acquisition fails based on the received permission busy information, the first storage cluster can be prevented from continuously waiting due to timeout or unknown error, thereby improving system response efficiency.

[0134] In some embodiments, upon receiving a busy response message from a second storage cluster, the first storage cluster may release the first processing privilege already obtained by the target data processing request on the first storage cluster, then suspend the target data processing request, wait for a first time period, and then reactivate the target data processing request. After reacquiring the first processing privilege from the first storage cluster, the target data and the privilege acquisition request for the target data processing request may be resent to the second storage cluster. To avoid conflicts, the waiting times for data processing requests issued by the host cluster received by the first data volume and the second data volume may differ significantly. For example, in an embodiment of the present invention, upon receiving a busy response message from the remote second storage cluster, the first waiting time for a target data processing request issued by the host received by the first data volume may be set to a shorter time period, such as 10ms. Upon receiving a busy response message from the remote first storage cluster, the second waiting time for a data processing request issued by the host received by the second data volume may be set to a slightly longer time period, such as 50ms, to avoid conflicts when reacquiring remote processing privileges and improve storage efficiency.

[0135] In some embodiments, the above method may further include the following operations: in response to the response information including the estimated duration, adjusting the first duration according to the resource status information and the estimated duration of the first storage cluster; the first duration of waiting for the configuration of the first storage cluster includes: waiting for the adjusted first duration.

[0136] In some embodiments, the estimated duration can be the duration required for the second storage cluster to release the second processing permission. Based on information such as the storage capacity, CPU, memory, and network traffic of the first storage cluster, as well as the estimated duration, the first duration required for the first storage cluster to wait for the target data processing request can be adjusted. For example, if the current CPU of the first storage cluster is higher than a predetermined CPU threshold, indicating that the first storage cluster is currently in a high-load state, the predetermined duration (the predetermined duration is determined based on CPU utilization) can be further extended while the first duration extends the estimated duration. For example, if the current CPU of the first storage cluster is lower than a predetermined CPU threshold, indicating that the first storage cluster is currently in a low-load state, the estimated duration can be extended from the first duration. This can avoid frequent awakening of the target data processing request within the estimated duration, and avoid frequent acquisition of the second processing permission, which results in the second processing permission being in a state of constant acquisition failure. This can improve the efficiency and accuracy of acquiring the second processing permission, thereby improving storage efficiency.

[0137] According to an embodiment of the present invention, by dynamically adjusting the first duration according to the estimated duration, resource waste or response delay caused by waiting for a fixed duration can be avoided.

[0138] In some embodiments, the above method may further include the following operations: recording the number of retries for reprocessing the target data processing request; in response to the number of retries being greater than or equal to a preset threshold, abandoning processing the target data processing request and sending a request failure message to the client.

[0139] In some embodiments, if the number of retries for reprocessing the target data processing request has reached three consecutive retries, and the result of reprocessing the target data processing request is a corresponding permission busy message returned by the remote second storage cluster, then the target data processing request can be abandoned in writing to the first storage cluster, and a response message indicating that the target data processing request failed to be processed can be returned to the host cluster, so that the host cluster can retry the target data processing request. Because the probability of a host cluster outside the first storage cluster and the second storage cluster writing to the same location in the first storage cluster and the second storage cluster at the same time is extremely small, this processing method has a relatively minor impact on the host business.

[0140] According to an embodiment of the present invention, a preset threshold for the number of retries is set, and processing of the target data processing request is abandoned after the number of retries exceeds the preset threshold. By limiting the number of times a target data processing request is reprocessed, it is possible to prevent the request from being infinitely retried due to anomalies such as persistent lock conflicts and network failures, thereby avoiding the ineffective use of resources such as CPU, memory, and network bandwidth, and ensuring the normal operation of the core business of the storage system.

[0141] In some embodiments, there may be multiple thresholds described above, and the above method may also include the following operations: determining a target threshold among multiple preset thresholds based on the cluster level and resource status information of the first storage cluster; the number of retries being greater than or equal to the preset threshold includes: the number of retries being greater than or equal to the target threshold.

[0142] In some embodiments, the cluster level of the first storage cluster may include storage tiers. For example, storage tiers may be pre-configured for different storage clusters. Storage tiers may also be determined based on information such as the size and importance of the data stored in the first storage cluster. Different storage tiers may be configured with different preset thresholds for retry operations to facilitate diversified management of the storage cluster. A preset threshold table may be configured for each storage tier, with each evaluation range in the table corresponding to a preset threshold.

[0143] In some embodiments, based on the resource status information of the first storage cluster, for example, based on the sum of the product of storage capacity, CPU, and network traffic and their respective weights, an evaluation value of the first storage cluster is determined, the evaluation value is matched with the evaluation range in the preset threshold table, and a target evaluation range that matches the evaluation value is determined from multiple evaluation ranges. The preset threshold corresponding to the target evaluation range is used as the target threshold, and the target threshold can be compared with the number of retries.

[0144] According to an embodiment of the present invention, the target threshold is determined based on the storage cluster level and resource status. Different target thresholds can be determined for different storage clusters and different resource statuses, so that the storage cluster can have different retry times, and diversified management of the storage cluster can be achieved.

[0145] In some embodiments, corresponding to the preset threshold of the first storage cluster, the second storage cluster may also have multiple preset thresholds. Based on the cluster level and resource status information of the first storage cluster, a target threshold among the multiple preset thresholds can be determined. Based on the target threshold, it is determined whether the second storage cluster needs to continue retrying or give up processing when processing the target data processing request.

[0146] Figure 7 A schematic diagram of a request processing method according to an embodiment of the present invention is shown.

[0147] like Figure 7 As shown, the request processing method of this embodiment is applied to a processing architecture including a first storage device 103 and a second storage device 104. The first storage device 103 can process a first data processing request 701, and the second storage device 104 can process a second data processing request 702. The first data processing request 701 can be issued by a first host, and the second data processing request 702 can be issued by a second host.

[0148] The first storage device 103 receives a first data processing request 701 from the host. Upon obtaining the first processing permission, the first storage device 103 sends a permission acquisition request to the second storage device 104 to obtain the second processing permission of the second storage device 104. Simultaneously, the second storage device 104 receives a second data processing request 702 and then, upon obtaining the second processing permission, sends a permission acquisition request to the first storage device 103 to obtain the first processing permission of the first storage device 103. Due to the simultaneous operations, when the first storage device 103 obtains the second processing permission, since the second processing permission is occupied, it receives a response message from the second storage device 104 stating that the second processing permission is busy. When the second storage device 104 obtains the first processing permission, since the first processing permission is occupied, it receives a response message from the first storage device 103 stating that the first processing permission is busy. Upon receiving the second processing permission busy response, the first storage device 103 may suspend the first data processing request for a first duration, and then reactivate the first data processing request after the first duration to obtain the second processing permission again. When receiving the notification that the first processing authority is busy, the second storage device 104 may suspend the second data processing request for a second period of time, and wake up the second data processing request again after the second period of time, and obtain the first processing authority again.

[0149] According to an embodiment of the present invention, by releasing the second processing authority or the first processing authority when the first processing authority is busy or the second processing authority is busy, and reacquiring the first processing authority or the second processing authority after waiting for the second period of time or the first period of time, continuous storage conflicts can be avoided and the reliability and security of storage can be improved.

[0150] It should be noted that, unless it is clearly stated that there is a sequence of execution between different operations shown in the flowchart in the embodiments of the present invention, or there is a sequence of execution between different operations in technical implementation, otherwise, the execution order of multiple operations may not be prioritized, and multiple operations may also be executed simultaneously.

[0151] Based on the above request processing method, the present invention also provides a request processing device. Figure 8 The device is described in detail.

[0152] Figure 8 A structural block diagram of a request processing device according to an embodiment of the present invention is shown.

[0153] like Figure 8As shown, the request processing device 800 of this embodiment is applied to the first storage cluster in the storage system, which includes the first storage cluster and the second storage cluster for parallel processing of data. The request processing device 800 may include a permission acquisition module 810 and a permission release module 820.

[0154] The permission acquisition module 810 is configured to obtain the first processing permission of the first storage cluster in response to receiving a target data processing request, and send a permission acquisition request to the second storage cluster, wherein the permission acquisition request is used to obtain the second processing permission of the second storage cluster.

[0155] The permission release module 820 is used to release the first processing permission in response to the failure to obtain the second processing permission, and reprocess the target data processing request after waiting for the first time period configured by the first storage cluster, wherein the second storage cluster is configured with a second time period, and the second time period is different from the first time period.

[0156] In some embodiments, the request processing device may further include a detection module, a pause module, and a response module.

[0157] The detection module is configured to perform a first data block conflict detection on the target data processing request and the first data processing request to obtain a first detection result, wherein the first data processing request is a data processing request other than the target data processing request among the data processing requests for the first storage cluster.

[0158] The pausing module is configured to suspend processing of the target data processing request in response to the first detection result indicating a data block conflict.

[0159] The response module is configured to obtain the first processing permission of the first storage cluster in response to the conflict being resolved.

[0160] In some embodiments, the detection module may include a first detection submodule, a first determination submodule, and a second determination submodule.

[0161] The first detection submodule is configured to perform overlapping area detection on the target logical block address and the first logical block address to obtain a first overlapping area detection result.

[0162] The first determining submodule is configured to determine, in response to the first overlapping area detection result indicating that no overlapping area exists, that the first detection result is that the data blocks do not conflict.

[0163] The second determining submodule is configured to determine that the first detection result is a data block conflict in response to the first overlapping area detection result indicating that an overlapping area exists.

[0164] In some embodiments, the request processing device may further include an address acquisition module.

[0165] The address acquisition module is configured to acquire the first logical block address included in the first data processing request according to a first mapping relationship before performing overlapping area detection on the target logical block address and the first logical block address.

[0166] In some embodiments, the request processing device may further include a determination module and an update module.

[0167] The first determining module is configured to determine a target request identifier for a target data processing request according to a logical unit number and a target logical block address.

[0168] The updating module is configured to update the first mapping relationship according to the target request identifier and the target logical block address.

[0169] In some embodiments, the request processing device may further include a second determination module.

[0170] The second determining module is configured to determine a target mapping relationship group in at least one mapping relationship group according to a target logical block address included in the target data processing request, wherein the target mapping relationship group includes at least one target first mapping relationship.

[0171] In some embodiments, the address acquisition module may include an address acquisition sub-module.

[0172] The address acquisition submodule is configured to acquire a first logical block address included in the first data processing request according to a target first mapping relationship.

[0173] In some embodiments, the second determination module may include a processing submodule and a third determination submodule.

[0174] The processing submodule is used to process the logical unit number and the target logical block address to obtain a processing result.

[0175] The third determining submodule is configured to determine a target mapping relationship group in the at least one mapping relationship group according to the processing result.

[0176] In some embodiments, the request processing device may further include a first writing module and a wake-up module.

[0177] The first writing module is configured to write a mapping relationship for the target data processing request into the hash table and mark the state of the target data processing request as a first state when the first detection result indicates a data block conflict.

[0178] The wake-up module is configured to wake up the target data processing request in response to the resolution of the data block conflict, and update the state of the target data processing request to a second state.

[0179] In some embodiments, the rights acquisition module may include a granting submodule.

[0180] The granting submodule is used to grant the first storage cluster a second processing permission in response to a second detection result indicating that the data blocks do not conflict, wherein the second detection result is obtained by the second storage cluster performing a second data block conflict detection on the permission acquisition request and the second data processing request according to the target logical block address, and the second data processing request is a data processing request for the second storage cluster.

[0181] In some embodiments, the assigning submodule may include a first determining unit, a detecting unit, a second determining unit, and a third determining unit.

[0182] The first determining unit is configured to determine a second logical block address for a second data processing request according to a second mapping relationship.

[0183] The detection unit is used to perform overlapping area detection on the target logical block address and the second logical block address to obtain a second overlapping area detection result.

[0184] The second determining unit is configured to determine, in response to the second overlapping area detection result indicating that the overlapping area does not exist, that the second detection result is that the data blocks do not conflict.

[0185] The third determining unit is configured to determine that the second detection result is a data block conflict in response to the second overlapping area detection result indicating that an overlapping area exists.

[0186] In some embodiments, the request processing device may further include a recording module and a sending module.

[0187] The recording module is used to record the number of retries for reprocessing the target data processing request.

[0188] The sending module is used to give up processing the target data processing request in response to the number of retries being greater than or equal to a preset threshold, and send a request failure message to the client.

[0189] In some embodiments, the request processing device may further include a third determination module.

[0190] The third determination module is configured to determine a target threshold among multiple preset thresholds according to the cluster level and resource status information of the first storage cluster; the number of retries being greater than or equal to the preset threshold includes: the number of retries being greater than or equal to the target threshold.

[0191] In some embodiments, the request processing device may further include a fourth determination module.

[0192] The fourth determining module is configured to determine that the acquisition of the second processing permission fails when receiving a response message indicating that the permission is busy sent by the second storage cluster.

[0193] In some embodiments, the request processing device may further include an adjustment module.

[0194] The adjustment module is used to adjust the first duration in response to the estimated duration included in the response information according to the resource status information and the estimated duration of the first storage cluster; the first duration of waiting for the configuration of the first storage cluster includes: the first duration after waiting for adjustment.

[0195] In some embodiments, the request processing device may further include a second writing module and a third writing module.

[0196] The second writing module is configured to write target processed data to a second data volume corresponding to the second storage cluster in response to the second processing permission being successfully acquired.

[0197] The third writing module is configured to write the target processed data into the first data volume corresponding to the first storage cluster in response to completion of writing the target processed data into the second data volume.

[0198] In some embodiments, the request processing device may further include a fifth determination module and a fourth writing module.

[0199] The fifth determining module is configured to determine a writing order of the target processed data according to the resource status information of the first data volume and the resource status information of the second data volume.

[0200] The fourth writing module is configured to write the target processed data into the first data volume and the second data volume according to a writing order.

[0201] According to embodiments of the present invention, any multiple modules in the permission acquisition module 810 and the permission release module 820 can be combined into a single module, or any one of them can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module. According to embodiments of the present invention, at least one of the permission acquisition module 810 and the permission release module 820 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the permission acquisition module 810 and the permission release module 820 can be at least partially implemented as a computer program module that, when executed, can perform the corresponding functionality.

[0202] Figure 9A block diagram of an electronic device suitable for implementing a request processing method according to an embodiment of the present invention is shown.

[0203] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 902 or programs loaded from a storage unit 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0204] The RAM 903 stores various programs and data required for the operation of the electronic device 900. The processor 901, ROM 902, and RAM 903 are connected to each other via a bus 904. The processor 901 executes the programs in the ROM 902 and / or RAM 903 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 902 and RAM 903. The processor 901 may also execute the programs stored in the one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0205] According to an embodiment of the present invention, electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to bus 904. Electronic device 900 may also include one or more of the following components connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 908 including a hard disk; and a communication section 909 including a network interface card such as a LAN card or modem. Communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. Removable media 911, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 910 as needed, so that computer programs read from the removable media can be installed into storage section 908 as needed.

[0206] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0207] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above, and / or one or more memories other than ROM 902 and RAM 903.

[0208] The embodiments of the present invention further include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to enable the computer system to implement the request processing method provided by the embodiments of the present invention.

[0209] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when executed by the processor 901. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0210] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0211] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909 and / or installed from a removable medium 911. When the computer program is executed by the processor 901, the above-described functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0212] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0213] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0214] It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or couplings fall within the scope of the present invention.

[0215] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A request processing method, applied to a first storage cluster in a storage system, wherein the storage system includes a first storage cluster and a second storage cluster for parallel processing of data; characterized in that: The method comprises: In response to receiving the target data processing request, obtaining a first processing permission of the first storage cluster, and sending a permission acquisition request to the second storage cluster, wherein the permission acquisition request is used to obtain a second processing permission of the second storage cluster; In response to a failure to obtain the second processing authority, the first processing authority is released, and the target data processing request is reprocessed after waiting for a first time period configured for the first storage cluster, wherein the second storage cluster is configured with a second time period, and the second time period is different from the first time period.

2. The method according to claim 1, characterized in that The method further comprises: performing a first data block conflict detection on the target data processing request and a first data processing request to obtain a first detection result, wherein the first data processing request is a data processing request other than the target data processing request among the data processing requests for the first storage cluster; In response to the first detection result indicating a data block conflict, suspending processing of the target data processing request; In response to the conflict being resolved, a first processing permission of the first storage cluster is acquired.

3. The method according to claim 2, characterized in that The target data processing request includes a target logical block address, and the first data processing request includes a first logical block address; The performing first data block conflict detection on the target data processing request and the first data processing request includes: Performing an overlapping area detection on the target logical block address and the first logical block address to obtain a first overlapping area detection result; In response to the first overlapping area detection result indicating that no overlapping area exists, determining the first detection result as data block non-conflict; In response to the first overlapping area detection result indicating the existence of an overlapping area, the first detection result is determined to be a data block conflict.

4. The method according to claim 3, characterized in that The first storage cluster is configured with a first mapping relationship, where the first mapping relationship represents a mapping relationship between a first request identifier for a first data processing request and a first logical block address; The method further comprises: Before performing overlapping area detection on the target logical block address and the first logical block address, the first logical block address included in the first data processing request is obtained according to the first mapping relationship.

5. The method according to claim 4, characterized in that The target data processing request also includes a logical unit number; The method further comprises: determining a target request identifier for the target data processing request according to the logical unit number and the target logical block address; The first mapping relationship is updated according to the target request identifier and the target logical block address.

6. The method according to claim 5, characterized in that The first mapping relationships include multiple first mapping relationships, and the multiple first mapping relationships are divided into at least one mapping relationship group; The method further comprises: determining a target mapping relationship group in at least one mapping relationship group according to a target logical block address included in the target data processing request, wherein the target mapping relationship group includes at least one target first mapping relationship; The acquiring, according to the first mapping relationship, the first logical block address included in the first data processing request includes: According to the target first mapping relationship, a first logical block address included in the first data processing request is obtained.

7. The method according to claim 6, characterized in that The first mapping relationship includes a hash table; Determining a target mapping relationship group in at least one mapping relationship group includes: Processing the logical unit number and the target logical block address to obtain a processing result; According to the processing result, a target mapping relationship group in the at least one mapping relationship group is determined.

8. The method according to claim 7, characterized in that The method further comprises: If the first detection result indicates a data block conflict, writing a mapping relationship for the target data processing request into the hash table and marking the state of the target data processing request as a first state; In response to the data block conflict being resolved, the target data processing request is awakened, and the state of the target data processing request is updated to a second state.

9. The method according to claim 1, characterized in that The permission acquisition request includes: the target logical block address in the target data processing request; Acquiring the second processing permission of the second storage cluster includes: In response to the second detection result indicating that the data blocks do not conflict, the first storage cluster is granted the second processing authority, wherein the second detection result is obtained by the second storage cluster performing a second data block conflict detection on the authority acquisition request and the second data processing request according to the target logical block address, and the second data processing request is a data processing request for the second storage cluster.

10. The method according to claim 9, characterized in that The second storage cluster is configured with a second mapping relationship, where the second mapping relationship represents a mapping relationship between a second request identifier of a second data processing request and a second logical block address; The performing second data block conflict detection on the permission acquisition request and the second data processing request includes: determining, according to the second mapping relationship, a second logical block address for the second data processing request; Performing overlapping area detection on the target logical block address and the second logical block address to obtain a second overlapping area detection result; In response to the second overlapping area detection result indicating that no overlapping area exists, determining the second detection result as data block non-conflict; In response to the second overlapping area detection result indicating the existence of an overlapping area, the second detection result is determined to be a data block conflict.

11. The method according to claim 1, characterized in that The method further comprises: Recording the number of retries for reprocessing the target data processing request; In response to the number of retries being greater than or equal to a preset threshold, processing the target data processing request is abandoned, and a request failure message is sent to the client.

12. The method according to claim 11, characterized in that The preset threshold value includes multiple ones; The method further comprises: determining a target threshold value among the plurality of preset threshold values ​​according to the cluster level and resource status information of the first storage cluster; The number of retries being greater than or equal to a preset threshold includes: the number of retries being greater than or equal to the target threshold.

13. The method according to claim 1, wherein The method further includes: determining that the acquisition of the second processing authority fails when receiving response information indicating that the authority is busy and sent by the second storage cluster.

14. The method according to claim 13, characterized in that The method further comprises: In response to the response information including the estimated duration, adjusting the first duration according to the resource status information of the first storage cluster and the estimated duration; The first duration of waiting for the first storage cluster configuration includes: the first duration of waiting after adjustment.

15. The method according to claim 1, wherein The permission acquisition request includes the target processing data in the target data processing request; The method further comprises: In response to the second processing permission being successfully acquired, writing the target processing data to a second data volume corresponding to the second storage cluster; In response to completion of writing the target processed data to the second data volume, the target processed data is written to the first data volume corresponding to the first storage cluster.

16. The method according to claim 15, characterized in that The method further comprises: determining a writing order of the target processed data according to the resource status information of the first data volume and the resource status information of the second data volume; The target processed data is written to the first data volume and the second data volume according to the writing order.

17. A request processing device, applied to a first storage cluster in a storage system, wherein the storage system includes a first storage cluster and a second storage cluster for processing data in parallel; characterized in that: The device comprises: a permission acquisition module, configured to, in response to receiving a target data processing request, acquire a first processing permission of the first storage cluster and send a permission acquisition request to the second storage cluster, wherein the permission acquisition request is used to acquire a second processing permission of the second storage cluster; and a permission release module, configured to release the first processing permission in response to a failure to obtain the second processing permission, and reprocess the target data processing request after waiting for a first time period configured for the first storage cluster, wherein the second storage cluster is configured with a second time period, and the second time period is different from the first time period.

18. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 16.

19. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

20. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images