Request processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a request processing method and device, electronic equipment, a storage medium and a program product, and can be applied to the technical field of computers. The request processing method is applied to a first storage cluster in a storage system, and the storage system comprises the first storage cluster and a second storage cluster which are used for processing data in parallel. The method comprises the steps that in response to a received target data processing request, a first processing permission of a first storage cluster is obtained, a permission obtaining request is sent to a second storage cluster, and the permission obtaining request is used for obtaining a second processing permission of the second storage cluster; and in response to a second processing permission acquisition failure, releasing the first processing permission, and after waiting for a first duration configured by the first storage cluster, reprocessing the target data processing request, the second storage cluster being configured with a second duration, the second duration being different from the first duration.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and more particularly to the field of storage software technologies. Specifically, the present invention relates to a request processing method, apparatus, electronic device, storage medium, and program product. Background Art

[0002] In the field of computer storage, in order to ensure the reliability of a storage system and the security of stored data, two storage devices can be used to form a storage dual-active system. Both storage devices are in a working state, redundant to each other, and the data on the two storage devices is exactly the same. When one of the storage devices fails, the other storage device can automatically and seamlessly take over the service. Currently, due to the occurrence of scenarios where data is stored at the same location during the actual application process of the storage dual-active system, a phenomenon of data storage conflict is triggered, thereby affecting the processing efficiency of the storage system. Summary of the Invention

[0003] In view of the above problems, the present invention provides a request processing method, apparatus, electronic device, storage medium, and program product that solve data write conflicts and improve the processing efficiency of a storage system.

[0004] One aspect of the present invention provides a request processing method applied to a first storage cluster in a storage system. The storage system includes a first storage cluster and a second storage cluster for parallel data processing. The method includes: in response to receiving a target data processing request, obtaining a first processing permission of the first storage cluster, and sending a permission acquisition request to the second storage cluster, where the permission acquisition request is used to obtain a second processing permission of the second storage cluster; in response to the failure of obtaining the second processing permission, releasing the first processing permission, and after waiting for a first duration configured for the first storage cluster, reprocessing the target data processing request, where the second storage cluster is configured with a second duration, and the second duration is different from the first duration.

[0005] Another aspect of the present invention further provides a request processing apparatus applied to a first storage cluster in a storage system. The storage system includes a first storage cluster and a second storage cluster for parallel data processing. The apparatus includes: a permission acquisition module, configured to, in response to receiving a target data processing request, obtain a first processing permission of the first storage cluster, and send a permission acquisition request to the second storage cluster, where the permission acquisition request is used to obtain a second processing permission of the second storage cluster; a permission release module, configured to, in response to the failure of obtaining the second processing permission, release the first processing permission, and after waiting for a first duration configured for the first storage cluster, reprocess the target data processing request, where the second storage cluster is configured with a second duration, and the second duration is different from the first duration.

[0006] Another aspect of the present invention further provides an electronic device, including: one or more processors; a memory for storing one or more computer programs, and the one or more processors execute the one or more computer programs to implement the steps of the above-mentioned request processing method.

[0007] Another aspect of the present invention further provides a computer-readable storage medium, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of the above-mentioned request processing method are implemented.

[0008] Another aspect of the present invention further provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above-mentioned request processing method are implemented.

[0009] According to an embodiment of the present invention, by responding to a target data processing request, obtaining a first processing permission of a first storage cluster, and sending a permission acquisition request to a second storage cluster; in response to the failure of obtaining the second processing permission, releasing the first processing permission, and after waiting for a first duration, reprocessing the target data processing request. Since during the request processing, when the acquisition of the second processing permission at the remote end fails, the local first processing permission will be released, and after waiting for the first duration, the target data processing request will be reprocessed, the problem of storage conflict is solved; and by setting different first and second durations, the probability of storage conflict occurring again when reprocessing the target data processing request can be reduced, and thus the processing efficiency of the storage system can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Through the following description of the embodiments of the present invention with reference to the drawings, the above content and other objects, features and advantages of the present invention will become clearer. In the drawings:

[0011] Figure 1A The architecture diagram of a storage dual-active system with a master-slave architecture in a related example is shown;

[0012] Figure 1B The architecture diagram of a storage dual-active system with asymmetric read and write in a related example is shown;

[0013] Figure 1C The architecture diagram of a storage dual-active system with symmetric read and write in a related example is shown;

[0014] Figure 1D The schematic diagram of a conflict existing in the architecture in a related example is shown;

[0015] Figure 2 The application scenario diagram of the request processing method according to an embodiment of the present invention is shown;

[0016] Figure 3Shows a flowchart of a request processing method according to an embodiment of the present invention;

[0017] Figure 4 Shows an architecture diagram of a storage dual-active system according to an embodiment of the present invention;

[0018] Figure 5A Shows a schematic diagram of detecting an overlapping area of logical block addresses according to an embodiment of the present invention;

[0019] Figure 5B Shows a schematic diagram of detecting an overlapping area of logical block addresses according to another embodiment of the present invention;

[0020] Figure 6 Shows a framework diagram of a first mapping relationship according to an embodiment of the present invention;

[0021] Figure 7 Shows a schematic diagram of a request processing method according to an embodiment of the present invention;

[0022] Figure 8 Shows a structural block diagram of a request processing device according to an embodiment of the present invention;

[0023] Figure 9 Shows a block diagram of an electronic device suitable for implementing a request processing method according to an embodiment of the present invention. Detailed Embodiments

[0024] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a thorough understanding of the embodiments of the present invention. However, it is obvious that one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.

[0025] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning that those skilled in the art usually understand this expression (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0028] In the field of storage, in order to ensure the reliability of the storage system and the security of the stored data, various disaster recovery and backup technologies have emerged. Not only can snapshot technology and remote replication technology be used for data backup, but also two storage devices can be used to form a storage dual-active system to store data.

[0029] The dual-active storage systems in the related art generally adopt a master-slave (Active-Passive) architecture, an asymmetric read-write dual-active (Active-Active) architecture, and a symmetric read-write Active-Active architecture.

[0030] Figures 1A to 1C Shows an example architecture of a storage dual-active system in related examples.

[0031] Figure 1A Shows an architecture diagram of a storage dual-active system with a master-slave architecture in related examples. In Figure 1A The shown architecture may include a first storage device 103 and a second storage device 104. Among them, the first storage device 103 is the master node, responsible for processing all requests, such as read requests and write requests; the second storage device 104 is the slave node, as a backup, does not process service requests, but will synchronize the data of the master node. For example, the first host 101 and the second host 102 send requests to the first storage device 103. After the first storage device 103 processes the requests, it copies the data to the second storage device 104.

[0032] When the master node, i.e., the first storage device 103, fails, the slave node, i.e., the second storage device 104, will take over automatically or manually and become the new master node to ensure the continuous availability of the storage system.

[0033] Figure 1B Shows an architecture diagram of a storage dual-active system with asymmetric read-write in related examples. In Figure 1B In the shown architecture, the first host 101 can send data requests to the first storage device 103, and the second host 102 can send data requests to the second storage device 104. The second storage device 104 can forward the received data requests to the first storage device 103, and after the first storage device 103 processes them, it synchronously copies the data to the second storage device 104.

[0034] Figure 1C The architecture diagram of the storage dual-active system with symmetric read and write in related examples is shown. In Figure 1C the shown architecture, the first host 101 can send data requests to the first storage device 103, and the second host 102 can send processed data requests to the second storage device 104. After the first storage device 103 processes the data requests sent by the first host 101, the obtained data can be synchronously mirrored to the second storage device 104. After the second storage device 104 processes the data requests sent by the second host 102, the obtained data can be synchronously mirrored to the first storage device 103. Figure 1C The shown architecture does not distinguish between the primary storage device and the secondary storage device.

[0035] Based on Figures 1A to 1C the architecture of the storage dual-active system shown, whether it is a storage dual-active system with a primary-secondary architecture or an asymmetric read-write storage dual-active system, there is a distinction between the primary storage device and the secondary storage device among the two storage devices in the dual-active storage system. After the secondary storage device receives a write request from the host, it needs to forward it to the primary storage device. The primary storage device sorts and processes the write requests received by itself and those forwarded by the secondary storage device, and then synchronizes the data to the secondary storage device. This situation will bring additional link processing delay and affect the performance of the storage system.

[0036] In the storage dual-active architecture with symmetric read and write, the first storage device 103 can establish a dual-active relationship with the second storage device 104. The storage of the first storage device 103 will synchronize to the second storage device 104; the storage of the second storage device 104 will synchronize to the first storage device 103. In this way, when the first storage device 103 fails, there is still complete data on the second storage device 104 for users to use; when the second storage device 104 fails, there is also complete data on the first storage device 103 for users to use. However, in the storage dual-active architecture with symmetric read and write, a scenario where data is stored simultaneously from two storage devices may occur. If the problem of this storage conflict is not handled, it may lead to inconsistent data on the two storage devices, as Figure 1D shown.

[0037] Figure 1D The schematic diagram showing the conflicts existing in the architecture in related examples is shown.

[0038] Such as Figure 1DAs shown, the host cluster where the first host and the second host are located can simultaneously send data processing requests to the first storage device 103 and the second storage device 104. After receiving the data processing request 1, the first storage device 103 needs to obtain the write processing permission first and then can start processing the data processing request; similarly, after receiving the data processing request 2, the second storage device 104 also needs to obtain the write processing permission first and then can start processing the data processing request. Since both the first storage device 103 and the second storage device 104 need to obtain the write permission first to start processing the data processing request, and when the data processing request 1 received by the first storage device 103 is synchronized to the second storage device 104, the second storage device 104 also needs to obtain the write permission first. When the data processing request 2 received by the second storage device 104 is synchronized to the first storage device 103, the first storage device 103 also needs to obtain the write permission first. Therefore, when the first storage device 103 and the second storage device 104 simultaneously receive write requests at the same location, it may occur that the data processing request 1 has obtained the first processing permission (such as the permission of a lock, lock 第一存储设备 ), and attempts to obtain the second processing permission (such as the permission of a lock, lock 第二存储设备 ) on the second storage device 104; while the data processing request 2 has obtained lock 第二存储设备 on the second storage device 104 and attempts to obtain lock 第一存储设备 on the first storage device 103. In this way, a deadlock problem caused by request conflicts will occur, reducing the processing efficiency of the storage system.

[0039] In view of this, an embodiment of the present invention provides a request processing method for solving the problem of storage conflicts and improving the storage efficiency of the storage system.

[0040] Figure 2 FIG. shows an application scenario diagram of the request processing method according to an embodiment of the present invention.

[0041] As Figure 2 shown, the application scenario 200 according to this embodiment may include a first terminal device 201, a second terminal device 202, a third terminal device 203, a network 204, a server 205, and a storage system 206. The network 204 is used to provide communication links between the first terminal device 201, the second terminal device 202, the third terminal device 203, and the server 205, and between the server 205 and the storage system 206. The network 204 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0042] Users can use the first terminal device 201, the second terminal device 202, and the third terminal device 203 to interact with the server 205 via the network 204 to receive or send messages, such as sending a target data processing request, etc. Various communication client applications can be installed on the first terminal device 201, the second terminal device 202, and the third terminal device 203, such as storage applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0043] The first terminal device 201, the second terminal device 202, and the third terminal device 203 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and so on.

[0044] The server 205 can be a server that provides various services, such as a background management server (only for example) that processes the target data processing requests sent by users using the first terminal device 201, the second terminal device 202, and the third terminal device 203 and stores the processed data in the storage system 206. The background management server can analyze and process data such as received requests, and feedback the processing results (such as storage results, web pages, information, or data obtained or generated according to the requests) to the terminal device.

[0045] The storage system 206 can be a system for storing data obtained based on data processing requests. The storage system 206 can include multiple storage clusters for parallel processing of data. The data obtained based on the target data processing request can be stored in the storage cluster.

[0046] It should be noted that the request processing method provided by the embodiments of the present invention can generally be executed by the server 205. Correspondingly, the request processing device provided by the embodiments of the present invention can generally be set in the server 205. The request processing method provided by the embodiments of the present invention can also be executed by a server or a server cluster different from the server 205 and capable of communicating with the first terminal device 201, the second terminal device 202, the third terminal device 203, the storage system 206, and / or the server 205. Correspondingly, the request processing device provided by the embodiments of the present invention can also be set in a server or a server cluster different from the server 205 and capable of communicating with the first terminal device 201, the second terminal device 202, the third terminal device 203, the storage system 206, and / or the server 205.

[0047] It should be understood, Figure 2The number of terminal devices, networks, servers, storage systems, and storage clusters in it is merely illustrative. According to actual requirements, there can be any number of terminal devices, networks, servers, storage systems, and storage clusters.

[0048] Based on the Figure 2 scenario described below, the request processing method of the embodiments of the present invention will be described in detail through Figures 3 to 6 the following.

[0049] Figure 3 FIG. shows a flowchart of the request processing method according to an embodiment of the present invention.

[0050] As Figure 3 shown, the request processing method of this embodiment includes operations S310 to S320.

[0051] In operation S310, in response to receiving a target data processing request, obtain the first processing permission of the first storage cluster, and send a permission acquisition request to the second storage cluster, where the permission acquisition request is used to obtain the second processing permission of the second storage cluster.

[0052] In operation S320, in response to the failure of obtaining the second processing permission, release the first processing permission, and after waiting for the first duration configured for the first storage cluster, re - process the target data processing request, where the second storage cluster is configured with a second duration, and the second duration is different from the first duration.

[0053] In some embodiments, the request processing method of the embodiments of the present invention can be applied to the first storage cluster in a storage system, and the storage system includes a first storage cluster and a second storage cluster for parallel data processing.

[0054] Figure 4 FIG. shows an architecture diagram of a storage dual - active system according to an embodiment of the present invention.

[0055] As Figure 4 shown, the architecture may include a first storage cluster 403 and a second storage cluster 404.

[0056] The first storage cluster 403 and the second storage cluster 404 can communicate through a Fibre Channel (FC) or a Remote Direct Memory Access over Converged Ethernet (ROCE) link. Among them, RDMA is the abbreviation of Remote Direct Memory Access, indicating remote direct memory access.

[0057] In some embodiments, a first data volume 401 can be created on the first storage cluster 403, and a second data volume 402 can be created on the second storage cluster 404. The first data volume 401 and the second data volume 402 can form a dual-active relationship and be mapped as a virtual dual-active volume to the first host 101 and the second host 102. The first data volume 401 can be the dual-active primary volume, and the second data volume 402 can automatically be the secondary volume. For the processing of target data requests, the first data volume 401 and the second data volume 402 can have different data processing policies.

[0058] In some embodiments, the above-mentioned target data processing request can be a request for processing and storing target data. The target data processing request can be a write data request related to updating or deleting data, a read data request related to querying data, etc. The target data processing request can include specific data information such as updates, deletions, and queries.

[0059] In some embodiments, the first processing permission can include an operation lock on the resources of the first storage cluster, such as a local lock permission, and the second processing permission can include an operation lock on the resources of the second storage cluster, such as a remote lock permission.

[0060] In some embodiments, if the first processing permission is successfully obtained or the second processing permission is successfully obtained, it can indicate that the resources of the first storage cluster or the second storage cluster have been occupied, and in this state, the target data processing request can be processed. If the first processing permission is obtained unsuccessfully or the second processing permission is obtained unsuccessfully, it can indicate that the resources of the first storage cluster or the second storage cluster have not been occupied, and in this state, the target data processing request can be not processed, otherwise, it may cause the problem of inconsistent data stored on the first storage cluster and the second storage cluster.

[0061] In some embodiments, in the case where the second processing permission is obtained unsuccessfully, it can indicate that the resources of the second storage cluster have not been occupied, and the resources of the second storage cluster are occupied by other storage clusters other than the first storage cluster. In this case, the first processing permission can be released, that is, the target data processing request can be not processed to avoid the problem of inconsistent data stored on the first storage cluster and the second storage cluster.

[0062] In some embodiments, the first duration can be the retry waiting duration of the first storage cluster. When the first storage cluster waits for the first duration, the data processing request can be awakened to reprocess the target data processing request, that is, to re-obtain the first processing permission and send a permission acquisition request to the second storage cluster to obtain the second processing permission of the second storage cluster.

[0063] In some other embodiments, the second storage cluster can also be used to process the target data processing request. The second duration can be the duration for which the second storage cluster waits to obtain the second processing permission. In response to receiving the target data processing request, the second storage cluster obtains the second processing permission of the second storage cluster and sends a permission acquisition request to the first storage cluster to obtain the first storage permission of the first storage cluster. In the case where the acquisition of the first storage permission fails, the second processing permission can be released, and the target data processing request can be reprocessed after waiting for the second duration.

[0064] In some embodiments, the first duration and the second duration are different. For example, the first duration can be much less than the second duration. For example, the first duration is 10 ms and the second duration is 50 ms. By setting the differentiated first duration and second duration, the probability of synchronization conflicts during the re-acquisition of the processing permission can be reduced, and the orderliness of the storage system processing can be improved.

[0065] According to the embodiments of the present invention, in response to the target data processing request, the first processing permission of the first storage cluster is obtained, and a permission acquisition request is sent to the second storage cluster; in response to the failure of obtaining the second processing permission, the first processing permission is released, and after waiting for the first duration, the target data processing request is reprocessed. Since during the request processing, in the case where the acquisition of the second processing permission at the remote end fails, the first processing permission locally is released, and the target data processing request is reprocessed after waiting for the first duration, the problem of storage conflicts is solved; and by setting the differentiated first duration and second duration, the probability of re-occurring storage conflicts when reprocessing the target data processing request can be reduced, thereby the storage efficiency can be improved, and the reliability and security of the storage system can be ensured.

[0066] In some embodiments, the first data volume of the first storage cluster and the second data volume of the second storage cluster can simultaneously receive and process data processing requests. Therefore, there may be a situation where data is written to the same logical block address (LBA) from the first data volume of the first storage cluster and the second data volume of the second storage cluster at the same time. After the first data volume receives the write data processing request from an external host and before obtaining the first processing permission, it can first check whether there is a local write conflict in the local resources. If there is a conflict, the data processing request enters a waiting state on the first storage cluster to avoid the problem of abnormal conditions caused by forcibly obtaining the first processing permission for the first storage cluster. After the conflict is resolved, it is awakened by other data processing requests except this data processing request; if there is no conflict, the first processing permission of the first storage cluster is obtained.

[0067] Based on this, the above request processing method may further include the following operations: performing first data block conflict detection on the target data processing request and the first data processing request to obtain a first detection result; in response to the first detection result indicating a data block conflict, suspending the processing of the target data processing request; and in response to the conflict being resolved, obtaining the first processing permission for the first storage cluster.

[0068] In some embodiments, the first data processing request may be a data processing request for the first storage cluster other than the target data processing request. For example, the data processing requests for the first storage cluster include data processing request a1, data processing request a2, and data processing request a3. Among them, if data processing request a3 is the target data processing request, then data processing request a1 and data processing request a2 may be the first data processing requests.

[0069] In some embodiments, the data block conflict detection may be used to detect whether there is a conflict between the target data block for the data to be written into the target data processing request and the first data block for the data to be written into the first data processing request. For example, the data block conflict detection may be used to detect whether there is a conflict between the target data block for the data to be written into data processing request a3 and the first data blocks for the data to be written into data processing request a1 and data processing request a2.

[0070] In some embodiments, if the target data block conflicts with the first data block, the processing of the target data processing request is suspended. If the target data block does not conflict with the first data block, or the conflict is resolved, the first processing permission for the first storage cluster may be obtained. In some embodiments, the first processing permission may be the permission to occupy the resources of the target data block.

[0071] According to an embodiment of the present invention, before obtaining the first processing permission, it is determined whether there is a local write conflict in the first storage cluster. If there is a conflict, the system enters a waiting state and waits for the conflict to be resolved before proceeding with the processing. This can avoid problems such as data inconsistency or system anomalies caused by forcibly obtaining the first processing permission, and thus improve the security of operations. On the other hand, after the conflict is resolved, the target data processing request is automatically reprocessed without manual intervention, which can ensure the continuity of the data processing flow.

[0072] In some embodiments, the target data processing request may include the target logical block address of the data to be written into the target processing request, and the first data processing request may include the first logical block address of the data to be written into the first data processing request.

[0073] In some embodiments, the process of performing the first data block conflict detection on the target data processing request and the first data processing request described above may include the following operations: detecting an overlapping area between the target logical block address and the first logical block address to obtain a first overlapping area detection result; in response to the first overlapping area detection result indicating that there is no overlapping area, determining that the first detection result is that the data blocks do not conflict; in response to the first overlapping area detection result indicating that there is an overlapping area, determining that the first detection result is that the data blocks conflict.

[0074] In some embodiments, when the first end position of the first logical block address is located at or after the target start position of the target logical block address, or the first start position of the first logical block address is located at or before the target end position of the target logical block address, it can be considered that there is an overlapping area between the target logical block address and the first logical block address.

[0075] Figure 5A FIG. shows a schematic diagram of detecting an overlapping area of logical block addresses according to an embodiment of the present invention; Figure 5B FIG. shows a schematic diagram of detecting an overlapping area of logical block addresses according to another embodiment of the present invention.

[0076] As Figure 5A shown, the target logical block address may include a target start position (e.g., start LBA2) and a target end position (e.g., end LBA2). The first logical block address may include a first start position (e.g., start LBA1) and a first end position (e.g., end LBA1). Since Figure 5A the start LBA2 of the target logical block address shown is less than the end LBA1 of the first logical block address, it is determined that there is a conflict between the target logical address and the first logical address.

[0077] As Figure 5B shown, the target logical block address may include a target start position (e.g., start LBA2) and a target end position (e.g., end LBA2). The first logical block address may include a first start position (e.g., start LBA1) and a first end position (e.g., end LBA1). Since Figure 5B the end LBA2 of the target logical block address shown is greater than the start LBA1 of the first logical block address, it is determined that there is a conflict between the target logical address and the first logical address.

[0078] In some embodiments, the first overlapping area detection result may characterize the overlapping situation between the target logical block address and the first logical block address. For example, there is an overlapping area or no overlapping area between the target logical block address and the first logical block address.

[0079] In some embodiments, when there is an overlapping area between the target logical block address and the first logical block address, a first detection result indicating a data block conflict can be obtained. When there is no overlapping area between the target logical block address and the first logical block address, a first detection result indicating that the data blocks do not conflict can be obtained.

[0080] According to an embodiment of the present invention, by comparing the target logical block address and the first logical block address, an accurate determination of data block conflicts can be achieved. Only when the target logical block address overlaps with the first logical block address is it determined as a conflict, thereby reducing unnecessary pauses and retry operations and improving the processing efficiency of the target data processing request.

[0081] In some embodiments, a first mapping relationship may be configured in the above-mentioned first storage cluster, and the first mapping relationship may represent the mapping relationship between the first request identifier for the first data processing request and the first logical block address.

[0082] In some embodiments, the first mapping relationship between the first request identifier and the first logical block address may be a key-value pair relationship. The first request identifier may be the key, and the first logical block address may be the value. One first mapping relationship may correspond to one first data processing request.

[0083] In some embodiments, before detecting the overlapping area between the target logical block address and the first logical block address, the first logical block address included in the first data processing request may be obtained according to the first mapping relationship.

[0084] In some embodiments, since one first mapping relationship corresponds to one first data processing request, before detecting the overlapping area between the target logical block address and the first logical block address, the first logical block address occupied by the first storage cluster when processing the first data processing request may be obtained according to the first mapping relationship.

[0085] According to an embodiment of the present invention, by obtaining the first logical block address according to the first mapping relationship before detecting the overlapping area between the target logical block address and the first logical block address, not only can the efficiency of obtaining the first logical block address be improved, but also the detection efficiency of the overlapping area between the target logical block address and the first logical block address can be improved.

[0086] In some embodiments, the above-mentioned target data processing request may further include a Logical Unit Number (LUN); the above method may further include the following operations: determining a target request identifier for the target data processing request according to the logical unit number and the target logical block address; and updating the first mapping relationship according to the target request identifier and the target logical block address.

[0087] In some embodiments, the process of obtaining the request identifier may be as shown in formula (1).

[0088] Request identifier = (Lun Id << 48) | (LBA / data management unit) (1);

[0089] Wherein, Lun Id may be a logical unit number identifier, and LBA may be a logical block address. The data management unit may be 32KB, 256KB, etc., and can be specifically determined according to the storage system. Lun Id << 48 represents that Lun Id is shifted 48 bits to the left.

[0090] In some embodiments, the target request identifier and the target logical block address obtained by the above operation may be supplemented into the first mapping relationship to update the first mapping relationship.

[0091] In some embodiments, by determining the target request identifier according to the logical unit number and the target logical block address, and updating the first mapping relationship according to the target request identifier and the target logical block address, the real-time update of the first mapping relationship can be achieved, which is not only convenient for determining the association status between the data processing request and the data storage location, but also convenient for obtaining the updated logical block address information when processing data requests after the processing time of the target data processing request.

[0092] In some embodiments, there may be multiple first mapping relationships described above, and the multiple first mapping relationships may be divided into at least one mapping relationship group. On this basis, the above method may further include the following operations: determining the target mapping relationship group in at least one mapping relationship group according to the target logical block address included in the target data processing request, wherein the target mapping relationship group includes at least one target first mapping relationship.

[0093] In some of these embodiments, the first mapping relationship may be stored in a hash table structure.

[0094] Figure 6 Shows the structural diagram of the first mapping relationship according to an embodiment of the present invention.

[0095] In some embodiments, the first mapping relationship may include, for example, Figure 6 the hash table framework shown.

[0096] In Figure 6 it, the total capacity of the hash table is 256 data buckets (Bucket), Figure 6Only 0 to 7 buckets are shown, and a bucket is the basic unit of hash table storage. Under each Bucket, there can be at least one chained hash (abbreviated as entry list), where entry1~entry4 can represent different chained hashes. The permissions of the logical block positions under each entry list are represented by lock, and lock1~lock6 can represent the permissions of different logical block positions. Figure 6 The push in it means putting an entry into the hash table, and the dashed line indicates the position where it is connected and put in.

[0097] Continue to refer to Figure 6 , taking Bucket 5 as an example. In the entry list of Bucket 5, each entry corresponds to 1 different key, that is, the request identifier. The request identifier can be determined by the formula shown in formula (1). Each data processing request will generate a corresponding key according to the Lun Id and LBA. For example, for the target data processing request, a target request identifier key is generated, and then the target request identifier key is compared with the key of the entry in the Bucket list. If the target request identifier key is different from the key of the entry in the Bucket list, it means that there are no other data processing requests except the target data processing request accessing the logical block corresponding to the target request identifier key.

[0098] If a key identical to the target request identifier key is found in the key of the entry in the Bucket list, it means that the logical block accessed by the target data processing request is the same as that of other data processing requests except the target data processing request. Then, the value is obtained according to the key, and further determine whether there is an overlap in the positions of the accessed same logical block. For example, in entry 4 of Bucket 5, lock 2, lock 3, and lock4 indicate that the logical blocks accessed by these 3 data processing requests are the same, but the positions are different. lock 3, lock 5, and lock 6 indicate that the positions accessed by these 3 data processing requests are the same. Through these two horizontal and vertical lists, all conflicting and non-conflicting data processing requests can be managed and tracked.

[0099] In some embodiments, the process of determining the target mapping relationship group in at least one mapping relationship group according to the target logical block address included in the target data processing request described above may include the following operations: processing the logical unit number and the target logical block address to obtain a processing result; according to the processing result, determining the target mapping relationship group in at least one mapping relationship group.

[0100] In some of these embodiments, determining the target mapping relationship group among at least one mapping relationship group may be determining the target data bucket number (Bucket Id) in at least one data bucket.

[0101] In some embodiments, the process of obtaining the target mapping relationship group may be as shown in formula (2).

[0102] Bucket Id = ((Lun Id<<48)|(LBA / data processing unit))% hash table length (2);

[0103] Wherein, the hash table length is the number of data buckets in the hash table, << represents a left shift, and % represents taking the remainder.

[0104] Based on formula (2), the process of obtaining a processing result based on the processing logic unit number and the target logical block address may include processing the Lun Id, for example, shifting it left by 48 bits (i.e., Lun Id<<48) to obtain the processing result of the logical unit number shifted left by 48 bits. Then process the logical block address (such as LBA), for example, determine the ratio of LBA to the data processing unit, and the obtained ratio can be used as the processing result. Then, after concatenating the logical unit number shifted left by 48 bits and the ratio of LBA to the data processing unit, and dividing by the number of data buckets, the obtained remainder is the number of the target data bucket, thereby obtaining Bucket Id.

[0105] According to an embodiment of the present invention, by determining the target mapping relationship group according to the logical unit number and the logical block address, and mapping the logical block address to a unique target mapping relationship group, it is possible to avoid the problem of uneven distribution caused by storing hash data in a single dimension (such as only according to the logical unit number or only according to the logical block address), thereby improving the load balancing of the hash table.

[0106] In some embodiments, the target mapping relationship group may include at least one target first mapping relationship. The process of obtaining the first logical block address included in the first data processing request according to the first mapping relationship may include the following operations: obtaining the first logical block address included in the first data processing request according to the target first mapping relationship. The target first mapping relationship indicates the mapping relationship between the first request identifier of the first data processing request and the first logical block address. According to the target first mapping relationship, the first logical block address related to the first data processing request can be obtained.

[0107] In some embodiments, the target first mapping relationship in the target mapping group can be traversed based on the target mapping group obtained according to the logical unit number and the target logical block address, and the first logical block address indicated by the target first mapping relationship and the target logical block address can be used for detection, thereby eliminating the need to traverse all mapping relationship groups on the first storage cluster, reducing the number of invalid comparisons of the first logical block address and the target logical block address, and improving the efficiency of conflict detection.

[0108] In some embodiments, in the first storage cluster, after determining the location of the target mapping group (e.g., bucket identifier (Bucket Id)) according to the target data processing request, the target first mapping relationship in the target mapping group can be traversed, and the corresponding value, i.e., the target logical block address, can be obtained according to the key key composed of the logical unit number and the target logical block in the target data processing request. The starting position and the ending position of the detailed information in the target logical block address, such as start LBA2 and end LBA2, are used to determine whether there is a conflict with the first logical block address, such as start LBA1 and end LBA1. If start LBA2 ≤end LBA1 or end LBA2 ≥start LBA1), it is determined that there is a conflict between the target logical block address and the first logical block address.

[0109] In some embodiments, the above method may further include the following operations: when the first detection result indicates a data block conflict, writing the mapping relationship for the target data processing request into a hash table, and marking the state of the target data processing request as the first state; in response to the resolution of the data block conflict, waking up the target data processing request, and updating the state of the target data processing request to the second state.

[0110] In some embodiments, if there is no conflict between the target logical block address and the first logical block address, the target data processing request is granted the first processing permission in the first storage cluster, and the mapping information between the target request identifier generated by the target processing request and the target logical block address is written into the hash table of the first storage cluster for management.

[0111] In some embodiments, if there is a conflict between the target logical block address and the first logical block address, mark the status of the target data processing request in the first storage cluster as the first status (e.g., the status of waiting for the first processing permission), and write the mapping relationship between the target request identifier generated by the target processing request and the target logical block address into the hash table of the first storage cluster. In response to the resolution of the conflict between the target logical block address and the first logical block address, that is, after waiting for the first data processing request conflicting with the target data processing request to release the first processing permission, re - wake up the target data processing request and update the first status to the second status (e.g., the status of obtaining the first processing permission) to grant the target data processing request the first processing permission in the first storage cluster.

[0112] According to the embodiments of the present invention, by marking the target data processing request as the first status or the second status, the conflict situation between the target logical block address and the first logical block address can be clearly distinguished, facilitating the system to schedule according to priority or timing, and avoiding omission or confusion of data processing requests. On the other hand, the second status can be used as a trigger condition after the conflict is resolved. When the conflict is resolved, the target data processing request is automatically re - processed, reducing manual intervention and improving the efficiency and intelligence of processing the target data processing request.

[0113] In some embodiments, if the target data processing request has obtained the first processing permission on the first storage cluster, the information and data in the target data processing request can be sent to the remote second storage cluster and the second processing permission of the second storage cluster can be obtained.

[0114] In some embodiments, the permission acquisition request may include: the target logical block address in the target data processing request; the process of obtaining the second processing permission of the second storage cluster may include the following operations: in response to the second detection result indicating that the data blocks do not conflict, grant the second processing permission to the first storage cluster, where the second detection result is obtained by the second storage cluster performing a second data block conflict detection on the permission acquisition request and the second data processing request according to the target logical block address, and the second data processing request is a data processing request for the second storage cluster.

[0115] In some embodiments, after receiving the target data processing request sent by the first storage cluster, the second storage cluster may refer to the above - mentioned conflict detection process between the target logical block address and the first logical block address, and perform a second data block conflict detection on the permission acquisition request and the second data processing request according to the address of the target logical block. The second data processing request may be the remaining data processing requests processed by the second storage cluster other than the target data processing request.

[0116] According to an embodiment of the present invention, by granting a second processing permission to a target data processing request when the target logical block address and the second logical block address do not conflict, it is possible to prevent the operation of writing data by the first storage cluster from conflicting with the existing operations of the second storage cluster, thereby ensuring data consistency and system stability in the dual-active and dual-write scenario.

[0117] In some embodiments, the process of performing a second data block conflict detection on the permission acquisition request and the second data processing request described above may include the following operations: determining a second logical block address for the second data processing request according to the second mapping relationship; performing an overlapping area detection on the target logical block address and the second logical block address to obtain a second overlapping area detection result; in response to the second overlapping area detection result indicating that there is no overlapping area, determining that the second detection result is that the data blocks do not conflict; in response to the second overlapping area detection result indicating that there is an overlapping area, determining that the second detection result is that the data blocks conflict.

[0118] In some embodiments, the process of performing a second data block conflict detection on the permission acquisition request and the second data processing request may refer to the process of performing a first data block conflict detection on the target data processing request and the first data processing request described above, that is, the process of performing a conflict detection on the target logical block address and the first logical block address.

[0119] In some embodiments, the second storage cluster may be configured with a second mapping relationship, and the second mapping relationship may represent the mapping relationship between the second request identifier of the second data processing request and the second logical block address. The process of obtaining the second request identifier and the process of obtaining the first request identifier described above may both refer to the process of obtaining the target request identifier described above, and may refer to formula (1), that is, the first request identifier or the second request identifier may be obtained according to the logical unit number and the logical block address associated with the first data processing request or the second data processing request respectively.

[0120] According to the second mapping relationship, a second logical block address for the second data processing request can be determined. The process of detecting the overlapping area between the target logical block address and the second logical block address may refer to the process of detecting the overlapping area between the target logical block address and the first logical block address described in FIG. 5 above. The second detection result may include second data block conflict or second data block non-conflict.

[0121] For example, the second logical block address may include a second start position (e.g., start LBA3) and a second end position (e.g., end LBA3). When start LBA3 ≤ end LBA1 or end LBA3 ≥ start LBA1, it can be determined that there is an overlapping area between the target logical address and the second logical address, and then the second detection result can be determined as a data block conflict. When start LBA3 > end LBA1 or end LBA3 < start LBA1, it can be determined that there is no overlapping area between the target logical address and the second logical address, and then the second detection result can be determined as no data block conflict.

[0122] In some embodiments, after the second storage cluster receives a target data processing request sent by the peer first storage cluster or a permission acquisition request associated with the target data processing request, it can adopt the above conflict detection method. By traversing the hash table on the second storage cluster (the hash table structure on the second storage cluster can be the same as the hash table structure on the first storage cluster, refer to Figure 6 ), it detects whether there is a conflict between the target data processing request sent from the first storage cluster or the permission acquisition request associated with the target data processing request and the second data processing request sent from the host to the second storage cluster. If there is no conflict, the second processing permission is granted to the target data processing request sent from the first storage cluster or the permission acquisition request associated with the target data processing request.

[0123] According to the embodiments of the present invention, by performing conflict detection on the second logical block address and the target logical block address, an accurate judgment of data block conflict can be achieved. Only when there is an overlapping area between the second logical block address and the target logical block address, the second detection result is determined as a data block conflict, which can reduce unnecessary pauses and retries of the second storage cluster. On the other hand, by using the second hash table to map the second data processing request for the second storage cluster to the corresponding mapping relationship group according to the second request identifier, conflict detection is only performed on the second target logical block and the target logical block within the mapping relationship group, without traversing the logical block addresses of all mapping relationship groups of the second storage cluster, reducing invalid comparison operations, reducing the consumption of computing resources and storage resources of the system, and improving the system operation efficiency.

[0124] In some embodiments, the above permission acquisition request may include the target processing data in the target data processing request. The above method may further include the following operations: in response to successful acquisition of the second processing permission, writing the target processing data to the second data volume for the second storage cluster; in response to the completion of writing the target processing data to the second data volume, writing the target processing data to the first data volume for the first storage cluster.

[0125] In some embodiments, when the acquisition of the second processing permission is successful, the target processing data can be directly written into the second data volume of the second storage cluster. After the writing is completed, the second processing permission of the second storage cluster can be released, and the writing result in the second storage cluster (such as the result that the target processing data has been written into the second data volume of the second storage cluster) can be returned to the first storage cluster. The first storage cluster can then write the target processing data in the target data processing request into the first data volume of the first storage cluster, thus completing the dual-write operation for the target data processing request.

[0126] According to the embodiments of the present invention, after acquiring the second processing permission, the data is first stored in the second data volume and then in the first data volume. Executing the remote writing only after the acquisition of the second processing permission is successful can ensure that the writing operation has legal permissions, avoid unauthorized access, and improve system stability. In addition, writing data to the second data volume of the second storage cluster first and then to the first data volume can reduce the number of data interactions and improve processing efficiency.

[0127] In some embodiments, the above method may further include the following operations: determining the writing order of the target processing data according to the resource status information of the first data volume and the resource status information of the second data volume; and writing the target processing data into the first data volume and the second data volume according to the writing order.

[0128] In some embodiments, the resource status information of the first data volume and the resource status information of the second data volume may include information such as storage capacity, central processing unit (CPU), and network traffic. The writing order of the target data can be determined according to these status information.

[0129] For example, when the storage capacity of the first data volume is greater than that of the second data volume, the target data can be first written to the first data volume and then to the second data volume. Another example is that the storage capacity, CPU, and network traffic can be configured with their respective weight information, and the write order of the target processed data is determined according to the sum of the products of the storage capacity, CPU, and network traffic and their respective weights. For example, a first weight can be assigned to the storage capacity to reflect the influence degree of the storage capacity on the first data volume or the second data volume. A second weight is assigned to the CPU utilization rate to represent the influence degree of the CPU computing resources on the first data volume or the second data volume. A third weight is assigned to the network traffic to reflect the influence degree of the network bandwidth or latency on the first data volume or the second data volume. The storage capacity, CPU utilization rate, and network traffic of the first data volume and the second data volume collected can be standardized to obtain the standardized storage capacity, CPU utilization rate, and network traffic. The standardized storage capacity, CPU utilization rate, and network traffic of the first data volume and the second data volume are multiplied and summed with the first weight, the second weight, and the third weight to obtain the load evaluation values corresponding to the first data volume and the second data volume respectively. If the load evaluation value of the first data volume is higher than that of the second data volume, the data is preferentially written to the second data volume; if the load evaluation value of the first data volume is lower than that of the second data volume, the data is preferentially written to the first data volume.

[0130] Different standardization processes can be adopted for the storage capacity, CPU utilization rate, and network traffic. For example, logarithmic transformation can be used for storage capacity standardization to uniformly convert to the same unit; for another example, CPU utilization rate standardization can use standard deviation standardization (Z-score standardization) to convert the CPU utilization rate to the range of [0, 100]. For another example, network traffic standardization can adopt time window normalization. For example, it is sliced by 5 minutes. According to the embodiments of the present invention, by intelligently adjusting the write order according to the resource status information of the first data volume or the second data volume, unilateral overload can be avoided and automatic load balancing among storage clusters can be achieved.

[0131] In some embodiments, when the response information of permission busy sent by the second storage cluster is received, it can be determined that the acquisition of the second processing permission fails.

[0132] In some embodiments, the above description is about the embodiments where the acquisition of the second processing permission is successful, while the acquisition of the second processing permission can also fail. For example, if on the second storage cluster, it is detected that the target data processing request sent from the first storage cluster or the permission acquisition request associated with the target data processing request conflicts with the second data processing request, in order to avoid deadlock problems, a response information of permission busy can be directly returned to the first storage cluster.

[0133] According to an embodiment of the present invention, by determining that the second permission acquisition fails according to the received permission busy information, it is possible to prevent the first storage cluster from continuously waiting due to timeout or unknown errors, thereby improving the system response efficiency.

[0134] In some embodiments, on the first storage cluster, after receiving the response information indicating permission busy returned by the second storage cluster, the first processing permission that the target data processing request has obtained on the first storage cluster can be released, and then the target data processing request can be suspended. After waiting for a first duration, the target data processing request is awakened, and after obtaining the first processing permission of the first storage cluster again, the target processing data and the permission acquisition request of the target data processing request are sent to the second storage cluster again. To avoid conflict problems from occurring again, there can be a significant difference in the waiting durations of the data processing requests sent by the host cluster received by the first data volume and the second data volume. For example, in an embodiment of the present invention, for the target data processing request sent by the host received by the first data volume, after receiving the response information indicating permission busy returned by the remote second storage cluster, the first duration of waiting on the first storage cluster can be set shorter, such as 10 ms; for the data processing request sent by the host received by the second data volume, after receiving the response information indicating permission busy returned by the remote first storage cluster, the second duration of waiting on the second storage cluster can be set slightly longer, such as 50 ms, to avoid conflict problems when obtaining the processing permission of the remote end again and improve the storage efficiency.

[0135] In some embodiments, the above method may further include the following operations: in response to the response information including an estimated duration, adjusting the first duration according to the resource status information of the first storage cluster and the estimated duration; waiting for the first duration configured by the first storage cluster includes: waiting for the adjusted first duration.

[0136] In some embodiments, the estimated duration may be the duration required for the second storage cluster to release the second processing permission. Based on information such as the storage capacity, CPU, memory, and network traffic of the first storage cluster, as well as the estimated duration, the first duration required for the first storage cluster to wake up the target data processing request can be adjusted. For example, if the CPU of the current first storage cluster is higher than the predetermined CPU threshold, indicating that the current first storage cluster is in a high-load state, then the predetermined duration (obtained based on CPU utilization) can be continuously extended when the first duration is extended by the estimated duration. For example, if the CPU of the current first storage cluster is lower than the predetermined CPU threshold, indicating that the current first storage cluster is in a low-load state, then the first duration can be extended by the estimated duration. This can avoid frequently waking up the target data processing request within the estimated duration and the state where the second processing permission is always obtained unsuccessfully due to frequently obtaining the second processing permission, thereby improving the efficiency and accuracy of obtaining the second processing permission and further improving the storage efficiency.

[0137] According to an embodiment of the present invention, by dynamically adjusting the first duration according to the estimated duration, resource waste or response delay caused by waiting for a fixed duration can be avoided.

[0138] In some embodiments, the above method may further include the following operations: recording the number of retries for reprocessing the target data processing request; and in response to the number of retries being greater than or equal to a preset threshold, abandoning the processing of the target data processing request and sending a request failure message to the client.

[0139] In some embodiments, if the number of retries for reprocessing the target data processing request has reached 3 consecutive retries, and the results obtained from reprocessing the target data processing request are all corresponding messages indicating that the remote second storage cluster is busy with permissions, then the target data processing request can be abandoned from being written into the first storage cluster, and a response message indicating that the processing of the target data processing request has failed can be returned to the host cluster for the host cluster to retry the target data processing request. Since the probability that the host cluster outside the first storage cluster and the second storage cluster writes to the same location in the first storage cluster and the second storage cluster simultaneously is extremely low, this processing method has a relatively minor impact on the host service.

[0140] According to an embodiment of the present invention, by setting a preset threshold for the number of retries and abandoning the processing of the target data processing request after the number of retries exceeds the preset threshold. By restricting the number of times of reprocessing the target data processing request, it is possible to prevent infinite retries of the request due to abnormal conditions such as continuous lock conflicts and network failures, avoid the ineffective occupation of resources such as CPU, memory, and network bandwidth, and ensure the normal operation of the core services of the storage system.

[0141] In some embodiments, there may be multiple threshold values described above, and the above method may further include the following operations: determining a target threshold value among multiple preset threshold values according to the cluster level and resource status information of the first storage cluster; the number of retry attempts being greater than or equal to the preset threshold value includes: the number of retry attempts being greater than or equal to the target threshold value.

[0142] In some embodiments, the cluster level of the first storage cluster may include a storage level. For example, different storage levels are pre-configured for different storage clusters; the storage level may also be determined according to information such as the scale and importance of the data stored in the first storage cluster. Different storage levels are configured with different preset threshold values for retry operations to facilitate the diversified management of the storage cluster. Under each storage level, a preset threshold value table may be configured, and each evaluation range in this table corresponds to a preset threshold value.

[0143] In some embodiments, according to the resource status information of the first storage cluster, for example, according to the sum of the products of the storage capacity, CPU, and network traffic and their respective weights, the evaluation value of the first storage cluster is determined. The evaluation value is matched with the evaluation ranges in the preset threshold value table, and the target evaluation range that matches the evaluation value is determined from multiple evaluation ranges. The preset threshold value corresponding to the target evaluation range is used as the target threshold value, and this target threshold value can be compared with the number of retry attempts.

[0144] According to the embodiments of the present invention, determining the target threshold value according to the storage cluster level and resource status can implement determining different target threshold values for different storage clusters and different resource statuses, so that the storage cluster can have different numbers of retry attempts, and can achieve the diversified management of the storage cluster.

[0145] In some embodiments, corresponding to the preset threshold value of the first storage cluster, the second storage cluster may also have multiple preset threshold values. According to the cluster level and resource status information of the first storage cluster, the target threshold value among the multiple preset threshold values can be determined, and according to this target threshold value, it is determined whether the second storage cluster needs to continue to retry processing or abandon processing when processing the target data processing request.

[0146] Figure 7 A schematic diagram of a request processing method according to an embodiment of the present invention is shown.

[0147] As Figure 7 shown, the request processing method of this embodiment is applied to a processing architecture including a first storage device 103 and a second storage device 104. The first storage device 103 can process a first data processing request 701, and the second storage device 104 can process a second data processing request 702. The first data processing request 701 may be issued by a first host, and the second data processing request 702 may be issued by a second host.

[0148] The first storage device 103 receives a first data processing request 701 from the host. Upon obtaining the first processing permission, the first storage device 103 sends a permission acquisition request to the second storage device 104 to obtain the second processing permission of the second storage device 104. Simultaneously, the second storage device 104 receives a second data processing request 702 and then, upon obtaining the second processing permission, sends a permission acquisition request to the first storage device 103 to obtain the first processing permission of the first storage device 103. Due to the simultaneous operations, when the first storage device 103 obtains the second processing permission, since the second processing permission is occupied, it receives a response message from the second storage device 104 stating that the second processing permission is busy. When the second storage device 104 obtains the first processing permission, since the first processing permission is occupied, it receives a response message from the first storage device 103 stating that the first processing permission is busy. Upon receiving the second processing permission busy response, the first storage device 103 may suspend the first data processing request for a first duration, and then reactivate the first data processing request after the first duration to obtain the second processing permission again. When receiving the notification that the first processing authority is busy, the second storage device 104 may suspend the second data processing request for a second period of time, and wake up the second data processing request again after the second period of time, and obtain the first processing authority again.

[0149] According to an embodiment of the present invention, by releasing the second processing authority or the first processing authority when the first processing authority is busy or the second processing authority is busy, and reacquiring the first processing authority or the second processing authority after waiting for the second period of time or the first period of time, continuous storage conflicts can be avoided and the reliability and security of storage can be improved.

[0150] It should be noted that, unless it is clearly stated that there is a sequence of execution between different operations shown in the flowchart in the embodiments of the present invention, or there is a sequence of execution between different operations in technical implementation, otherwise, the execution order of multiple operations may not be prioritized, and multiple operations may also be executed simultaneously.

[0151] Based on the above request processing method, the present invention also provides a request processing device. Figure 8 The device is described in detail.

[0152] Figure 8 A structural block diagram of a request processing device according to an embodiment of the present invention is shown.

[0153] like Figure 8As shown in the figure, the request processing device 800 of this embodiment is applied to the first storage cluster in the storage system. The storage system includes a first storage cluster and a second storage cluster for parallel data processing. The request processing device 800 may include a permission acquisition module 810 and a permission release module 820.

[0154] The permission acquisition module 810 is configured to, in response to receiving a target data processing request, acquire the first processing permission of the first storage cluster and send a permission acquisition request to the second storage cluster, where the permission acquisition request is used to acquire the second processing permission of the second storage cluster.

[0155] The permission release module 820 is configured to, in response to the failure of acquiring the second processing permission, release the first processing permission, and after waiting for the first duration configured for the first storage cluster, re-process the target data processing request, where the second storage cluster is configured with a second duration, and the second duration is different from the first duration.

[0156] In some embodiments, the request processing device may further include a detection module, a pause module, and a response module.

[0157] The detection module is configured to perform a first data block conflict detection on the target data processing request and the first data processing request to obtain a first detection result, where the first data processing request is a data processing request for the first storage cluster other than the target data processing request.

[0158] The pause module is configured to, in response to the first detection result indicating a data block conflict, pause processing the target data processing request.

[0159] The response module is configured to, in response to the conflict being resolved, acquire the first processing permission of the first storage cluster.

[0160] In some embodiments, the detection module may include a first detection sub-module, a first determination sub-module, and a second determination sub-module.

[0161] The first detection sub-module is configured to perform an overlapping area detection on the target logical block address and the first logical block address to obtain a first overlapping area detection result.

[0162] The first determination sub-module is configured to, in response to the first overlapping area detection result indicating no overlapping area, determine that the first detection result is that the data blocks do not conflict.

[0163] The second determination sub-module is configured to, in response to the first overlapping area detection result indicating an overlapping area, determine that the first detection result is that the data blocks conflict.

[0164] In some embodiments, the request processing device may further include an address acquisition module.

[0165] The address acquisition module is configured to acquire the first logical block address included in the first data processing request according to a first mapping relationship before performing overlapping area detection on the target logical block address and the first logical block address.

[0166] In some embodiments, the request processing device may further include a determination module and an update module.

[0167] The first determining module is configured to determine a target request identifier for a target data processing request according to a logical unit number and a target logical block address.

[0168] The updating module is configured to update the first mapping relationship according to the target request identifier and the target logical block address.

[0169] In some embodiments, the request processing device may further include a second determination module.

[0170] The second determining module is configured to determine a target mapping relationship group in at least one mapping relationship group according to a target logical block address included in the target data processing request, wherein the target mapping relationship group includes at least one target first mapping relationship.

[0171] In some embodiments, the address acquisition module may include an address acquisition sub-module.

[0172] The address acquisition submodule is configured to acquire a first logical block address included in the first data processing request according to a target first mapping relationship.

[0173] In some embodiments, the second determination module may include a processing submodule and a third determination submodule.

[0174] The processing submodule is used to process the logical unit number and the target logical block address to obtain a processing result.

[0175] The third determining submodule is configured to determine a target mapping relationship group in the at least one mapping relationship group according to the processing result.

[0176] In some embodiments, the request processing device may further include a first writing module and a wake-up module.

[0177] The first writing module is configured to write a mapping relationship for the target data processing request into the hash table and mark the state of the target data processing request as a first state when the first detection result indicates a data block conflict.

[0178] The wake-up module is configured to wake up the target data processing request in response to the resolution of the data block conflict, and update the state of the target data processing request to a second state.

[0179] In some embodiments, the rights acquisition module may include a granting submodule.

[0180] An authorization sub-module, configured to, in response to the second detection result indicating that data blocks do not conflict, authorize the first storage cluster with a second processing permission, where the second detection result is obtained by the second storage cluster performing a second data block conflict detection on a permission acquisition request and a second data processing request according to a target logical block address, and the second data processing request is a data processing request for the second storage cluster.

[0181] In some embodiments, the authorization sub-module may include a first determination unit, a detection unit, a second determination unit, and a third determination unit.

[0182] The first determination unit is configured to determine a second logical block address for the second data processing request according to the second mapping relationship.

[0183] The detection unit is configured to perform an overlapping region detection on the target logical block address and the second logical block address to obtain a second overlapping region detection result.

[0184] The second determination unit is configured to, in response to the second overlapping region detection result indicating that there is no overlapping region, determine that the second detection result is that the data blocks do not conflict.

[0185] The third determination unit is configured to, in response to the second overlapping region detection result indicating that there is an overlapping region, determine that the second detection result is that the data blocks conflict.

[0186] In some embodiments, the request processing device may further include a recording module and a sending module.

[0187] The recording module is configured to record the number of retries for reprocessing the target data processing request.

[0188] The sending module is configured to, in response to the number of retries being greater than or equal to a preset threshold, abandon processing the target data processing request and send a request failure message to the client.

[0189] In some embodiments, the request processing device may further include a third determination module.

[0190] The third determination module is configured to determine a target threshold among a plurality of preset thresholds according to the cluster level and resource status information of the first storage cluster; the number of retries being greater than or equal to the preset threshold includes: the number of retries being greater than or equal to the target threshold.

[0191] In some embodiments, the request processing device may further include a fourth determination module.

[0192] The fourth determination module is configured to determine that the acquisition of the second processing permission fails in the case of receiving a response message indicating that the permission is busy sent by the second storage cluster.

[0193] In some embodiments, the request processing device may further include an adjustment module.

[0194] The adjustment module is configured to, in response to the response information including an estimated duration, adjust the first duration according to the resource status information of the first storage cluster and the estimated duration; the first duration for waiting for the configuration of the first storage cluster includes: waiting for the adjusted first duration.

[0195] In some embodiments, the request processing device may further include a second writing module and a third writing module.

[0196] The second writing module is configured to, in response to successful acquisition of the second processing permission, write target processing data to a second data volume for the second storage cluster.

[0197] The third writing module is configured to, in response to the completion of writing the target processing data to the second data volume, write the target processing data to a first data volume for the first storage cluster.

[0198] In some embodiments, the request processing device may further include a fifth determination module and a fourth writing module.

[0199] The fifth determination module is configured to determine the writing order of the target processing data according to the resource status information of the first data volume and the resource status information of the second data volume.

[0200] The fourth writing module is configured to write the target processing data to the first data volume and the second data volume according to the writing order.

[0201] According to the embodiments of the present invention, any multiple modules among the permission acquisition module 810 and the permission release module 820 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to the embodiments of the present invention, at least one of the permission acquisition module 810 and the permission release module 820 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, etc., implemented by hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in any appropriate combination of them. Alternatively, at least one of the permission acquisition module 810 and the permission release module 820 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions may be executed.

[0202] Figure 9A block diagram of an electronic device suitable for implementing a request processing method according to an embodiment of the present invention is shown.

[0203] As Figure 9 shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The processor 901 can include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 can also include on-board memory for caching purposes. The processor 901 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0204] In the RAM 903, various programs and data required for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of the method flow according to an embodiment of the present invention by executing the programs in the ROM 902 and / or the RAM 903. It should be noted that the program can also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 can also perform various operations of the method flow according to an embodiment of the present invention by executing the programs stored in the one or more memories.

[0205] According to an embodiment of the present invention, the electronic device 900 can further include an input / output (I / O) interface 905, and the input / output (I / O) interface 905 is also connected to the bus 904. The electronic device 900 can further include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read from it can be installed into the storage section 908 as needed.

[0206] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiments of the present invention is implemented.

[0207] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the above-described ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903.

[0208] An embodiment of the present invention further includes a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to cause the computer system to implement the request processing method provided by the embodiments of the present invention.

[0209] When the computer program is executed by the processor 901, the above functions defined in the system / apparatus of the embodiments of the present invention are executed. According to an embodiment of the present invention, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[0210] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and is downloaded and installed through the communication part 909, and / or installed from the removable medium 911. The program code included in the computer program may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0211] In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiments of the present invention are executed. According to the embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc. described above can be implemented by computer program modules.

[0212] According to the embodiments of the present invention, the program code for executing the computer program provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedures and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0213] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0214] Those skilled in the art can understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0215] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A request processing method, applied to a first storage cluster in a storage system, the storage system including a first storage cluster and a second storage cluster for parallel data processing; characterized in that, The method includes: In response to receiving a target data processing request, obtaining a first processing permission of the first storage cluster, and sending a permission acquisition request to the second storage cluster, where the permission acquisition request is used to obtain a second processing permission of the second storage cluster; In response to the failure of obtaining the second processing permission, releasing the first processing permission, and after waiting for a first duration configured by the first storage cluster, reprocessing the target data processing request, where the second storage cluster is configured with a second duration, and the second duration is different from the first duration.

2. The method according to claim 1, wherein The method further includes: Performing a first data block conflict detection on the target data processing request and a first data processing request to obtain a first detection result, where the first data processing request is a data processing request for the first storage cluster other than the target data processing request; In response to the first detection result indicating a data block conflict, suspending the processing of the target data processing request; In response to the conflict being resolved, obtaining the first processing permission of the first storage cluster.

3. The method according to claim 2, wherein The target data processing request includes a target logical block address, and the first data processing request includes a first logical block address; The performing a first data block conflict detection on the target data processing request and the first data processing request includes: Performing an overlapping region detection on the target logical block address and the first logical block address to obtain a first overlapping region detection result; In response to the first overlapping region detection result indicating no overlapping region, determining that the first detection result is that the data blocks do not conflict; In response to the first overlapping region detection result indicating an overlapping region, determining that the first detection result is that the data blocks conflict.

4. The method according to claim 3, wherein The first storage cluster is configured with a first mapping relationship, and the first mapping relationship represents the mapping relationship between a first request identifier for the first data processing request and the first logical block address; The method further includes: Before performing the overlapping region detection on the target logical block address and the first logical block address, obtaining the first logical block address included in the first data processing request according to the first mapping relationship.

5. The method according to claim 4, characterized in that, The target data processing request further includes a logical unit number; The method further includes: Determining a target request identifier for the target data processing request according to the logical unit number and the target logical block address; Updating the first mapping relationship according to the target request identifier and the target logical block address.

6. The method according to claim 5, characterized in that, There are multiple first mapping relationships, and the multiple first mapping relationships are divided into at least one mapping relationship group; The method further includes: Determining a target mapping relationship group in at least one mapping relationship group according to the target logical block address included in the target data processing request, where the target mapping relationship group includes at least one target first mapping relationship; The obtaining the first logical block address included in the first data processing request according to the first mapping relationship includes: Obtaining the first logical block address included in the first data processing request according to the target first mapping relationship.

7. The method according to claim 6, characterized in that, The first mapping relationship includes a hash table; Determining a target mapping relationship group in at least one mapping relationship group includes: Processing the logical unit number and the target logical block address to obtain a processing result; According to the processing result, a target mapping relationship group in the at least one mapping relationship group is determined.

8. The method according to claim 7, wherein The method further comprises: If the first detection result indicates a data block conflict, writing a mapping relationship for the target data processing request into the hash table and marking the state of the target data processing request as a first state; In response to the data block conflict being resolved, the target data processing request is awakened, and the state of the target data processing request is updated to a second state.

9. The method according to claim 1, characterized in that, The permission acquisition request includes: the target logical block address in the target data processing request; Acquiring the second processing permission of the second storage cluster includes: In response to the second detection result indicating that the data blocks do not conflict, the first storage cluster is granted the second processing authority, wherein the second detection result is obtained by the second storage cluster performing a second data block conflict detection on the authority acquisition request and the second data processing request according to the target logical block address, and the second data processing request is a data processing request for the second storage cluster.

10. The method according to claim 9, wherein The second storage cluster is configured with a second mapping relationship, where the second mapping relationship represents a mapping relationship between a second request identifier of a second data processing request and a second logical block address; The performing second data block conflict detection on the permission acquisition request and the second data processing request includes: determining, according to the second mapping relationship, a second logical block address for the second data processing request; Performing overlapping area detection on the target logical block address and the second logical block address to obtain a second overlapping area detection result; In response to the second overlapping area detection result indicating that no overlapping area exists, determining the second detection result as data block non-conflict; In response to the second overlapping area detection result indicating the existence of an overlapping area, the second detection result is determined to be a data block conflict.

11. The method according to claim 1, characterized in that, The method further comprises: Recording the number of retries for reprocessing the target data processing request; In response to the number of retries being greater than or equal to a preset threshold, processing the target data processing request is abandoned, and a request failure message is sent to the client.

12. The method according to claim 11, wherein The preset threshold value includes multiple ones; The method further comprises: determining a target threshold value among the plurality of preset threshold values according to the cluster level and resource status information of the first storage cluster; The number of retries being greater than or equal to a preset threshold includes: the number of retries being greater than or equal to the target threshold.

13. The method according to claim 1, wherein The method further includes: determining that the acquisition of the second processing authority fails when receiving response information indicating that the authority is busy and sent by the second storage cluster.

14. The method according to claim 13, wherein The method further comprises: In response to the response information including the estimated duration, adjusting the first duration according to the resource status information of the first storage cluster and the estimated duration; The first duration of waiting for the first storage cluster configuration includes: the first duration of waiting after adjustment.

15. The method according to claim 1, wherein The permission acquisition request includes the target processing data in the target data processing request; The method further comprises: In response to successful acquisition of the second processing permission, write the target processing data to the second data volume for the second storage cluster; In response to completion of writing the target processing data to the second data volume, write the target processing data to the first data volume for the first storage cluster.

16. The method according to claim 15, characterized in that, The method further includes: Determine the writing order of the target processing data according to the resource status information of the first data volume and the resource status information of the second data volume; Write the target processing data to the first data volume and the second data volume according to the writing order.

17. A request processing device is applied to a first storage cluster in a storage system. The storage system includes a first storage cluster and a second storage cluster for parallel data processing. It is characterized in that The apparatus includes: A permission acquisition module, configured to, in response to receiving a target data processing request, acquire a first processing permission for the first storage cluster, and send a permission acquisition request to the second storage cluster, where the permission acquisition request is used to acquire a second processing permission for the second storage cluster; A permission release module, configured to, in response to failure to acquire the second processing permission, release the first processing permission, and re-process the target data processing request after waiting for a first duration configured by the first storage cluster, where the second storage cluster is configured with a second duration, and the second duration is different from the first duration.

18. An electronic device, comprising: One or more processors; A memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 16.

19. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 16.

20. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 16.

Images