An iSCSI service exception IO processing method, system and device and storage medium

Abstract

The application relates to an iSCSI service abnormal IO processing method, system and device and a storage medium. An iSCSI client sends an abandonment instruction to a storage side iSCSI service according to any IO request timeout, wherein the abandonment instruction contains a mark of the IO request, the storage side iSCSI service records the mark and sends the abandonment instruction to a Ceph service, the Ceph service uses the mark to locate a Ceph service IO corresponding to the IO request, and makes the Ceph service IO out of a Ceph cache queue; when the Ceph service IO carries the mark and returns to the storage side iSCSI service to return the iSCSI client through the storage side iSCSI service, the storage side iSCSI service matches the mark bound by the Ceph service IO with all the recorded marks, if the marks are matched, the return result of the Ceph service IO is released, and the return result is not returned to the iSCSI client, so that IO abnormality is avoided to cause the Ceph service and the iSCSI service to be deadlocked, and the iSCSI service and the Ceph service of a storage system can normally work when IO abnormality occurs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of iSCSI service abnormal IO processing, and particularly relates to an iSCSI service abnormal IO processing method, system, device and storage medium. BACKGROUND

[0002] iSCSI (Internet Small Computer System Interface) is a storage technology based on Internet and SCSI-3 protocol. iSCSI uses TCP / IP 860 and 3260 ports as a channel for exchanging SCSI commands between two computers. Through a high-speed local area network hub, an iSCSI client and a storage node form a storage local area network. The storage node is simulated as a local storage device of the iSCSI client through iSCSI service.

[0003] As shown in the prior art, Figure 1 In a distributed storage system of Ceph and iSCSI framework, the network card of the iSCSI client is connected with the network card of the Ceph storage node through an iSCSI gateway. The network between the iSCSI client and the storage node provides iSCSI service. The iSCSI client simulates the storage node as a local storage device. The iSCSI client waits for IO to be flushed to the storage side disk before performing operations on the opened volume. At present, the iSCSI service on the storage side waits for IO that is timed out and not returned. If part of the IO cannot be returned to the iSCSI client due to abnormal conditions on the storage side, the operation will be stuck, resulting in operation abnormality. SUMMARY

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides an iSCSI service abnormal IO processing method, system, device and storage medium.

[0005] In a first aspect, the present application provides an iSCSI service abnormal IO processing method, comprising:

[0006] The iSCSI client sends an abandon command to the storage-side iSCSI service upon timeout of any IO request. This abandon command includes a flag for the IO request. The storage-side iSCSI service records this flag and sends the abandon command to the Ceph service. The Ceph service uses the flag to locate the Ceph service IO corresponding to the IO request and dequeues the Ceph service IO from the Ceph cache queue. When the Ceph service IO returns to the storage-side iSCSI service carrying the flag, the storage-side iSCSI service matches the flag bound to the Ceph service IO with all its recorded flags. If a match is found, the return result of the Ceph service IO is released without being returned to the iSCSI client, thus preventing IO exceptions from causing the Ceph service and iSCSI service to freeze.

[0007] Furthermore, SCSI commands that generate IO requests in the iSCSI service are identified by the aforementioned tag, with different tags for different IO requests.

[0008] Furthermore, when the storage-side iSCSI service submits an IO request from the iSCSI client to the Ceph service, it simultaneously passes the tag of the IO request to the Ceph service. The Ceph service binds the tag to the Ceph service IO generated for the IO request so that the Ceph service IO corresponding to the timed-out IO request can be located through the tag in the abandon instruction for the timed-out IO request. The tag bound to the Ceph service IO is placed in the Ceph cache queue to realize queue control of Ceph service IO.

[0009] Furthermore, a preset configurable timeout threshold is provided. The iSCSI client counts the return time of each IO request. If the return time of any IO request exceeds the timeout threshold, the iSCSI client sends an abandon command for that IO request to the storage-side iSCSI service.

[0010] Furthermore, if a timeout IO request does not return to the iSCSI client, the iSCSI client reconstructs the IO request and configures a new tag. The new tag is different from the tag of the IO request before reconstruction to prevent the storage-side iSCSI service from releasing the reconstructed IO request.

[0011] Furthermore, the iSCSI client is configured to place I / O requests in a request queue, which controls the sending of I / O requests to the storage-side iSCSI service.

[0012] Further, the storage-side iSCSI service records the tags contained in all the discard instructions it receives by a tag list, and deletes the tag in the tag list that matches the tag of the Ceph service IO when the tag of the Ceph service IO matches a tag in the tag list.

[0013] In a second aspect, the present application provides an iSCSI service abnormal IO processing system, characterized in that comprising: an instruction generation module, which is configured to generate an IO request and a discard instruction;

[0014] A request tag module, which is configured to configure a tag with an identification function for the generated IO request;

[0015] A timeout detection module, which is configured to detect a timeout IO request, and in the case of a timeout IO request, the instruction generation module generates a discard instruction carrying the tag of the timeout IO request and sends the discard instruction to the storage-side iSCSI service;

[0016] A tag recording module, which is configured to record the tag in the discard instruction;

[0017] A Ceph cache queue control module, which is configured to locate the Ceph service IO corresponding to the IO request by the tag in the discard instruction, and make the Ceph service IO de-queue from the Ceph cache queue;

[0018] An IO release module, which is configured to match the tag bound to the Ceph service IO with the recorded tag, and if the tag bound to the Ceph service IO matches the recorded tag, release the return result of the Ceph service IO without returning it to the iSCSI client, and if the tag bound to the Ceph service IO does not match the tag in the tag list, the storage-side iSCSI service transmits the return result of the Ceph service IO to the iSCSI client.

[0019] In a third aspect, the present application provides an iSCSI service abnormal IO processing device, comprising: an iSCSI client, and a storage-side connected with the iSCSI client through an iSCSI service provided by the iSCSI client over a network;

[0020] The iSCSI client comprises at least one first processing unit, a first storage unit, a first network card and a first bus unit, and the first bus unit connects the first processing unit, the first network card and the first storage unit.

[0021] The storage-side comprises at least one second processing unit, a second storage unit, a second network card and a second bus unit, and the second bus unit connects the second processing unit, the second network card and the second storage unit.

[0022] The first storage unit and the second storage unit computer program, the computer program of the first storage unit is executed by the first processing unit, the computer program of the second storage unit is executed by the second processing unit to realize the iSCSI service exception IO processing method.

[0023] In a fourth aspect, the present application provides a storage medium for realizing an iSCSI service exception IO processing method, the storage medium stores a computer program, and the computer program realizes the iSCSI service exception IO processing method when executed by a processor.

[0024] Compared with the prior art, the above technical solution provided by the embodiments of the present application has the following advantages:

[0025] In the present application, the iSCSI client sends a discard instruction to the storage-side iSCSI service according to any IO request timeout, wherein the discard instruction contains a mark of the IO request, the storage-side iSCSI service records the mark and sends the discard instruction to the Ceph service, the Ceph service locates the Ceph service IO corresponding to the IO request by using the mark, and makes the Ceph service IO out of the Ceph cache queue; when the Ceph service IO carrying the mark returns to the storage-side iSCSI service, the storage-side iSCSI service matches the mark bound to the Ceph service IO with all the recorded marks, and if the marks match, the return result of the Ceph service IO is released, and is not returned to the iSCSI client, so as to avoid the Ceph service and the iSCSI service from being deadlocked due to IO exception, and ensure that the iSCSI service and the Ceph service of the storage system can work normally when the IO exception occurs.

[0026] In the present application, when the timeout IO request is not returned to the iSCSI client, the iSCSI client reestablishes the IO request and configures a new mark, and attempts to realize the IO request delayed by the exception IO by re-creating the IO request, so that the new mark of the re-created IO request is different from the mark of the IO request before reestablishment, so as to avoid the reestablished IO request from being released by the storage-side iSCSI service. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0029] Figure 1 A schematic diagram of distributed storage for a ceph-iSCSI architecture;

[0030] Figure 2 A flowchart of an iSCSI service abnormal IO processing method provided by an embodiment of the present application;

[0031] Figure 3 A flowchart of a storage system processing normal IO requests provided by an embodiment of the present application;

[0032] Figure 4 A flowchart of a storage-side iSCSI service managing a discard instruction tag by a tag list provided by an embodiment of the present application;

[0033] Figure 5 A schematic diagram of an iSCSI service abnormal IO processing system provided by an embodiment of the present application;

[0034] Figure 6 A schematic diagram of an iSCSI service abnormal IO processing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0036] It should be noted that, in this document, the term “comprising” or “including” or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without more limitations, the element defined by the statement “including a…” does not exclude the presence of another identical element in the process, method, article or device including the element.

[0037] Embodiment 1

[0038] Referring to Figure 2 The present application provides an iSCSI service abnormal IO processing method, comprising:

[0039] A preset configurable timeout threshold in an iSCSI client.

[0040] Specifically, the timeout threshold is set as a variable of a configurable value, and the user configures the timeout threshold through an interface, such as configuring the value of the timeout threshold as 30 seconds.

[0041] The iSCSI client counts the result return time of each IO request, and detects whether the result return time of each IO request exceeds the timeout threshold.

[0042] In a preferred embodiment, the iSCSI client configures a request queue for controlling the sending of IO requests to the storage-side iSCSI service, the iSCSI client adds the generated IO request to the request queue, uses the request queue to control the sending order of the IO requests to the storage-side iSCSI service, starts counting the result return time of the IO request when the IO request is sent from the iSCSI client, and ends the counting of the result return time when the result of the IO request is returned to the iSCSI client or the result return time exceeds the timeout threshold.

[0043] If the return time of any IO request exceeds the timeout threshold, the iSCSI client sends an abandonment instruction for the IO request to the storage-side iSCSI service. The abandonment instruction contains a mark of the IO request, and the iSCSI client identifies the SCSI command forming the IO request in the iSCSI service through the mark. The marks of different IO requests are different.

[0044] In the implementation process, refer to Figure 3 As shown in the figure, the process of the storage system processing a normal IO request in the iSCSI service abnormal IO processing method disclosed in the present application includes: the iSCSI client sends an IO request and a mark thereof to the storage-side iSCSI service; the storage-side iSCSI service submits the IO request to the Ceph service, and in this process, the storage-side iSCSI service passes the mark of the IO request into the Ceph service; the Ceph service generates a Ceph service IO for the IO request; the Ceph service binds the mark with the Ceph service IO generated for the IO request. The Ceph service controls the processing order of the Ceph service IO through a Ceph cache queue. Specifically, the mark bound with the Ceph service IO is placed in the Ceph cache queue, and the processing order of the corresponding Ceph service IO is controlled through the order of the mark in the Ceph cache queue.

[0045] After the abandonment instruction is sent to the storage-side iSCSI service, the storage-side iSCSI service records the mark and sends the abandonment instruction to the Ceph service.

[0046] In the implementation, the storage-side iSCSI service records all the tags contained in the received discard instructions by a tag list; specifically, refer to Figure 4 As shown in the figure, the process of the storage-side iSCSI service managing the tags in the discard instruction by a tag list includes: the storage-side iSCSI service initializes a tag list, the storage-side iSCSI service parses the discard instruction to obtain the tags in the discard instruction, the storage-side iSCSI service adds the tags to the tag list by an append instruction, and the storage-side iSCSI service detects whether the tag bound to the received Ceph service IO matches the tags in the tag list, and deletes the tags in the tag list that match the tag bound to the Ceph service IO by a remove instruction when a match is found.

[0047] When the discard instruction for the timeout IO is generated and the tag of the timeout IO request is transmitted to the Ceph service, the Ceph service locates the Ceph service IO corresponding to the timeout IO request by using the tag, and makes the Ceph service IO de-queue from the Ceph cache queue.

[0048] In the implementation, after the Ceph service receives the discard instruction transmitted by the storage-side iSCSI, the Ceph service locates the tag of the timeout IO request in the Ceph cache queue by using the tag in the discard instruction, and makes the corresponding tag in the Ceph cache queue de-queue to realize the de-queue of the Ceph service IO from the Ceph cache queue.

[0049] When the Ceph service IO carrying the tag is returned to the storage-side iSCSI service to be returned to the client via the storage-side iSCSI service, the storage-side iSCSI service matches the tag bound to the Ceph service IO with all the recorded tags (tags in the tag list), if the tag of the Ceph service IO matches the tags in the tag list, the storage-side iSCSI service releases the return result of the Ceph service IO without returning it to the iSCSI client, and if the tag bound to the Ceph service IO does not match the tags in the tag list, the storage-side iSCSI service transmits the return result of the Ceph service IO to the iSCSI client. In this way, the Ceph service and the iSCSI service are prevented from being stuck due to IO exceptions.

[0050] When the timeout IO request is not returned to the iSCSI client, the iSCSI client rebuilds the IO request and configures a new tag, which is different from the tag of the IO request before rebuilding, so that the storage side treats the rebuilt IO request as a new IO request and avoids releasing the rebuilt IO request by the storage-side iSCSI service.

[0051] Embodiment 2

[0052] Refer to Figure 5As shown, the embodiment of the present application provides an iSCSI service abnormal IO processing system, comprising:

[0053] The iSCSI client is configured with an instruction generation module, a request marking module, a timeout detection module and a request reconstruction module, wherein:

[0054] The instruction generation module is configured to generate SCSI instructions to form IO requests and an abort instruction, the abort instruction (ABORT) being a SCSI instruction used to abort SCSI instructions.

[0055] The request marking module is configured to configure the generated IO requests with marks having an identification function.

[0056] The timeout detection module detects IO requests that have timed out, and in the case of IO requests that have timed out, controls the instruction generation module to generate an abort instruction carrying the mark of the IO request that has timed out and sends the abort instruction to the storage-side iSCSI service.

[0057] The request reconstruction module is configured to count IO requests that have timed out, and uses the instruction generation module to recreate the IO requests that have timed out, and uses the request marking module to configure the recreated IO requests with new marks.

[0058] The iSCSI service abnormal IO processing system further comprises a mark recording module, a mark binding module, a Ceph cache queue module, a Ceph cache queue control module and an IO release module configured on the storage side, wherein:

[0059] The mark recording module is configured to record the marks in the abort instructions received by the storage-side iSCSI service, and the recorded marks are used to identify the Ceph service IOs that have timed out when the Ceph service returns the Ceph service IO results to the storage-side iSCSI service.

[0060] The IO requests generated by the instruction generation module are sent to the storage-side iSCSI service, and the storage-side iSCSI service submits the IO requests to the Ceph service, in the process, the storage-side iSCSI service passes the marks of the IO requests to the Ceph service, and the mark binding module binds the marks with the Ceph service IOs generated by the Ceph service for the IO requests.

[0061] The Ceph cache queue module constructs a Ceph cache queue, and places the marks bound with the Ceph service IOs in the Ceph cache queue, and controls the processing order of the corresponding Ceph service IOs by the order of the marks in the Ceph cache queue to achieve queue control of the Ceph service IOs.

[0062] The cep cache queue control module uses the mark in the discard instruction to locate the corresponding timeout IO request cep service IO, and makes the cep service IO de-queue from the cep cache queue.

[0063] The IO release module matches the mark bound by the cep service IO with the recorded mark, and if the mark bound by the cep service IO matches the recorded mark, it means that the cep service IO is timeout, and the return result of the cep service IO is released and not returned to the iSCSI client, and if the mark bound by the cep service IO does not match the mark in the mark list, the storage side iSCSI service transmits the return result of the cep service IO to the iSCSI client.

[0064] For the storage side, since the IO request reconfigured by the reconstruction request module is configured with a new mark, the storage side processes the reconstructed IO request as a new IO request.

[0065] Embodiment 3

[0066] The embodiment of the present application provides an iSCSI service abnormal IO processing device, which comprises an iSCSI client and a storage side connected with an iSCSI service provided by the iSCSI client through a network.

[0067] The iSCSI client comprises at least one first processing unit, a first storage unit, a first network card and a first bus unit, and the first bus unit is connected with the first processing unit, the first network card and the first storage unit.

[0068] The storage side comprises at least one second processing unit, a second storage unit, a second network card and a second bus unit, and the second bus unit is connected with the second processing unit, the second network card and the second storage unit.

[0069] The first storage unit and the second storage unit are computer programs, the computer program of the first storage unit is executed by the first processing unit, and the computer program of the second storage unit is executed by the second processing unit, so as to realize the iSCSI service abnormal IO processing method in the embodiment 1. The specific process of the iSCSI service abnormal IO processing method is not repeated here.

[0070] Embodiment 4

[0071] The embodiment of the present application provides a storage medium for realizing an iSCSI service abnormal IO processing method, and the storage medium stores a computer program, and the computer program is executed by a processor to realize the iSCSI service abnormal IO processing method in the embodiment 1. The specific process of the iSCSI service abnormal IO processing method is not repeated here.

[0072] In the embodiments of the present application, it should be understood that the disclosed structure and method can be implemented in other manners. For example, the described embodiments of the structure are merely schematic, and the division of the units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, structures or units, and can be in electrical, mechanical or other forms.

[0073] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0074] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in a unit. The integrated unit can be implemented in a form of hardware, or in a form of software functional unit.

[0075] The above descriptions are merely specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for handling abnormal I / O in iSCSI services, characterized in that, include: For SCSI commands that form IO requests in iSCSI services, they are identified by tags, and the tags are different for different IO requests; When the storage-side iSCSI service submits an IO request from an iSCSI client to the ceph service, it simultaneously passes the tag of the IO request to the ceph service. The ceph service binds the tag to the ceph service IO generated by the ceph service for the IO request so that the ceph service IO corresponding to the timed-out IO request can be located by the tag in the abandon instruction for the timed-out IO request. The tag bound to the ceph service IO is placed in the ceph cache queue to realize queue control of the ceph service IO. The iSCSI client sends an abandon command to the storage-side iSCSI service upon timeout of any IO request. This abandon command includes a flag for the IO request. The storage-side iSCSI service records this flag and sends the abandon command to the Ceph service. The Ceph service uses the flag to locate the Ceph service IO corresponding to the IO request and dequeues the Ceph service IO from the Ceph cache queue. When the Ceph service IO returns to the storage-side iSCSI service carrying the flag, the storage-side iSCSI service matches the flag bound to the Ceph service IO with all its recorded flags. If a match is found, the return result of the Ceph service IO is released without being returned to the iSCSI client, thus preventing IO exceptions from causing the Ceph service and iSCSI service to freeze.

2. The iSCSI service abnormal I / O handling method according to claim 1, characterized in that, A preset configurable timeout threshold is set. The iSCSI client counts the return time of each IO request. If the return time of any IO request exceeds the timeout threshold, the iSCSI client sends an abandon command for that IO request to the storage-side iSCSI service.

3. The iSCSI service abnormal I / O handling method according to claim 1, characterized in that, If a timeout IO request does not return to the iSCSI client, the iSCSI client reconstructs the IO request and configures a new tag. The new tag is different from the tag of the IO request before reconstruction to prevent the storage-side iSCSI service from releasing the reconstructed IO request.

4. The iSCSI service abnormal I / O handling method according to claim 1, characterized in that, The iSCSI client is configured to place I / O requests in a request queue, which controls the sending of I / O requests to the storage-side iSCSI service.

5. The iSCSI service abnormal I / O handling method according to claim 1, characterized in that, The storage-side iSCSI service records all the tags contained in the abandoned instructions it receives through a tag list. When a tag of the ceph service IO matches a tag in the tag list, the tag that matches the tag of the ceph service IO is deleted from the tag list.

6. An iSCSI service exception I / O handling system, characterized in that, include: The iSCSI client is configured with a command generation module, a request marking module, and a timeout detection module, among which... The instruction generation module is used to generate I / O requests and abort instructions; The request tagging module is used to configure tags with identification functions for the generated IO requests; The timeout detection module detects timed-out IO requests, and when an IO request times out, the instruction generation module generates an abandon instruction carrying a flag of the timed-out IO request and sends the abandon instruction to the storage-side iSCSI service. The storage side is configured with a tag recording module, a tag binding module, a Ceph cache queue module, a Ceph cache queue control module, and an IO release module; among them, The marker recording module is used to record the markers in the abandonment instruction; The tag binding module binds the tag to the Ceph service IO generated by the Ceph service in response to the IO request; The ceph cache queue module constructs a ceph cache queue, places the tags bound to ceph service IO into the ceph cache queue, and controls the processing order of the corresponding ceph service IO by the order of the tags in the ceph cache queue to achieve queue control of ceph service IO. The ceph cache queue control module uses the flag in the abandon instruction to locate the ceph service IO corresponding to the IO request and dequeues the ceph service IO from the ceph cache queue. The IO release module matches the tag bound to the ceph service IO with the tag of the record. If the tag bound to the ceph service IO matches the tag of the record, the return result of the ceph service IO is released and not returned to the iSCSI client. If the tag bound to the ceph service IO does not match the tag in the tag list, the storage-side iSCSI service will pass the return result of the ceph service IO to the iSCSI client.

7. An iSCSI service exception I / O handling device, characterized in that, include: iSCSI client, storage side connected to iSCSI client via iSCSI service provided over network; The iSCSI client includes at least one first processing unit, a first storage unit, a first network interface card (NIC), and a first bus unit, wherein the first bus unit is connected to the first processing unit, the first NIC, and the first storage unit. The storage side includes at least one second processing unit, a second storage unit, a second network interface card (NIC), and a second bus unit, wherein the second bus unit is connected to the second processing unit, the second NIC, and the second storage unit; The computer programs for the first storage unit and the second storage unit, wherein the computer program for the first storage unit is executed by the first processing unit, and the computer program for the second storage unit is executed by the second processing unit, implement the iSCSI service abnormal I / O handling method as described in any one of claims 1-5.

8. A storage medium for implementing an iSCSI service exception I / O handling method, the storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the iSCSI service abnormal I / O handling method as described in any one of claims 1-5.

Images