Request processing method and device based on class distributed system, equipment and medium

Abstract

The invention relates to the technical field of storage, and discloses a request processing method, device and equipment based on a class distributed system and a medium, the class distributed system comprises a shared network device, a plurality of storage controllers and at least one storage node, each storage controller can perform data operation in each storage node, and each storage node is connected with the shared network device. The method is applied to shared network equipment. The method comprises the following steps: receiving a first data operation request sent by a host, wherein the first data operation request comprises data information of target data to be operated and a target storage node for storing the target data; splitting the first data operation request into a plurality of sub-data operation requests under the condition of determining that the first data operation request meets a splitting condition based on the data information of the target data; and issuing the sub-data operation request to a plurality of storage controllers to enable the plurality of storage controllers to perform data operation in parallel at the target storage node. The host stability can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of storage technology, and in particular to a request processing method, apparatus, device and medium based on a distributed system. Background Art

[0002] A multi-node collaborative quasi-distributed system is a system composed of multiple independent computer nodes (such as physical servers, virtual machines, containers, etc.). In a quasi-distributed system, nodes communicate and collaborate through the network to jointly complete one or more tasks. These nodes can be relatively independent logically, but can collaborate with each other when performing specific tasks.

[0003] At present, the quasi-distributed systems of large-scale data centers usually include multiple hosts and multiple storage controllers. The storage controller is used to perform data operations in the storage node. By installing multipath software in each host, the load balancing of the storage controller can be achieved. However, since the multipath software has compatibility issues in different types of hosts and is highly dependent on the host operating system version, the host often crashes due to multipath software problems and is not stable enough. Summary of the invention

[0004] In view of this, the present disclosure provides a request processing method, a request processing device, an electronic device and a computer-readable storage medium based on a distributed system, which can improve the stability of the host.

[0005] In a first aspect, the present disclosure provides a request processing method, wherein the quasi-distributed system includes a shared network device, a plurality of storage controllers and at least one storage node, each of the storage controllers can perform data operations in each of the storage nodes, and the method is applied to the shared network device; the method includes:

[0006] receiving a first data operation request sent by a host, wherein the first data operation request includes data information of target data to be operated and a target storage node for storing the target data;

[0007] When it is determined based on the data information of the target data that the first data operation request satisfies a splitting condition, splitting the first data operation request into a plurality of sub-data operation requests;

[0008] The sub-data operation request is sent to multiple storage controllers, so that the multiple storage controllers perform data operations on the target storage node in parallel.

[0009] In a second aspect, the present disclosure provides a request processing device based on a quasi-distributed system, wherein the quasi-distributed system includes a shared network device, a plurality of storage controllers and at least one storage node, each of the storage controllers can perform data operations in each of the storage nodes, and the method is applied to the shared network device; the device includes:

[0010] A request receiving module, configured to receive a first data operation request sent by a host, wherein the first data operation request includes data information of target data to be operated and a target storage node for storing the target data;

[0011] a request splitting module, configured to split the first data operation request into a plurality of sub-data operation requests when it is determined that the first data operation request satisfies a splitting condition based on the data information of the target data;

[0012] The request sending module is used to send the sub-data operation request to multiple storage controllers, so that the multiple storage controllers perform data operations in parallel on the target storage node.

[0013] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the above method by executing the computer instructions.

[0014] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the above method.

[0015] In the technical solutions of some embodiments of the present disclosure, the data operation request sent by the host that meets the splitting condition is split in the shared network device, and the sub-data operation request obtained by the split is sent to multiple storage controllers, and the multiple storage controllers perform data operations in parallel in the target storage node. In this way, under the premise of achieving load balancing of the storage controller, it is not necessary to install multipath software in the host, thereby avoiding the instability problem caused by the multipath software to the host. In summary, the solution of the present disclosure can achieve the effect of improving the stability of the host. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 is a schematic diagram of modules of a distributed system provided by an embodiment of the present disclosure;

[0018] Figure 2 is a flowchart of a request processing method provided by some embodiments of the present disclosure;

[0019] Figure 3 is a module schematic diagram of a request processing device provided in some embodiments of the present disclosure;

[0020] Figure 4 It is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0023] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "some embodiments" or "the embodiment" should be understood as "at least some embodiments". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0024] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0025] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0026] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scopes of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0027] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0029] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] Combined with reference Figure 1 , which is a module diagram of a distributed system-like system provided for one embodiment of the present disclosure. Figure 1 In the embodiment, the quasi-distributed system includes multiple hosts, at least one network sharing device, multiple storage controllers and at least one storage node. The network sharing device may include but is not limited to routers, switches, server gateways, etc. The host may send a data operation request to the storage controller through the network sharing device. The storage controller performs data operations in the storage node based on the received data operation request. Specifically, the data operation request may include but is not limited to data read, write, modify, delete and other requests. Based on the data read request, the storage controller may perform data read, write, modify, delete and other operations in the storage node.

[0031] In this embodiment, the network sharing device may include a first connection component and a second connection component. The first connection component is used to establish a connection with the host, receive a data operation request sent by the host, and return a data operation response returned by the storage controller to the host. The second connection component is used to connect to the storage controller, send the data operation request sent by the host to the storage controller, and receive a data operation response returned by the storage controller (such as whether the data operation is successful, the data read from the storage node, etc.). The first connection component and the second connection component can be two program code modules running in the network sharing device.

[0032] The host can establish a many-to-many connection relationship with the storage controller through a network sharing device. In simple terms, for any host, the host can be connected to multiple storage controllers through a network sharing device, and for any storage controller, the storage controller can be connected to multiple hosts through a network sharing device. For example Figure 1 In the embodiment, host 1 can be connected to storage controllers 1-k respectively through network sharing devices. Storage controller 1 can be connected to hosts 1-n respectively through network sharing devices.

[0033] Similarly, there can be a many-to-many connection relationship between storage controllers and storage nodes. That is, for any storage controller, the storage controller can be connected to multiple storage nodes, and for any storage node, the storage node can be connected to multiple storage controllers. Based on this many-to-many connection relationship, each storage controller can perform data operations in each storage node, and multiple storage controllers can perform data operations in the same storage node. For example, storage controller 1, storage controller 2, and storage controller 3 can perform data operations in storage node 1 at the same time, and storage controller 1 can perform data operations in storage nodes 1, 2, and 3.

[0034] based on Figure 1 The architecture shown in the present disclosure provides a request processing method, which can solve the problem of insufficient host stability caused by multipath software in some technologies. The request processing method can be applied to Figure 1 See also Figure 2 , which is a flowchart of a request processing method provided in some embodiments of the present disclosure. Figure 2 The request processing method includes the following steps:

[0035] Step S201: receiving a first data operation request sent by a host, wherein the first data operation request includes data information of target data to be operated and a target storage node for storing the target data.

[0036] Specifically, the data information may include, but is not limited to, the attribute information of the target data and the storage information of the target data in the target storage node. The attribute information may include, but is not limited to, the data volume of the target data, the data encoding method, etc. The storage information may include, but is not limited to, the storage address and storage method of the target data in the target storage node. It should be noted that when the first data operation request is a data write request, the first data operation request may include, in addition to the data information of the target data and the target storage node, the entity data of the target data to be written to the storage node.

[0037] Shared network devices via Figure 1 After receiving the first data operation request, the first connection component shown can obtain data information and a target storage node by parsing the first data operation request.

[0038] Step S202: when it is determined based on the data information of the target data that the first data operation request satisfies the splitting condition, the first data operation request is split into a plurality of sub-data operation requests.

[0039] In this embodiment, the splitting condition may include the following two conditions:

[0040] The data volume of the target data exceeds a first data volume threshold;

[0041] The storage address of the target data crosses the boundary.

[0042] When the target data meets any of the above conditions, the first data operation request can be split into multiple sub-data operation requests, each sub-data operation request is used to operate part of the target data, and different sub-data operation requests are used to operate different parts of the target data.

[0043] For example, assuming that the first data volume threshold is 100 MB, when the data volume of the target data exceeds 100 MB, the first data operation request may be split into five sub-data operation requests, each of which is used to operate 20 MB of target data.

[0044] For another example, assuming that the storage address of the target data is from address a to address b. If there is a boundary address between address a and address b, the first data operation request can be split into two sub-data operation requests, the first sub-data operation request is used to operate the target data between address a and the boundary address, and the second sub-data operation request is used to operate the target data between the boundary address and address b.

[0045] Furthermore, when the target data satisfies both of the above splitting conditions, the first data operation request can be split for the first time according to the boundary address to obtain multiple sub-data operation requests. For any sub-data operation request, if the amount of data in the sub-data operation request exceeds the first data amount threshold, the sub-data operation request can be split for the second time.

[0046] When splitting the data operation request, the data volume in each sub-data operation request may be within a preset data volume range, such as 15MB to 30MB. The first data volume threshold may be greater than the maximum value within the data volume range. For example, the first data volume threshold may be 150MB.

[0047] For different sub-data operation requests, the request identifier of the sub-data operation request, the storage address of the sub-data, etc. may be different.

[0048] It should be noted that the above is only an exemplary description of the splitting method of the data operation request. In practical applications, the splitting logic can be set according to actual needs, and the present disclosure does not limit this.

[0049] Step S203: Send the sub-data operation request to multiple storage controllers, so that the multiple storage controllers perform data operations in parallel on the target storage node.

[0050] Specifically, when the number of sub-data operation requests is less than or equal to the number of storage controllers, different sub-data operation requests can be sent to different storage controllers; when the number of sub-data operation requests is greater than the number of storage controllers, the sub-data operation requests can be distributed as evenly as possible among the storage controllers. For example, assuming that the number of storage controllers is 5 and the number of sub-data operation requests is 11, 3 sub-data operation requests can be assigned to one of the storage controllers, and 2 sub-data operation requests can be assigned to each of the other storage controllers. Each storage controller can perform data operations in the target storage node in parallel based on the received sub-data operation requests. In this way, on the one hand, the load balancing of the storage controllers can be guaranteed to prevent the problem of excessive request processing pressure on some storage controllers and too low request processing pressure on some storage controllers; on the other hand, multiple storage controllers can perform data operations in the storage nodes in parallel, which can improve data processing efficiency.

[0051] In summary, in the technical solutions of some embodiments of the present disclosure, the data operation request sent by the host that meets the splitting condition is split in the shared network device, and the sub-data operation request obtained by the split is sent to multiple storage controllers, and the multiple storage controllers perform data operations in parallel in the target storage node. In this way, under the premise of achieving load balancing of the storage controller, it is not necessary to install multipath software in the host, thereby avoiding the instability problem caused by the multipath software to the host. In summary, the solution of the present disclosure can achieve the effect of improving the stability of the host.

[0052] In some embodiments, when the first data operation request does not meet the split condition, the method of the present disclosure may further include:

[0053] receiving at least one second data operation request sent by the host;

[0054] When the following conditions are met, the first data operation request and at least one second data operation request are aggregated into a total data operation request, and the total data operation request is sent to the storage controller:

[0055] 11) The data amounts in the first data operation request and each of the second data operation requests are smaller than a second data amount threshold, and the second data amount threshold is smaller than a minimum data amount allowed in a single sub-data operation request;

[0056] 12) The storage addresses in the first data operation request and each second data operation request point to the same storage area or to consecutive storage areas.

[0057] Specifically, the relationship between the first data volume threshold, the data volume in a single sub-data operation request, and the second data volume threshold may be as follows:

[0058] The first data volume threshold>the maximum data volume allowed in a single sub-data operation request;

[0059] The second data volume threshold is less than the minimum data volume allowed in a single sub-data operation request.

[0060] For example, assuming that the data volume range of a single sub-data operation request is preset to be 15MB to 30MB, the first data volume threshold may be 50MB, and the second data volume threshold may be 2MB. When the data volume in the first data operation request is greater than 50MB, the first data operation request is split into multiple sub-data operation requests; when the data volume in the first data operation request is between 2MB and 50MB, the first data operation request is not split or aggregated; when the data volume in the first data operation request is less than 2MB, the first data operation request is aggregated with one or more second data operation requests that meet the above conditions 11) and 12) into a total data operation request.

[0061] By means of request aggregation, multiple requests can be processed simultaneously in the same time period, thereby improving the request processing speed and request response speed of the shared network device. For example, assume that the second data volume threshold is 2MB. Host 1 sends a first data operation request 1 with a data volume of 1MB, and host 2 sends a second data operation request 2 with a data volume of 0.5MB and a second data operation request 3 with a data volume of 1.5MB. Under normal circumstances, data operation requests 1, 2, and 3 can be arranged in the request queue of the shared network device in the order in which the requests are sent, and processed in sequence by the shared network device. That is, the shared network device can process data operation request 2 after processing data operation request 1 and returning a response to data operation request 1. After processing data operation request 2 and returning a response to data operation request 2, data operation request 3 is processed. It can be seen from this that data operation request 3 can only be processed after data operation requests 1 and 2 are processed, so the response time of data operation request 3 may be extended. In the embodiment of the present disclosure, when the storage addresses in the data operation requests 1, 2, and 3 point to the same storage area, the data operation requests 1, 2, and 3 are aggregated into a total data operation request by request aggregation, and the total data operation request can be sent to the storage controller at one time. After the storage controller completes the data operations corresponding to the data operation requests 1, 2, and 3 at the same time, it returns a response, so that the response time of the data operation request 3 can be greatly shortened.

[0062] Furthermore, when sending the total data operation request to the storage controller, if the amount of data in the total data operation request exceeds the first data amount threshold, the total data operation request can be split into multiple sub-data operation requests, and the split sub-data operation requests are sent to multiple storage controllers, so that multiple storage controllers can perform data operations in parallel on the target storage node. In this way, on the one hand, the processing efficiency of the storage controller in processing data operation requests can be greatly improved, and the delay of request response can be reduced; on the other hand, the load balancing between storage controllers can be guaranteed.

[0063] It should be noted that, in the above-mentioned embodiment, the number of sub-data operation requests obtained by splitting the total data operation request may be different from the total number of the first data operation request and the second data operation request participating in the aggregation. For example, assuming that the first data volume is at a threshold value of 30MB, the preset data volume range of a single sub-data operation request is 15MB to 20MB. After aggregating the first data operation request with a data volume of 1MB and 100 second data operation requests with a data volume of 0.5MB, the data volume of the total data operation request obtained is 51MB. According to the data volume range of the sub-data operation request of 15MB to 20MB, the total data operation request can be split into 2 sub-data operation requests. Through this comparison, it can be seen that before aggregation, the number of the first data operation request and the second data operation request is 101, and after the request aggregation and splitting, the number of sub-data operation requests obtained is 2.

[0064] Further, based on the above comparison, assuming that the number of storage controllers is 10, and these 10 network sharing devices can perform data processing in parallel, then when the network sharing device processes the first data operation request and the second data operation request in sequence, each storage controller needs to process at least 10 requests on average to complete the processing of these 101 data operation requests. In the method disclosed in the present invention, only two storage controllers are required to process once to complete the processing of these 101 data operation requests. It can be seen that the present invention can greatly improve the request processing efficiency and shorten the request response time based on request aggregation and splitting.

[0065] In some embodiments, when a data operation request is split into multiple sub-data operation requests, a first association relationship between the data operation request and each sub-data operation request may be created; and when multiple data operation requests are aggregated into a total data operation request, a second association relationship between the total data operation request and each data operation request may be created. Specifically, the following three situations may be described.

[0066] 1) Only the data operation request is split into multiple sub-data requests. In this case, a first association relationship between the data operation request and each of the split sub-data requests is established. For example, assuming that sub-data operation requests A1, A2, and A3 are split from data operation request A, and sub-data operation requests B1 and B2 are split from data operation request B, then the corresponding first association relationship can be as follows:

[0067] Data operation request A<>sub-data operation request A1

[0068] Data operation request A<>sub-data operation request A2

[0069] Data operation request A<>sub-data operation request A3

[0070] Data operation request B<>sub-data operation request B1

[0071] Data operation request B<>sub-data operation request B2

[0072] 2) Aggregate multiple data operation requests into one total data operation request. In this case, establish a second association relationship between the total data operation request and each data operation request. For example, assuming that data operation requests C1, C2, and C3 are aggregated into a total data operation request C, and data operation requests D1 and D2 are aggregated into a total data operation request D, then the corresponding second association relationship can be as follows:

[0073] Data operation request C1<>Total data operation request C

[0074] Data operation request C2<>Total data operation request C

[0075] Data operation request C3<>Total data operation request C

[0076] Data operation request D1<>Total data operation request D

[0077] Data operation request D2<>Total data operation request D

[0078] 3) First, multiple data operation requests are aggregated into a total data operation request, and then the total data operation request is split into multiple sub-data operation requests. In this case, a second association relationship between the total data operation request and each data operation request can be established first, and then a first association relationship between the total data operation request and each sub-data request obtained by splitting can be established. For example, assuming that data operation requests E1, E2, E3, and E4 are aggregated into a total data operation request E, and then the total data operation request E is split into sub-data operation requests E5 and E6, then the corresponding second association relationship and first association relationship can be as follows:

[0079] Data operation request E1<>Total data operation request E

[0080] Data operation request E2<>Total data operation request E

[0081] Data operation request E3<>Total data operation request E

[0082] Data operation request E4<>Total data operation request E

[0083] Total data operation request E<>sub-data operation request E5

[0084] Total data operation request E<>sub-data operation request E6

[0085] After sending the sub-data operation request or the total data operation request to the storage controller, the data operation response returned by the storage controller can be received, and the data operation response can be split or aggregated according to the first association relationship and / or the second association relationship to obtain a response returned to the host. Specifically, for any data operation response M returned by the storage controller, the aggregation or splitting of the response can be completed according to the following steps:

[0086] 1) If the data operation request M1 corresponding to the data operation response M is in the first association relationship, it means that the data operation request M1 is a sub-data operation request. At this time, the data operation request M11 associated with the sub-data operation request M1 can be searched in the first association relationship, and it is determined whether the responses of all the sub-data operation requests associated with the data operation request M11 have been received. If so, the responses of all the sub-data operation requests are aggregated to obtain the response of the data operation request M11; if not, continue to wait.

[0087] 2) After obtaining the data response of the data operation request M11, you can continue to search whether the data operation request M11 exists in the second association relationship. If not, the response of the aggregated data operation request M11 can be directly sent to the host. If it exists, it means that the data operation request M11 is a total data operation request. At this time, you can search for multiple data operation requests associated with the total data operation request M11 in the second association relationship, and split the response of the aggregated data operation request M11 into the responses of the multiple data operation requests, and return the responses of each data operation request to the host in turn.

[0088] 3) If the data operation request M1 corresponding to the data operation response M is in the second association relationship, it means that the data operation request M1 is the total data operation request. At this time, multiple data operation requests associated with the total data operation request M1 can be searched in the second association relationship, and the data operation response M is split into responses to the multiple data operation requests, and the responses of the various data operation requests are returned to the host in sequence.

[0089] By creating the first association relationship and the second association relationship, the response returned by the storage controller and the request sent by the host can be associated and mapped to prevent confusion between the response and the request.

[0090] In some embodiments, after receiving a data operation request sent by the host, if a communication anomaly with the host is detected, the corresponding data operation request sent by the host but not yet sent to the storage controller can be released, and the response returned by the storage controller can be discarded for the data operation request sent by the host and sent to the storage controller. In this way, the storage controller does not need to sense whether the communication with the host is faulty, thereby simplifying the data processing logic on the storage controller side.

[0091] In some embodiments, after receiving a data operation request sent by the host, if a communication anomaly with one of the target storage controllers is detected, the data operation request sent to the target storage controller is forwarded to other storage controllers other than the target storage controller. In this way, the host does not need to sense whether the communication with the storage controller is faulty, thereby simplifying the data processing logic on the host side.

[0092] In short, the present disclosure can isolate the host from the storage controller by setting the above control logic in the network sharing device. That is, the network sharing device implements the relevant fault processing logic, so that the host only needs to send the data request, and does not need to pay attention to the processing logic when the communication with the storage controller fails, and the storage controller only needs to receive the data operation request, perform data operation in the storage node according to the data operation request and return the data response, and does not need to pay attention to the processing logic when the communication with the host fails. In this way, the processing logic of the host and the storage controller can be greatly simplified. For example, in some technologies, when the storage controller fails, the network sharing device directly returns a response of data operation failure to the host. On the host side, it is necessary to set the relevant fault processing logic, and the host re-initiates the data operation request to the storage controller that has not failed through the network sharing device based on the response of the data operation failure. In this way, on the one hand, the processing logic on the host side will become more complicated, and on the other hand, the efficiency of data operation is greatly reduced. However, in the scheme of the present disclosure, after detecting that the target storage controller has failed, the network sharing device can directly send the target data operation request sent to the target storage controller to other non-faulty storage controllers. After other storage controllers that are not faulty complete processing the target data operation request, the network sharing device can return the response to the target data operation request to the host. In this way, the host is not aware of the fault of the network sharing device, thereby achieving fault isolation between the host and the storage controller, simplifying the processing logic on the host side, and ensuring business continuity.

[0093] To sum up, in the scheme disclosed herein, on the one hand, the processing efficiency of data operation requests can be greatly improved by splitting or aggregating the requests sent by the host through a network sharing device. On the other hand, it avoids the installation of multipath software in the host, thereby realizing the decoupling of the distribution of data operation requests from the host operating system, greatly improving the stability of the host.

[0094] Combined with reference Figure 1 In some embodiments, the method of the present disclosure can be completed based on the following steps:

[0095] 1) Hosts, network sharing devices, storage controllers, and storage nodes can discover each other through device discovery and protocols such as LLDP and Discovery.

[0096] 2) The host and the network sharing device establish a connection, exchange connection properties, and create a queue and data buffer for exchanging data between the host and the network sharing device.

[0097] 3) The network sharing device and the storage controller establish a connection, exchange connection properties, and at the same time establish a queue and a data buffer for exchange data between the network sharing device and the storage controller.

[0098] 4) The network sharing device applies for resources for splitting and aggregating the data operation request based on the connection attributes and the connection data, for example, storage resources for storing the first association relationship and the association relationship.

[0099] 5) The network sharing device processes the data operation request sent by the host based on the request processing method disclosed in the present invention.

[0100] 6) After the connection is established, each link maintains the link status through port status, heartbeat or timeout strategy. Once a link failure is found, the network sharing device can achieve rapid link switching.

[0101] Corresponding to the above method, the present disclosure also provides a request processing device based on a distributed system. Figure 3 , which is a module diagram of a request processing device provided in some embodiments of the present disclosure.

[0102] Figure 3 In the example, the request processing device includes:

[0103] The request receiving module 301 is used to receive a first data operation request sent by a host, where the first data operation request includes data information of target data to be operated and a target storage node for storing the target data;

[0104] A request splitting module 302 is used to split the first data operation request into multiple sub-data operation requests when it is determined that the first data operation request meets the splitting condition based on the data information of the target data;

[0105] The request sending module 303 is used to send the sub-data operation request to multiple storage controllers, so that the multiple storage controllers perform data operations in parallel on the target storage node.

[0106] In some embodiments, the data information of the target data includes the data volume and storage address of the target data; the splitting condition in the request splitting module 302 includes at least one of the following conditions:

[0107] The data volume of the target data exceeds a first data volume threshold;

[0108] The storage address of the target data crosses the boundary.

[0109] In some embodiments, the request processing device further includes a request aggregation module 304. When the first data operation request does not meet the splitting condition, the request aggregation module 304 is used to:

[0110] receiving at least one second data operation request sent by the host;

[0111] When the following conditions are met, the first data operation request and at least one second data operation request are aggregated into a total data operation request, and the total data operation request is sent to the storage controller:

[0112] The data amounts in the first data operation request and each of the second data operation requests are both smaller than a second data amount threshold, and the second data amount threshold is smaller than a minimum data amount allowed in a single sub-data operation request;

[0113] The storage addresses in the first data operation request and each second data operation request point to the same storage area or to continuous storage areas.

[0114] In some embodiments, the request aggregation module 304 is specifically configured to:

[0115] If the amount of data in the total data operation request exceeds a first data amount threshold, splitting the total data operation request into multiple sub-data operation requests;

[0116] The split sub-data operation requests are sent to multiple storage controllers, so that the multiple storage controllers perform data operations in parallel on the target storage node.

[0117] In some embodiments, the request processing apparatus further includes a request association module 305, and the request association module 305 is used to:

[0118] In the case of splitting the data operation request into multiple sub-data operation requests, creating a first association relationship between the data operation request and each sub-data operation request; and

[0119] In the case of aggregating multiple data operation requests into a total data operation request, a second association relationship between the total data operation request and each data operation request is created.

[0120] In some embodiments, after sending the sub-data operation request or the total data operation request to the storage controller, the request sending module 303 is further used to:

[0121] Receive a data operation response returned by the storage controller;

[0122] The data operation response is split or aggregated according to the first association relationship and / or the second association relationship to obtain a response returned to the host.

[0123] In some embodiments, after receiving the data operation request sent by the host, the request receiving module 301 is further used to:

[0124] When a communication anomaly with the host is detected, the corresponding data operation request sent by the host but not yet sent to the storage controller is released, and the response returned by the storage controller is discarded for the data operation request sent by the host and sent to the storage controller;

[0125] When a communication anomaly with one of the target storage controllers is detected, the data operation request sent to the target storage controller is forwarded to other storage controllers other than the target storage controller.

[0126] The request processing device based on a quasi-distributed system in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0127] The data device disclosed in the present invention splits the data operation request that meets the splitting condition sent by the host in the shared network device, and sends the split sub-data operation request to multiple storage controllers, and the multiple storage controllers perform data operations in parallel in the target storage node. In this way, under the premise of achieving load balancing of the storage controller, it is not necessary to install multipath software in the host, thereby avoiding the instability problem caused by the multipath software to the host. In summary, the solution disclosed in the present invention can achieve the effect of improving the stability of the host.

[0128] The present disclosure also provides an electronic device having the above Figure 3 A request processing device based on a distributed system is shown.

[0129] Combined with reference Figure 4 , is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. Figure 4As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0130] The processor 10 may be a first PCIe device, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0131] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0132] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0134] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0135] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0136] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0137] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A request processing method based on a distributed system, characterized in that: The quasi-distributed system comprises a shared network device, a plurality of storage controllers and at least one storage node, each of the storage controllers can perform data operations in each of the storage nodes, and the method is applied to the shared network device; the method comprises: receiving a first data operation request sent by a host, wherein the first data operation request includes data information of target data to be operated and a target storage node for storing the target data; In a case where it is determined based on the data information of the target data that the first data operation request satisfies a splitting condition, splitting the first data operation request into a plurality of sub-data operation requests; The sub-data operation request is sent to multiple storage controllers, so that the multiple storage controllers perform data operations on the target storage node in parallel.

2. The method according to claim 1, characterized in that The data information of the target data includes the data volume and storage address of the target data; the splitting condition includes at least one of the following conditions: The data volume of the target data exceeds a first data volume threshold; The storage address of the target data crosses the boundary.

3. The method according to claim 2, characterized in that In a case where the first data operation request does not satisfy a split condition, the method further includes: receiving at least one second data operation request sent by the host; When the following conditions are met, the first data operation request and the at least one second data operation request are aggregated into a total data operation request, and the total data operation request is sent to the storage controller: The data amounts in the first data operation request and each of the second data operation requests are smaller than a second data amount threshold, and the second data amount threshold is smaller than a minimum data amount allowed in a single sub-data operation request; The storage addresses in the first data operation request and each of the second data operation requests point to the same storage area or to continuous storage areas.

4. The method according to claim 3, characterized in that The sending the total data operation request to the storage controller includes: If the data volume in the total data operation request exceeds the first data volume threshold, splitting the total data operation request into multiple sub-data operation requests; The split sub-data operation requests are sent to multiple storage controllers, so that the multiple storage controllers perform data operations in parallel on the target storage node.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: In the case of splitting the data operation request into multiple sub-data operation requests, creating a first association relationship between the data operation request and each sub-data operation request; and In the case of aggregating multiple data operation requests into a total data operation request, a second association relationship between the total data operation request and each data operation request is created.

6. The method according to claim 5, characterized in that After sending the sub-data operation request or the total data operation request to the storage controller, the method further includes: Receive a data operation response returned by the storage controller; The data operation response is split or aggregated according to the first association relationship and / or the second association relationship to obtain a response returned to the host.

7. The method according to claim 1 or 3, characterized in that: After receiving the data operation request sent by the host, the method further includes: When a communication anomaly with the host is detected, for a data operation request sent by the host but not yet sent to the storage controller, the corresponding data operation request is released, and for a data operation request sent by the host and sent to the storage controller, a response returned by the storage controller is discarded; When a communication anomaly with one of the target storage controllers is detected, the data operation request sent to the target storage controller is forwarded to other storage controllers other than the target storage controller.

8. A request processing device based on a distributed system, characterized in that: The quasi-distributed system comprises a shared network device, a plurality of storage controllers and at least one storage node, each of the storage controllers can perform data operations in each of the storage nodes, and the apparatus is applied to the shared network device; the apparatus comprises: A request receiving module, configured to receive a first data operation request sent by a host, wherein the first data operation request includes data information of target data to be operated and a target storage node for storing the target data; a request splitting module, configured to split the first data operation request into a plurality of sub-data operation requests when it is determined that the first data operation request satisfies a splitting condition based on the data information of the target data; The request sending module is used to send the sub-data operation request to multiple storage controllers, so that the multiple storage controllers perform data operations in parallel on the target storage node.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the request processing method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the request processing method according to any one of claims 1 to 7.

Images