Data acquisition method, device and electronic equipment

Abstract

A data acquisition method, device and electronic device, the method comprising: when there are multiple service nodes between a client and a target storage device, determining an operation request received by the client, determining the execution operation corresponding to the operation request based on the operation type in the operation request, when an abnormal service node is determined from multiple service nodes, controlling the client to switch data transmission from the abnormal service node to the normal service node, extracting request data corresponding to the operation request from the target storage device based on the operation request and the normal service node, and sending the request data to the client. Through the above method, multiple links are created between the client and the target storage device to ensure that the connection between the client and the destination subsystem corresponding to the cloud hard disk in the target storage device is not interrupted, so that the client can obtain the data requested in the operation request from the target storage device.

Description

Technical Field

[0001] The present application relates to the field of cloud computing storage technology, and in particular to a data acquisition method, device and electronic device. Background Art

[0002] With the development of cloud computing storage technology, the application of cloud computing storage technology is becoming more and more extensive, and users need to transmit more and more information. In the process of data transmission, data storage plays an important role. Usually, non-volatile memory based on architecture (NVMe over Fabrics, NVMe-OF) technology is used to store massive data information. The NVMe-OF technology sends requests and responses between the client and the target storage device through the network based on the message model to realize the access between the client and the target storage device. The specific process of realizing the client's access to the target storage device based on NVMe-OF technology is as follows:

[0003] Specifically, when NVMe-OF technology is used for cloud computing storage, the connection diagram between the client and the target storage device is as follows: Figure 1 As shown, in Figure 1 In the example, the link between the client and the target storage device is a single-path connection. There is a service node on each link. The schematic diagram of the cloud hard disk management system in the target storage device is as follows: Figure 2 As shown, in Figure 2 In the present invention, the cloud hard disk management system includes n clients, m agents, c control nodes, and a service node, where n, m, and c are all positive integers. The agent is used to forward the client's request to the control node, and the control node is used to receive the client's request forwarded by the agent, and select the corresponding service node to perform the operation based on the request. The service node is used to establish a connection with the target storage device and create a target subsystem corresponding to the cloud hard disk for the client to connect.

[0004] Based on the above description, when the client needs to read the stored data from the target storage device, if an abnormality occurs in the link between the client and the target storage device, or an abnormality occurs in the service node between the client and the target storage device, the connection between the client and the destination subsystem corresponding to the cloud hard disk will be interrupted, and the client will not be able to read the stored data from the cloud hard disk in the target storage device. Summary of the invention

[0005] The present application provides a data acquisition method, device and electronic device, which ensures that the client can read the stored data from the cloud hard disk in the target storage device by ensuring that the connection between the client and the target subsystem of the cloud hard disk is not interrupted.

[0006] In a first aspect, the present application provides a data acquisition method, the method comprising:

[0007] When there are multiple service nodes between the client and the target storage device, determining an operation request received by the client, and determining an execution operation corresponding to the operation request based on an operation type in the operation request;

[0008] When an abnormal service node is determined from the multiple service nodes, controlling the client to switch data transmission from the abnormal service node to a normal service node;

[0009] Based on the operation request and the normal service node, request data corresponding to the operation request is extracted from the target storage device, and the request data is sent to the client.

[0010] Through the above method, multiple service nodes are added between the client and the target storage device, and the cloud hard disk is managed based on the multiple links between the client and the target storage device. When the service node between the client and the target storage device is abnormal, the link between the client and the cloud hard disk in the target storage device can be normal by switching the abnormal service node to a normal service node, so that the client can read the stored data from the cloud hard disk in the target storage device.

[0011] In a possible design, determining the execution operation corresponding to the operation request based on the operation type in the operation request includes:

[0012] When it is determined that the operation type is a cloud hard disk mount request, link information of each cloud hard disk in the target storage device is determined, wherein the link information includes a correspondence between a link identifier and a service IP of a service node in the link;

[0013] Obtaining the cloud hard disk identifier and the link identifier of each cloud hard disk in the cloud hard disk mount request;

[0014] Forward each cloud disk ID and each link ID to the service node through the proxy node;

[0015] The service node is controlled to create a destination subsystem based on each cloud hard disk identifier and each link identifier, and to determine the link to which each cloud hard disk belongs, and to store link information corresponding to each link of each cloud hard disk into a preset information library.

[0016] In a possible design, determining the execution operation corresponding to the operation request based on the operation type in the operation request includes:

[0017] When determining that the operation type is a cloud hard disk unmounting request, determining a mount record of the cloud hard disk;

[0018] Determine each link record in the mount record, and obtain the service node corresponding to each link record;

[0019] Control the service node to perform uninstallation of the cloud hard disk.

[0020] In a possible design, determining the execution operation corresponding to the operation request based on the operation type in the operation request includes:

[0021] When it is determined that the operation type is a cloud hard disk expansion request, extracting the cloud hard disk identifier of each cloud hard disk to be expanded in the cloud hard disk expansion request;

[0022] Traversing the link information of each cloud hard disk from the target storage device, and determining the service nodes corresponding to the links to which each cloud hard disk identifier belongs;

[0023] When the control node sends a cloud disk expansion request to each service node, it controls each service node to perform the expansion operation.

[0024] In a possible design, after the control service node performs the expansion operation, it also includes:

[0025] Obtaining the expansion result of the service node, and controlling the service node to send the expansion result to the client;

[0026] Determine a successfully expanded link from the expansion result, and compare the expanded link with a preset link;

[0027] Determine a first capacity expansion link that is inconsistent with the preset link, and use the first capacity expansion link as a capacity expansion failure link;

[0028] The link where the expansion failed is deleted.

[0029] In a possible design, determining the execution operation corresponding to the operation request based on the operation type in the operation request includes:

[0030] When it is determined that the operation type is a client link detection type, traverse all cloud hard disks mounted in the target storage device according to a preset period;

[0031] Extracting first link information corresponding to each cloud hard disk, and obtaining second link information corresponding to at least one link created by the client;

[0032] Obtain missing link information based on each first link information and at least one second link information;

[0033] The client is controlled based on the missing link information to rebuild the missing link corresponding to the missing link information.

[0034] In a possible design, determining the execution operation corresponding to the operation request based on the operation type in the operation request includes:

[0035] When it is determined that the operation type is a service node reconstruction type, all mounted cloud hard disks of the service node are obtained;

[0036] Obtain the link identifier and cloud disk information of each cloud disk, and create the destination terminal system information based on the link identifier and the cloud disk information, wherein the cloud disk information includes: the cloud disk identifier and the client identifier to which the cloud disk belongs, and the service node where the link identifier of the cloud disk is located;

[0037] In response to the destination subsystem information being successfully established, the link connection corresponding to the abnormal service node is restored.

[0038] In a second aspect, the present application provides a data acquisition device, the device comprising:

[0039] A determination module, configured to determine an operation request received by the client when there are multiple service nodes between the client and the target storage device, and determine an execution operation corresponding to the operation request based on an operation type in the operation request;

[0040] A switching module, configured to control the client to switch data transmission from the abnormal service node to the normal service node when an abnormal service node is determined from the multiple service nodes;

[0041] An extraction module is used to extract request data corresponding to the operation request from the target storage device based on the operation request and the normal service node, and send the request data to the client.

[0042] In one possible design, the determination module is specifically used to determine that when the operation type is a cloud hard disk mount request, determine the link information of each cloud hard disk in the target storage device, obtain the cloud hard disk identifier of each cloud hard disk and the link identifier of each cloud hard disk in the cloud hard disk mount request, forward each cloud hard disk identifier and each link identifier to the service node through the proxy node, control the service node to create a destination terminal system based on each cloud hard disk identifier and each link identifier, determine the link to which each cloud hard disk belongs, and store the link information corresponding to each link of each cloud hard disk in a preset information library.

[0043] In a possible design, the determination module is also used to determine that when the operation type is a cloud hard disk unloading request, determine the mounting record of the cloud hard disk, determine each link record in the mounting record, and obtain the service node corresponding to each link record, and control the service node to perform the unloading of the cloud hard disk.

[0044] In one possible design, the determination module is also used to determine that the operation type is a cloud hard disk expansion request, extract the cloud hard disk identifier of each cloud hard disk to be expanded in the cloud hard disk expansion request, traverse the link information of each cloud hard disk from the target storage device, and determine the service nodes corresponding to the links to which each cloud hard disk identifier belongs. When the control node sends a cloud hard disk expansion request to each service node, it controls each service node to perform the expansion operation.

[0045] In a possible design, the determination module is further used to obtain the expansion result of the service node, and control the service node to send the expansion result to the client, determine the expansion link that has been successfully expanded from the expansion result, compare the expansion link with the preset link, determine a first expansion link that is inconsistent with the preset link, and use the first expansion link as a failed expansion link, and delete the failed expansion link.

[0046] In one possible design, the determination module is also used to determine that when the operation type is a client link detection type, traverse all cloud hard disks mounted in the target storage device according to a preset period, extract the first link information corresponding to each cloud hard disk, and obtain the second link information corresponding to at least one link created by the client, obtain the missing link information based on each first link information and at least one second link information, and control the client to rebuild the missing link corresponding to the missing link information based on the missing link information.

[0047] In one possible design, the determination module is also used to determine that when the operation type is a type of rebuilding a service node, obtain all mounted cloud hard disks of the service node, obtain the link identifier and cloud hard disk information of each cloud hard disk, create the destination terminal system information based on the link identifier and the cloud hard disk information, and in response to the successful establishment of the destination terminal system information, restore the link connection corresponding to the abnormal service node.

[0048] In a third aspect, the present application provides an electronic device, including:

[0049] Memory, used to store computer programs;

[0050] The processor is used to implement the above-mentioned data acquisition method steps when executing the computer program stored in the memory.

[0051] In a fourth aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned data acquisition method are implemented.

[0052] For each aspect from the first to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the connection between the client and the target storage device provided for this application;

[0054] Figure 2 A schematic diagram of a cloud hard disk management system in the target storage device provided by this application;

[0055] Figure 3 A flowchart of the steps of a data acquisition method provided in this application;

[0056] Figure 4 A schematic diagram of the connection between the client and the target storage device provided for this application;

[0057] Figure 5 A schematic diagram of the management system components between the client and the target storage device provided for this application;

[0058] Figure 6 A flowchart of the cloud hard disk mounting operation provided for this application;

[0059] Figure 7 A schematic diagram of the process of selecting a service node by a control node provided in this application;

[0060] Figure 8 A schematic diagram of the process of creating a destination subsystem for the service node provided by this application;

[0061] Fig. 9 A schematic diagram of the uninstallation process of the cloud hard disk provided for this application;

[0062] Fig.10 A schematic diagram of the process of performing cloud hard disk expansion on the client provided in this application;

[0063] Fig.11 A schematic diagram of the process of expanding the capacity of a cloud hard disk by a control node provided for this application;

[0064] Fig.12 Schematic diagram of the client link check process provided for this application;

[0065] Fig.13A schematic diagram of the recovery process of an abnormal service node provided for this application;

[0066] Fig.14 A structural schematic diagram of a data acquisition device provided in this application;

[0067] Fig.15 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0069] In the previous technology, there is only one link between the client and the target storage device, and there is a service node on the link. When the client needs to obtain the stored data from the cloud hard disk in the target storage device, if an abnormality occurs in the link or the service node in the link, the connection between the client and the destination subsystem corresponding to the cloud hard disk in the target storage device will be interrupted, and the client will be unable to obtain the stored data from the target storage device.

[0070] In order to solve the above-described problems, the embodiments of the present application provide a data acquisition method for ensuring that the connection between the client and the destination subsystem corresponding to the cloud hard disk in the target storage device is not interrupted, thereby ensuring that the client can obtain the data stored in the target storage device. Among them, the method and device described in the embodiments of the present application are based on the same technical concept. Since the principles of the problems solved by the method and the device are similar, the embodiments of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0071] Reference Figure 3 The present application provides a data acquisition method, which can ensure that the connection between the client and the target subsystem corresponding to the cloud hard disk in the target storage device is not interrupted, thereby ensuring that the client can obtain the stored data from the target storage device. The implementation process of the method is as follows:

[0072] Step S31: when there are multiple service nodes between the client and the target storage device, determine the operation request received by the client, and determine the execution operation corresponding to the operation request based on the operation type in the operation request.

[0073] The embodiment of the present application is to ensure that the client can obtain the stored data from the target storage device. The client and the target storage device are connected through a link. There is a service node on the link. The service node is used to establish a connection with the target storage device and create a destination subsystem corresponding to the cloud hard disk for the client to connect. The connection diagram between the client and the target storage device is as follows: Figure 4 As shown, in Figure 4 In the example, there are n service nodes between the client and the target storage device. Since each link has a service node, there are n links between the client and the target storage device.

[0074] It should be noted that the schematic diagram of the management system components between the client and the target storage device is as follows Figure 5 As shown, in Figure 5 In the embodiment, there are proxy nodes, control nodes, and service nodes between the client and the target storage device. a, b, c, and n are all positive integers. The client is used to receive cloud hard disk operation requests sent by users and forward the operation requests to the control node through the proxy node. The proxy node can forward the operation request to the control node. When an abnormality occurs in the proxy node, other normal proxy nodes automatically take over the service corresponding to the abnormal proxy node. The control node is used to receive the operation request sent by the proxy node and determine a service node from n service nodes.

[0075] The above describes the establishment of n links between the client and the target storage device, each link corresponds to a link identifier, and each service node on the link corresponds to a service IP. The link identifier and the service identifier are unique. Since the service node is on the link, there is a corresponding relationship between the service IP and the link identifier. In the embodiment of the present application, the corresponding relationship is used as link information. The link information is shown in Table 1:

[0076]

[0077]

[0078] Table 1

[0079] In the above Table 1, the link identifiers and service IPs corresponding to the three links are recorded. Link identifier 1 corresponds to service IP1, link identifier 2 corresponds to service IP2, and link identifier 3 corresponds to service IP3. The link information of other links can be found in the examples in the above Table 1, and no further explanation is given here.

[0080] In addition, the link information described above is stored in the target storage device, and the client needs to establish links corresponding to all link information stored in the target storage device.

[0081] When the server determines the operation request received by the client, it determines the execution operation corresponding to the operation request based on the operation type in the operation request. Since the operation type of the operation request is different based on different real-time scenarios, the operation request corresponds to different execution operations based on different operation types. The execution operations corresponding to each operation type are specifically as follows:

[0082] When the server determines from the operation request that the operation type is a cloud hard disk mount request, it needs to perform the cloud hard disk mount operation. When the client has not established a connection with the target storage device, the cloud hard disk mount operation needs to determine the link information of each cloud hard disk in the target storage device, and obtain the cloud hard disk identifier of each cloud hard disk and the link identifier of each cloud hard disk in the cloud hard disk mount request, and then forward the cloud hard disk identifier and the link identifier to the service node through the proxy node. The service node creates a destination subsystem based on the cloud hard disk identifier of each cloud hard disk and each link identifier to realize the connection between the client and the cloud hard disk. After the connection between the client and the cloud hard disk is established, since the number of cloud hard disks is at least one, it is also necessary to determine the link to which each cloud hard disk belongs, and store the link information corresponding to the link of each cloud hard disk in the preset information library in the target storage device.

[0083] Based on the above description, the process diagram of cloud disk mounting operation is as follows: Figure 6 As shown, in Figure 6 In the example, since the client has not established a connection with the target storage device, the number of link identifiers that need to be initialized is 0. If there are q link information in the target storage device, q is a positive integer, and the client needs to establish q links. After the client establishes the link, it needs to confirm whether the current link that has been successfully established is the last link. If so, it means that the links corresponding to all link information stored in the target storage device have been successfully established. If not, it means that there is still link information in the target storage device that has not been created. The server needs to control the proxy node to notify the control node to create a destination subsystem. The destination subsystem in the embodiment of the present application is target, and target will be used instead of the destination subsystem for subsequent description. The destination subsystem of each link corresponds to a service node in the link. The destination subsystem is used to establish a connection with the client and a connection with the cloud hard disk. The control node needs to select a service node. The specific process of selecting a service node is as follows:

[0084] The flowchart of the control node selecting the service node in the embodiment of the present application is as follows: Figure 7 As shown, in Figure 7In the process, after the server controls the proxy node to notify the control node to create a target, the server determines whether the target exists by accessing the preset information library in the target storage device. If the target to be created does not exist, the service node is notified to delete the target. If the target to be created exists, the service IP of the service node corresponding to the target is obtained, and then it is determined whether the service IP is in the exclusion selection list. When the service IP is in the exclusion selection list, the target is an invalid target, and the server will notify the service node to delete the target. When the service IP is not in the exclusion selection list, the server will notify the service node to create a target. When the service node successfully creates the target, it will notify the client that the target is successfully created. When the service node fails to create, it is necessary to determine the service node with the least number of connected cloud hard disks. If the service node is successfully obtained, the service node is controlled to create the target. If the service node fails to be obtained, the client is notified that the service node acquisition failed.

[0085] Based on the above description, the control node can determine a service node from all service nodes, and the service node is not in the exclusion selection list, thereby ensuring that the obtained service node successfully creates a target.

[0086] Further, if the control node determines a service node, it connects to the service node target returned by the control node, and then determines whether the client is successfully connected to the service node corresponding to the service node target returned by the control node. When the connection is successful, the number of link identifiers is increased by 1, and it is determined whether the currently successfully created link is the last link; when the connection fails, the service node corresponding to the service node target returned by the control node is placed in the exclusion selection list, and the service node is not among the multiple service nodes that the control node can select, and the server continues to control the proxy node to notify the control node to create the target.

[0087] Based on the above process, the cloud hard disk can be mounted. After the cloud hard disk is mounted successfully, there are multiple cloud hard disks and data corresponding to the cloud hard disks in the target storage device.

[0088] After the server control agent node notifies the control node to create a target, the service node will establish a connection with the target storage device and create a destination subsystem corresponding to the cloud hard disk for the client to connect to. Since the control node has determined the service node to create the destination subsystem, the process diagram of the service node creating the destination subsystem is as follows: Figure 8 As shown, in Figure 8In the process, after receiving the request to create a target, the service node determines whether the target record exists. If the target record exists, it determines whether the preset capacity of the service node is equal to the actual storage capacity of the target storage device. When the displayed capacity is equal to the actual storage capacity, it determines whether the link identifier corresponding to the target is equal to the preset link identifier. If they are equal, the cloud hard disk corresponding to the target is deleted from the link identifier table corresponding to the service node. If they are not equal, the target record and the cloud hard disk link identifier record are deleted, and the server then controls the service node to delete the cloud hard disk from the link identifier table corresponding to the service node.

[0089] When the target record does not exist, the control service node needs to create a new target, add a new mounted cloud hard disk in the link identification table corresponding to the service node, and add a mounted cloud hard disk record in the preset information library. The mounted cloud hard disk record is the link identification corresponding to the new target and the service IP corresponding to the service node. Then, the target's serial number and service IP are returned to the control node. The serial number is the unique identifier of the target.

[0090] Based on the above description, the service node determined by the control node can create a destination subsystem.

[0091] When the server determines from the operation request that the operation type is a cloud hard disk unmounting request, it needs to determine the mount record of the cloud hard disk and read each link record and the service node corresponding to each link record from the mount record. The server then controls the service node to perform the unmounting of the cloud hard disk.

[0092] Specifically, the cloud hard disk uninstall process diagram is as follows: Fig. 9 As shown, in Fig. 9 In the process, when the service node receives a request to uninstall a cloud hard disk, it extracts the link information of the cloud hard disk to be uninstalled from the cloud hard disk uninstall request, and at the same time, queries the cloud hard disk mount record from the preset information library to determine whether the cloud hard disk is in the cloud hard disk mount record. If the cloud hard disk is not in the cloud hard disk mount record, the result of the completed uninstall process is returned to the client; if the cloud hard disk is in the cloud hard disk mount record, each link record is traversed, and it is determined whether the link corresponding to the cloud hard disk is the last link. When the link corresponding to the cloud hard disk is the last link, the result of the completed uninstall process is returned to the client; when the link corresponding to the cloud hard disk is not the last link, the service node corresponding to the link of the uninstalled cloud hard disk is obtained, and the service node is notified to uninstall the target corresponding to the cloud hard disk.

[0093] When the server determines from the operation request that the operation type is a cloud hard disk expansion request, the server needs to extract the cloud hard disk identifiers of each cloud hard disk to be expanded in the cloud hard disk expansion request, and traverse the link information of each cloud hard disk in the target storage device to determine the service nodes corresponding to the links to which each cloud hard disk identifier belongs. When the control node sends a cloud hard disk expansion request to each service node, the server controls each service node to perform the expansion operation.

[0094] After the service node is expanded, the expansion result of the service node is obtained, and the server controls the service node to send the expansion result to the client, determines the expansion link that has been successfully expanded from the expansion result, and compares the successfully expanded link with the preset link in the target storage device, determines the first expansion link that is inconsistent with the preset link, and regards the first expansion link that is inconsistent with the preset link as a failed expansion link, and deletes the failed expansion link.

[0095] Specifically, the expansion of the cloud hard disk requires the client to execute the expansion process and the control node to execute the expansion process. The process diagram of the client executing the cloud hard disk expansion is as follows: Fig.10 As shown, in Fig.10 In the process, the client notifies the control node to expand the cloud hard disk. After the control node expands the cloud hard disk, it obtains the expansion success link and compares the expansion success link with the preset link in the target storage device. The link that is inconsistent with the preset link is regarded as the expansion failure link. If the expansion success link is consistent with the preset link in the target storage device, the expansion is determined to be successful.

[0096] Based on the above description, the process diagram of the control node expanding the cloud hard disk is as follows: Fig.11 As shown, in Fig.11 In the process, after receiving the cloud disk expansion request, the control node queries the link information of the cloud disk, traverses each link, and determines whether the link corresponding to the cloud disk is the last link. If the link corresponding to the cloud disk is the last link, it checks whether the expansion success list is empty. When the expansion success list is empty, the control node stops the client expansion failure. When the expansion success list is not empty, the control node notifies the client of the expansion success and returns the expansion success list. When the link corresponding to the cloud disk is not the last link, the service IP and link information of the expanded service node are recorded in the expansion success list.

[0097] When the server determines that the operation type is the client link detection type, it will traverse all mounted cloud hard disks in the target storage device according to the preset period, extract the first link information corresponding to each cloud hard disk, and obtain the second link information corresponding to at least one link created by the client, and then obtain the missing link information based on the first link information and the second link information. The server then controls the client to create the corresponding missing link based on the missing link information.

[0098] Further, the client link check process diagram is as follows Fig.12 As shown, in Fig.12 In the process, the server obtains all mounted cloud hard disks of the client. When the mounted cloud hard disk is the last cloud hard disk, the client link check process is exited; when the mounted cloud hard disk is not the last cloud hard disk, the number of links of the cloud hard disk is obtained. When the number of links connected to the cloud hard disk is not 1, the next link is checked; when the number of links connected to the cloud hard disk is 1, the server notifies the control node to obtain all first link information of the cloud hard disk, and also needs to obtain all second link information corresponding to the client, compare the first link information with the second link information, determine the inconsistent link information, and determine the new link identifier based on the link information, and execute the new link connection process based on the new link identifier and the cloud hard disk information, and then check the next link to determine whether the current cloud hard disk is the last cloud hard disk.

[0099] When the server determines that the operation type is to rebuild the service node type, it obtains all mounted cloud hard disks of the service node, as well as the link identifier and cloud hard disk information of each cloud hard disk, and creates the destination terminal system information based on the link identifier and the cloud hard disk information. After the service node successfully creates the destination terminal system information corresponding to the cloud hard disk, the link connection corresponding to the abnormal service node is restored based on the link detection mechanism in the NVMe-OF technology. Since the link detection mechanism in the NVMe-OF technology and the NVMe-OF technology are both well known to those skilled in the art, they will not be elaborated here.

[0100] Specifically, the recovery process diagram of the abnormal service node is as follows: Fig.13 As shown, in Fig.13 In the process, when the server detects an abnormal service node, the service node will perform a cloud hard disk target reconstruction, and the server will query the preset information library to obtain all the mounted cloud hard disk information of the service node. When the cloud hard disk is the last cloud hard disk, the creation process ends; when the cloud hard disk is not the last cloud hard disk, the cloud hard disk identifier and the link identifier of the service node are determined, and target reconstruction is performed based on the link identifier and the cloud hard disk identifier, and then the next cloud hard disk record is obtained, and it is determined whether the next cloud hard disk is the last cloud hard disk.

[0101] Based on the above description, multiple service nodes are added to the cloud hard disk management architecture, and the cloud hard disk is mounted, unmounted, and expanded based on the cloud hard disk management architecture, as well as client link detection and reconstruction of service nodes, etc. The service node is determined through the control node, thereby improving the reliability of the cloud hard disk management system.

[0102] Step S32: when an abnormal service node is determined from the multiple service nodes, controlling the client to switch data transmission from the abnormal service node to a normal service node.

[0103] When the server detects that there is an abnormal service node among the multiple service nodes connected to the client, it will control the client to switch data transmission from the abnormal service node to the normal service node. Since the above has described the cloud hard disk mounting process, the cloud hard disk unmounting process and the service node recovery process, the service node switching process will refer to the above description and will not be elaborated here.

[0104] Step S33: extracting request data corresponding to the operation request from the target storage device based on the operation request and the normal service node, and sending the request data to the client.

[0105] After the client is switched from the abnormal service node to the normal service node, since the operation request indicates the requested data that the user needs to obtain, after the client switches to the normal service node, the normal service node will obtain the requested data from the cloud hard disk of the target storage device based on the operation request, and return the requested data to the client, so that the client obtains the data from the target storage device.

[0106] Based on the above description, multiple links are created between the client and the target storage device, each link corresponds to a service node, and the architecture based on the multiple links can realize the cloud hard disk mounting, unmounting, expansion, as well as link detection, link reconstruction, abnormal node recovery and other operations. Moreover, when there is an abnormal service node between the client and the target storage device, the link for data transmission can be normal by switching the abnormal service node to a normal service node, ensuring that the connection between the client and the destination subsystem corresponding to the cloud hard disk in the target storage device is not interrupted, so that the client can obtain the data requested in the operation request from the target storage device.

[0107] Based on the same inventive concept, a data acquisition device is also provided in the embodiment of the present application. The thread-bound device is used to implement the function of a data acquisition method. Fig.14 , the device comprises:

[0108] Determination module 1401, used for determining an operation request received by the client when there are multiple service nodes between the client and the target storage device, and determining an execution operation corresponding to the operation request based on an operation type in the operation request;

[0109] A switching module 1402 is used to control the client to switch data transmission from the abnormal service node to the normal service node when an abnormal service node is determined from the multiple service nodes;

[0110] The extraction module 1403 is used to extract the request data corresponding to the operation request from the target storage device based on the operation request and the normal service node, and send the request data to the client.

[0111] In one possible design, the determination module 1401 is specifically used to determine that when the operation type is a cloud hard disk mount request, determine the link information of each cloud hard disk in the target storage device, obtain the cloud hard disk identifier of each cloud hard disk and the link identifier of each cloud hard disk in the cloud hard disk mount request, forward each cloud hard disk identifier and each link identifier to the service node through the proxy node, control the service node to create a destination subsystem based on each cloud hard disk identifier and each link identifier, determine the link to which each cloud hard disk belongs, and store the link information corresponding to each link of each cloud hard disk in a preset information library.

[0112] In a possible design, the determination module 1401 is also used to determine that when the operation type is a cloud hard disk unloading request, determine the mounting record of the cloud hard disk, determine each link record in the mounting record, and obtain the service node corresponding to each link record, and control the service node to perform the unloading of the cloud hard disk.

[0113] In one possible design, the determination module 1401 is also used to determine that the operation type is a cloud hard disk expansion request, extract the cloud hard disk identifier of each cloud hard disk to be expanded in the cloud hard disk expansion request, traverse the link information of each cloud hard disk from the target storage device, and determine the service nodes corresponding to the links to which each cloud hard disk identifier belongs. When the control node sends a cloud hard disk expansion request to each service node, it controls each service node to perform the expansion operation.

[0114] In a possible design, the determination module 1401 is further used to obtain the expansion result of the service node, and control the service node to send the expansion result to the client, determine the expansion link that has been successfully expanded from the expansion result, compare the expansion link with the preset link, determine the first expansion link that is inconsistent with the preset link, and use the first expansion link as a failed expansion link, and delete the failed expansion link.

[0115] In one possible design, the determination module 1401 is also used to determine that when the operation type is a client link detection type, traverse all cloud hard disks mounted in the target storage device according to a preset period, extract the first link information corresponding to each cloud hard disk, and obtain the second link information corresponding to at least one link created by the client, obtain the missing link information based on each first link information and at least one second link information, and control the client to rebuild the missing link corresponding to the missing link information based on the missing link information.

[0116] In one possible design, the determination module 1401 is also used to determine that when the operation type is a type of rebuilding a service node, obtain all mounted cloud hard disks of the service node, obtain the link identifier and cloud hard disk information of each cloud hard disk, create the destination terminal system information based on the link identifier and the cloud hard disk information, and in response to the successful establishment of the destination terminal system information, restore the link connection corresponding to the abnormal service node.

[0117] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the functions of the aforementioned data acquisition device, referring to Fig.15 , the electronic device comprises:

[0118] At least one processor 1501, and a memory 1502 connected to the at least one processor 1501. The specific connection medium between the processor 1501 and the memory 1502 is not limited in the embodiment of the present application. Fig.15 In the example, the processor 1501 and the memory 1502 are connected via the bus 1500. The bus 1500 is Fig.15 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 1500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 1501 can also be called a controller, and there is no limitation on the name.

[0119] In the embodiment of the present application, the memory 1502 stores instructions that can be executed by at least one processor 1501. The at least one processor 1501 can execute a data acquisition method discussed above by executing the instructions stored in the memory 1502. The processor 1501 can implement Fig.14 The functions of each module in the device shown.

[0120] Among them, processor 1501 is the control center of the device, which can use various interfaces and lines to connect various parts of the entire control device, and monitor the device as a whole by running or executing instructions stored in memory 1502 and calling data stored in memory 1502, various functions of the device and processing data.

[0121] In one possible design, the processor 1501 may include one or more processing units, and the processor 1501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1501. In some embodiments, the processor 1501 and the memory 1502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0122] Processor 1501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a data acquisition method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0123] Memory 1502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 1502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. Memory 1502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0124] By designing and programming the processor 1501, the code corresponding to the data acquisition method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1 A data acquisition step of the embodiment shown. How to design and program the processor 1501 is a technique known to those skilled in the art and will not be described in detail here.

[0125] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes a data acquisition method discussed above.

[0126] In some possible implementations, various aspects of a data acquisition method provided by the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of a data acquisition method according to various exemplary implementations of the present application described above in this specification.

[0127] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0131] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data acquisition method, characterized in that: include: When there are multiple service nodes between the client and the target storage device, determine the operation request received by the client, and when it is determined that the operation type in the operation request is a cloud hard disk mount request, determine the link information of each cloud hard disk in the target storage device, wherein the link information includes the corresponding relationship between the link identifier and the service IP of the service node in the link; Obtaining the cloud hard disk identifier and the link identifier of each cloud hard disk in the cloud hard disk mount request; Forward each cloud disk ID and each link ID to the service node through the proxy node; Control the service node to create a destination subsystem based on each cloud hard disk identifier and each link identifier, determine the link to which each cloud hard disk belongs, and store the link information corresponding to each link of each cloud hard disk into a preset information library, wherein the destination subsystem is used to establish a connection with the client and the cloud hard disk; When an abnormal service node is determined from the multiple service nodes, controlling the client to switch data transmission from the abnormal service node to a normal service node; Based on the operation request and the normal service node, request data corresponding to the operation request is extracted from the target storage device, and the request data is sent to the client.

2. The method according to claim 1, characterized in that Also includes: When determining that the operation type is a cloud hard disk unmounting request, determining a mount record of the cloud hard disk; Determine each link record in the mount record, and obtain the service node corresponding to each link record; Control the service node to perform uninstallation of the cloud hard disk.

3. The method according to claim 1, characterized in that Also includes: When it is determined that the operation type is a cloud hard disk expansion request, extracting the cloud hard disk identifier of each cloud hard disk to be expanded in the cloud hard disk expansion request; Traversing the link information of each cloud hard disk from the target storage device, and determining the service nodes corresponding to the links to which each cloud hard disk identifier belongs; When the control node sends a cloud disk expansion request to each service node, it controls each service node to perform the expansion operation.

4. The method according to claim 3, characterized in that After the control service node performs the expansion operation, it also includes: Obtaining the expansion result of the service node, and controlling the service node to send the expansion result to the client; Determine a successfully expanded link from the expansion result, and compare the expanded link with a preset link; Determine a first capacity expansion link that is inconsistent with the preset link, and use the first capacity expansion link as a capacity expansion failure link; The link where the expansion failed is deleted.

5. The method according to claim 1, characterized in that Also includes: When it is determined that the operation type is a client link detection type, traverse all cloud hard disks mounted in the target storage device according to a preset period; Extracting first link information corresponding to each cloud hard disk, and obtaining second link information corresponding to at least one link created by the client; Obtain missing link information based on each first link information and at least one second link information; The client is controlled based on the missing link information to rebuild the missing link corresponding to the missing link information.

6. The method according to claim 1, characterized in that Also includes: When it is determined that the operation type is a service node reconstruction type, all mounted cloud hard disks of the service node are obtained; Obtain the link identifier and cloud disk information of each cloud disk, and create the destination terminal system information based on the link identifier and the cloud disk information, wherein the cloud disk information includes: the cloud disk identifier and the client identifier to which the cloud disk belongs, and the service node where the link identifier of the cloud disk is located; In response to the destination subsystem information being successfully established, the link connection corresponding to the abnormal service node is restored.

7. A data acquisition device, characterized in that: include: A determination module, used to determine, when there are multiple service nodes between the client and the target storage device, an operation request received by the client, and when it is determined that the operation type in the operation request is a cloud hard disk mount request, determine the link information of each cloud hard disk in the target storage device, wherein the link information includes a corresponding relationship between a link identifier and a service IP of a service node in the link; Obtaining the cloud hard disk identifier and the link identifier of each cloud hard disk in the cloud hard disk mount request; Forward each cloud disk ID and each link ID to the service node through the proxy node; Control the service node to create a destination subsystem based on each cloud hard disk identifier and each link identifier, determine the link to which each cloud hard disk belongs, and store the link information corresponding to each link of each cloud hard disk into a preset information library, wherein the destination subsystem is used to establish a connection with the client and the cloud hard disk; A switching module, configured to control the client to switch data transmission from the abnormal service node to the normal service node when an abnormal service node is determined from the multiple service nodes; An extraction module is used to extract request data corresponding to the operation request from the target storage device based on the operation request and the normal service node, and send the request data to the client.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 6 when executing the computer program stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 6 are implemented.

Images