Distributed storage full optimization path read-write method and electronic device

Abstract

This application provides a fully optimized path read / write method and electronic device for distributed storage. By introducing segment weight information to characterize whether the OSD associated with each segment is local to the storage node, this application assists the business server in dynamically determining the optimal storage node for handling IO requests—that is, the storage node where the OSD accessed by the IO request is located—based on the latest segment weight information returned by each storage node. This fully adapts to the data distribution characteristics of distributed storage, such as data segmentation and OSD mapping, thereby maximizing the performance of the distributed storage system, effectively reducing cross-node OSD access for IO requests, lowering node network hop overhead, and ultimately improving data processing performance such as data read / write.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular to a fully optimized path read / write method and electronic device for distributed storage. Background Technology

[0002] Currently, to improve the reliability and performance of distributed storage systems and ensure the reliable operation of business systems, multi-path technology is typically used to access storage resources within the storage system. Multi-path technology can be understood as allowing the same distributed storage system to connect to the same business system through multiple physical paths. This enables the business system to access storage resources in the storage system through these multiple physical paths to perform corresponding data processing, such as data reading and writing. These multiple physical paths can refer to the communication paths between each storage node in the distributed storage system and the business system.

[0003] For the aforementioned multi-path access scenarios, various path selection strategies are typically employed in practical applications. For example, a round-robin strategy might be used, sequentially using each path to send input / output (IO) requests to the distributed storage system. Alternatively, a failover strategy might be employed, where all IO requests are transmitted through the primary path, switching to a backup path only when the primary path fails. However, these path selection strategies cannot fully adapt to the inherent data distribution characteristics of distributed storage, such as data segmentation and object storage device (OSD) mapping. This results in the distributed storage system's performance not being fully utilized, thereby reducing data processing performance. Summary of the Invention

[0004] In view of this, this application provides a fully optimized path read / write method and electronic device for distributed storage to improve data processing performance.

[0005] This application provides a fully optimized path read / write method for distributed storage. The method is applied to a business server, which connects to different storage nodes in the distributed storage system via different paths. The method includes: Weight request instructions are issued by connecting different paths of different storage nodes in the distributed storage system. The weight request instruction carries a Logical Unit Number (LUN) identifier, which is used to instruct any storage node that receives the weight request instruction to return a weight response instruction carrying the weights of each segment associated with the LUN identifier. If the OSD associated with a segment is a local OSD of the storage node, the weight of the segment is the first weight. If the OSD associated with the segment is not a local OSD of the storage node, the weight of the segment is the second weight. For each storage node's returned weight response instruction, determine the path set associated with the LUN identifier; the path set includes the weight of each segment associated with the LUN identifier carried in the weight response instruction returned by each storage node. When an IO request for a target segment is received, a target storage node is determined based on the set of target paths associated with the LUN identifier of the target LUN to which the target segment belongs, and IO processing is performed on the target storage node; the weight of the target segment returned by the target storage node in the set of target paths is the first weight.

[0006] This application also provides a fully optimized path read / write method for distributed storage, which is applied to any storage node in a distributed storage system. The method includes: The system receives a weight request instruction from the connected service server, the weight request instruction carrying a LUN identifier, and returns a weight response instruction to the service server carrying the weights of each segment associated with the LUN identifier; wherein, if the object storage device OSD associated with a segment is a local OSD of this storage node, the weight of the segment is the first weight, and if the OSD associated with the segment is not a local OSD of this storage node, the weight of the segment is the second weight. The service server connects to different storage nodes in the distributed storage system via different paths. When the service server receives weight response instructions from each storage node, it determines the path set associated with the LUN identifier. The path set includes the weights of each segment associated with the LUN identifier carried in the weight response instructions returned by each storage node. When it receives an input / output IO request for a target segment, it determines the target storage node based on the target path set associated with the LUN identifier of the target LUN to which the target segment belongs, and performs IO processing on the target storage node. The weight of the target segment returned by the target storage node in the target path set is the first weight.

[0007] This application embodiment also provides a fully optimized path read / write device for distributed storage. This device is applied to a business server, which connects to different storage nodes in the distributed storage system via different paths. The device includes: The distribution module is used to distribute weight request instructions through different paths connecting different storage nodes in the distributed storage system. The weight request instruction carries a LUN identifier, which is used to instruct any storage node that receives the weight request instruction to return a weight response instruction carrying the weights of each segment associated with the LUN identifier. If the OSD associated with a segment is a local OSD of the storage node, the weight of the segment is the first weight; if the OSD associated with the segment is not a local OSD of the storage node, the weight of the segment is the second weight. The determination module is used to determine the set of paths associated with the LUN identifier in response to the weight response instructions returned by each storage node; the set of paths includes the weights of each segment associated with the LUN identifier carried by the weight response instructions returned by each storage node. The processing module is configured to, when receiving an IO request for a target segment, determine a target storage node based on a set of target paths associated with the LUN identifier of the target LUN to which the target segment belongs, and perform IO processing on the target storage node; the weight of the target segment returned by the target storage node in the set of target paths is a first weight.

[0008] This application also provides a fully optimized path read / write device for distributed storage, which is applied to any storage node in a distributed storage system. The device includes: The receiving module is used to receive a weight request instruction sent by the connected service server. The weight request instruction carries a LUN identifier and returns a weight response instruction to the service server, which carries the weights of each segment associated with the LUN identifier. If the OSD associated with a segment is a local OSD of the storage node, the weight of the segment is the first weight; if the OSD associated with the segment is not a local OSD of the storage node, the weight of the segment is the second weight. The service server connects to different storage nodes in the distributed storage system via different paths. When the service server receives weight response instructions from each storage node, it determines the path set associated with the LUN identifier. The path set includes the weights of each segment associated with the LUN identifier carried in the weight response instructions returned by each storage node. When it receives an IO request for a target segment, it determines the target storage node based on the target path set associated with the LUN identifier of the target LUN to which the target segment belongs, and performs IO processing on the target storage node. The weight of the target segment returned by the target storage node in the target path set is the first weight.

[0009] This application also provides an electronic device, which includes: Processor; and A computer-readable storage medium storing computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method described above.

[0010] This application also provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the above method.

[0011] As can be seen from the above technical solutions, in this embodiment, by introducing segment weight information to characterize whether the OSD associated with each segment is local to the storage node, the business server can dynamically determine the optimal storage node for handling IO requests based on the latest segment weight information returned by each storage node. This means that the storage node where the OSD accessed by the IO request is located locally can be used. This can fully adapt to the data distribution characteristics of distributed storage itself, such as data segmentation and OSD mapping, so as to give full play to the performance of the distributed storage system, thereby effectively reducing cross-node OSD access of IO requests, reducing node network jump overhead, and thus improving the performance of data processing such as data read and write. Attached Figure Description

[0012] Figure 1 This is a schematic diagram illustrating the implementation of a distributed storage system provided in an embodiment of this application.

[0013] Figure 2 This is a flowchart illustrating a fully optimized path read / write method for distributed storage provided in an embodiment of this application.

[0014] Figure 3 This is a schematic diagram illustrating the path implementation provided in the embodiments of this application.

[0015] Figure 4 This is a schematic diagram illustrating the implementation of the path set provided in the embodiments of this application.

[0016] Figure 5 This is a schematic diagram illustrating the implementation of a request command format provided in an embodiment of this application.

[0017] Figure 6 This is a schematic diagram illustrating the implementation of a response command format provided in an embodiment of this application.

[0018] Figure 7 This is a flowchart illustrating another fully optimized path read / write method for distributed storage provided in an embodiment of this application.

[0019] Figure 8 This is a schematic diagram of the structure of a distributed storage fully optimized path read / write device provided in an embodiment of this application.

[0020] Figure 9 This is a schematic diagram of another distributed storage fully optimized path read / write device provided in an embodiment of this application.

[0021] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] The application scenarios of the embodiments of this application will be described below: Cloud computing is a distributed system built on interconnected servers. It integrates and schedules various hardware and software resources to form heterogeneous and autonomous services on the Internet, providing on-demand cloud computing services to enterprises and individual users. Cloud computing features ultra-large scale, high reliability, high flexibility, on-demand autonomous service, diversified services, resource pooling, and support for heterogeneity and virtualization. It can be widely used in many technical fields such as industry, agriculture, and services. Cloud computing typically consists of three parts: distributed computing, distributed storage, and distributed routing. Distributed storage provides data storage space for distributed computing and usually needs to meet requirements such as high performance, high availability, and high scalability.

[0023] Currently, traditional business systems typically use block storage protocols such as Internet Small Computer System Interface (iSCSI), Fibre Channel (FC), or Non-Volatile Memory Express over Fabrics (NVMf) to interface with backend distributed storage systems like Ceph. The distributed storage system then provides block device interfaces to the business systems. As business demands for storage performance increase, improving the performance of distributed storage systems becomes increasingly important. Therefore, how to efficiently adapt distributed storage systems to front-end business needs to fully leverage their storage performance has become a pressing technical challenge.

[0024] In this embodiment, see Figure 1 The distributed storage system shown typically uses the Target protocol access layer to directly connect to the front-end business system, without needing to perform protocol conversion through a gateway. This embodiment uses a native SCSI LUN to connect with the Target (i.e., the server on the distributed storage system side). The data allocated to the Target's LUN is divided into contiguous data segments according to address, and each data segment is encapsulated as a Segment (e.g., Segment0, Segment1, ..., SegmentN, where N is an integer greater than 1). Then, a preset algorithm maps each Segment to a group of OSDs in the distributed storage system (e.g., Segment0 is mapped to OSD1, OSD2, and OSD3). The storage space of the OSDs is uniformly managed by the storage management engine in the distributed storage system, such as the MStor engine, and the data corresponding to the Segment is ultimately stored in the OSD. This effectively reduces the performance loss caused by multi-layer protocol conversion and object encapsulation.

[0025] In this embodiment, current business systems typically employ multi-pathing technology to improve the reliability and performance of distributed storage systems. This multi-pathing technology can be understood as allowing the same distributed storage system to establish connections with the same business system through multiple physical paths, enabling the business system to access storage resources within the distributed storage system via these multiple physical paths. These multiple physical paths can refer to different paths connecting the business server where the business system resides to different storage nodes within the distributed storage system. Current business systems can utilize multi-pathing software provided by the operating system to access storage resources in the distributed storage system using these multiple physical paths, thus creating the need for optimal path selection. Currently, in practical applications, the following three path selection strategies are commonly used: Failover strategy: All IO requests are transmitted through the primary path by default, and the system only switches to the backup path when the primary path fails. Round Robin strategy: Issues IO requests to the distributed storage system in turn using all available paths in sequence; The Least Queue Depth strategy selects the path with the fewest currently incomplete IO requests (QueueDepth) to send IO requests.

[0026] However, none of the above path selection strategies can fully adapt to the data distribution characteristics of distributed storage itself, such as data segmentation and object storage device (OSD) mapping. They cannot perceive the actual storage location of data, which leads to the distributed storage system's performance not being fully utilized, thereby reducing data processing performance.

[0027] Based on this, to solve the above problems, this application further proposes a fully optimized path read / write method for distributed storage. To make the method provided by this application easier to understand, the method is described in detail below with reference to the accompanying drawings and embodiments.

[0028] See Figure 2 , Figure 2 This is a flowchart illustrating a method provided in an embodiment of this application. In this embodiment, the method is applied to a business server; specifically, the method can be executed by multipathing software deployed on the business server. The business server connects to different storage nodes in a distributed storage system through different paths.

[0029] like Figure 2 As shown, the process may include the following steps: Step 201: Send weight request instructions through different paths connecting different storage nodes in the distributed storage system; the weight request instructions carry LUN identifiers, which are used to instruct any storage node that receives the weight request instructions to return a weight response instruction carrying the weights of each segment associated with the LUN identifiers.

[0030] In this embodiment, see Figure 3The diagram illustrates the different paths (path 1, path 2, path 3, and path 4) that connect the business server to the various storage nodes (i.e., storage node 1, storage node 2, storage node 3, and storage node 4) in the distributed storage system. The business server can use these four paths to send IO requests to the distributed storage system. These IO requests can include, for example, write IO requests and read IO requests. Taking SegmentN as an example, it is mapped to OSD1 (deployed locally on storage node 1), OSD5 (deployed locally on storage node 3), and OSD7 (deployed locally on storage node 4) in the distributed storage system.

[0031] In a distributed storage system, upon receiving an I / O request, any storage node determines the address segment corresponding to the data to be processed based on the I / O start address (such as the start LBA address) and the data size indicated by the I / O request. Then, it determines the OSD corresponding to the data to be processed (i.e., the OSD to which the segment corresponding to the data is mapped) based on the segment, and processes the data based on the OSD, such as reading or writing data from that OSD. Here, LBA is an abbreviation for Logical Block Addressing.

[0032] Based on this, for read I / O requests, if the read I / O request for SegmentN is issued through paths 1, 3, and 4, the corresponding storage node can directly access its local disk (or local OSD, with a one-to-one correspondence between OSD and disk) to perform the corresponding read I / O processing after receiving the request. This eliminates the need to access disks on other storage nodes across the network, i.e., cross-node disk access. Therefore, paths 1, 3, and 4 are the shortest paths (i.e., the optimal paths). However, if the read I / O request for SegmentN is issued through path 2, the corresponding storage node must still access the disk across nodes to perform the corresponding read I / O processing after receiving the request. Therefore, path 2 is the second shortest path (i.e., the non-optimal path).

[0033] Similarly, for write I / O requests, if a write I / O request for Segment N is issued via paths 1, 3, and 4, the corresponding storage node, upon receiving the request, only needs to traverse the network twice to access the OSDs on the other two storage nodes to complete the write I / O process, since it already has one OSD to access (i.e., a copy of the data available locally). Therefore, paths 1, 3, and 4 are the shortest paths (i.e., the optimal paths). However, if a write I / O request for Segment N is issued via path 2, the corresponding storage node, upon receiving the request, will need to traverse the network three times to access the OSDs on the other three storage nodes to complete the write I / O process, since it does not already have the OSD to access locally. Therefore, path 2 is the second shortest path (i.e., the non-optimal path).

[0034] As described above, under the same LUN, different IO starting addresses correspond to different segments, and different segments correspond to different OSDs, resulting in differences in the optimal path for different IO requests. Therefore, this embodiment determines different weights for different segments under each LUN at the storage node level. Specifically, for a storage node, if the OSD associated with a segment (i.e., the OSD mapped to the segment) is a local OSD of that storage node, then the weight of that segment is the first weight (which indicates that the path corresponding to that storage node is the optimal path). If the OSD associated with that segment is not a local OSD of that storage node, then the weight of that segment is the second weight (which indicates that the path corresponding to that storage node is a non-optimal path). This is used to subsequently select the optimal delivery path for IO requests for any segment under that LUN based on the weights of each segment associated with any LUN under each storage node, thereby improving IO processing performance.

[0035] Based on the above description, in this embodiment, as an example, at least one LUN (which can be referred to as a specified LUN) can be specified in advance based on actual application requirements; and after the path connection between the business server and different storage nodes in the distributed storage system is successfully established, for each specified LUN, a weight request instruction is sent to the distributed storage system through the established different paths; wherein, the weight request instruction carries the LUN identifier of the specified LUN, so as to instruct any storage node that receives the weight request instruction to return a weight response instruction carrying the weights of each segment associated with the above-mentioned LUN identifier.

[0036] It should be noted that this embodiment does not specifically limit the timing of triggering the above-mentioned weight request instruction. For example, the weight request instruction can also be triggered based on the received manual instruction.

[0037] Step 202: For the weight response instructions returned by each storage node, determine the set of paths associated with the LUN identifier; the set of paths includes the weights of each segment associated with the LUN identifier carried by the weight response instructions returned by each storage node.

[0038] In this embodiment, as one example, after receiving the weight response instructions returned by each storage node in response to the aforementioned weight request instruction, the service server can integrate the weights of each segment associated with the LUN identifier carried in each received weight response instruction to obtain a path set associated with the aforementioned LUN identifier. This path set includes the weights of each segment associated with the aforementioned LUN identifier carried in the weight response instruction returned by each storage node. For example, it can be... Figure 4 The table shown records the weights of each segment associated with a LUN identifier returned by each storage node.

[0039] Step 203: When an IO request for a target segment is received, the target storage node is determined based on the target path set associated with the LUN identifier of the target LUN to which the target segment belongs, and IO processing is performed on the target storage node; the weight of the target segment returned by the target storage node in the target path set is the first weight.

[0040] In this embodiment, as one example, when an IO request for any LUN is received, the target segment corresponding to the IO request can be determined based on the IO start address and data size corresponding to the data to be processed indicated by the IO request. For example, the actual address segment corresponding to the data to be processed can be determined based on the sum of the IO start address and data size corresponding to the data to be processed, and the segment where the actual address segment under the LUN is located is taken as the target segment corresponding to the IO request. Then, the target storage node can be determined based on the target path set associated with the LUN identifier of the LUN, wherein the weight of the target segment returned by the target storage node in the target path set is the first weight, and corresponding IO processing is performed on the target storage node.

[0041] Optionally, in this embodiment, as described above, the target path set may contain multiple storage nodes whose returned target segments have a weight of the first weight. Based on this, this embodiment may randomly select one of the multiple storage nodes as the target storage node, or may select the multiple storage nodes in turn in sequence according to the polling mechanism described above, etc. There are no specific limitations on this here.

[0042] Optionally, in this embodiment, the aforementioned IO processing for the target storage node may include, for example, issuing the IO request through a path connected to the target storage node, so that when the storage node receives the IO request, it processes the IO request based on multiple OSDs to which the target segment is mapped in the distributed storage system. Specifically, when the storage node receives the IO request, it can determine the target segment corresponding to the IO request based on the IO start address and data size corresponding to the data to be processed indicated by the IO request, and determine the OSD corresponding to the target segment based on the locally recorded segment-OSD mapping relationship (or based on a preset distributed algorithm), so as to process the data to be processed indicated by the IO request based on the OSD corresponding to the target segment, such as reading or writing data from the OSD.

[0043] This concludes the process. Figure 2 The process is shown below.

[0044] pass Figure 2 As can be seen from the process shown, in this embodiment of the application, by introducing segment weight information to characterize whether the OSD associated with each segment is local to the storage node, the business server is assisted in dynamically determining the optimal storage node for handling IO requests based on the latest segment weight information returned by each storage node. This means that the storage node where the OSD accessed by the IO request is located locally exists. This can fully adapt to the data distribution characteristics of distributed storage itself, such as data segmentation and OSD mapping, so as to give full play to the performance of the distributed storage system, thereby effectively reducing cross-node OSD access of IO requests, reducing node network jump overhead, and thus improving the performance of data processing such as data read and write.

[0045] The following is a summary of the above. Figure 2 The fully optimized path read / write method for distributed storage shown will be further described below: In this embodiment, the set of paths associated with any LUN identifier recorded on the business server side is dynamically updatable. For example, since distributed storage systems may experience events such as node failures, node additions, and disk failures, the OSD associated with a segment may change, which in turn may cause the segment's weight to change. Therefore, to avoid affecting the accuracy and reliability of path selection, it is necessary to periodically obtain the segment weights determined by the storage node side in order to dynamically update the set of paths recorded on the business server side.

[0046] Specifically, as an example, for any LUN identifier, the business server can periodically obtain the shortest path hit rate associated with the LUN identifier by connecting different paths of different storage nodes in the distributed storage system; if it is found that the shortest path hit rate returned by any storage node is lower than the preset hit rate threshold, the server returns to the step of issuing weight request instructions by connecting different paths of different storage nodes in the distributed storage system to obtain the weights of each segment associated with the LUN identifier again, so as to update the locally recorded set of paths associated with the LUN identifier.

[0047] In this embodiment, there are many ways to specifically implement obtaining the shortest path hit rate associated with the LUN identifier: For example, as an implementation, the business server periodically sends hit rate request instructions through different paths connecting different storage nodes in the distributed storage system. The hit rate request instruction carries the LUN identifier, which is used to instruct any storage node that receives the hit rate request instruction to return a hit rate response instruction carrying the shortest path hit rate associated with the LUN identifier. Then, the server receives the hit rate response instructions returned by each storage node in response to the hit rate request instruction, thus obtaining the shortest path hit rate associated with the LUN identifier.

[0048] For example, as another embodiment, by connecting different paths of different storage nodes in the distributed storage system, the shortest path hit rate associated with the LUN identifier can be received periodically and actively reported by each storage node. In this way, the shortest path hit rate associated with the LUN identifier can be obtained.

[0049] Optionally, in this embodiment, the shortest path hit rate associated with the LUN identifier under any storage node is determined based on the number of at least one IO requests received by the storage node for the LUN corresponding to the LUN identifier within a specified time period (denoted as the first number) and the number of times the OSD accessed by each IO request is the local OSD of the storage node (denoted as the first number). For example, the ratio of the first number to the first number can be used as the shortest path hit rate associated with the LUN identifier under the storage node.

[0050] The specified time period refers to the period from the end of the previous statistical period for the shortest path hit rate to the current time.

[0051] The specific implementations of the aforementioned weight request command, weight response command, hit rate request command, and hit rate response command are described below: In this embodiment, as an example, the aforementioned weight request command, weight response command, hit rate request command, and hit rate response command can be constructed using SCSI custom commands supported by the SCSI protocol standard. Here, SCSI custom commands refer to an extension function within the SCSI protocol standard. The SCSI Protocol Standards Committee (T10) has reserved a portion of opcodes for hardware manufacturers to implement their proprietary, non-standardized extension functions.

[0052] For example, see Figure 5 The request command format shown is a 16-byte × 8-bit structure, which may contain the following fields: Manufacturer-defined opcode field (corresponding to byte 0): The opcode can be customized to 0xC2; Sub-function command number field (corresponding to byte1): used to distinguish between two types of core sub-functions; among them, sub-function command number 0x01 indicates a weight request, used to obtain the weight of the Segment; sub-function 0x02 indicates a hit rate request, used to obtain the shortest path hit rate; Parameter fields (bytes 2-5): used to carry the LUN identifier; Data length field (corresponding to byte 6-byte 13): used to indicate the length of the data expected to be returned by this command; Flag field (corresponding to byte 14) and SCSI general control field (corresponding to byte 15): used to carry general SCSI protocol information such as command control and status feedback.

[0053] Based on the above Figure 5 The requested command format shown can be used to construct weight request commands and hit rate request commands.

[0054] For example, see Figure 6 The response command format shown may include the following fields: Vendor-defined version number field (corresponding to byte 0): Used to identify the protocol version of this custom response command; Manufacturer-defined status field (corresponding to byte1): used to report the command execution result, and can take the value of "success" or "failure"; Response payload length field (corresponding to byte2-byte5): used to indicate the length of the payload data; Payload data field (corresponding to byte6-byteN): Used to indicate payload data, such as the weights of each returned segment.

[0055] Based on the above Figure 6 The response command format shown can be used to construct weighted response commands and hit rate response commands.

[0056] The following describes the fully optimized path read / write method for distributed storage provided in this application embodiment from the perspective of the storage node: See Figure 7 , Figure 7 This is a flowchart illustrating another fully optimized path read / write method for distributed storage provided in an embodiment of this application. This method can be applied to any storage node in a distributed storage system; optionally, the distributed storage system can be, for example, a distributed block storage system. Figure 7 As shown, the process may include the following steps: Step 701: Receive a weight request instruction from the connected service server. The weight request instruction carries a LUN identifier.

[0057] Step 702: Return a weight response instruction to the business server, carrying the weights of each segment associated with the aforementioned LUN identifier.

[0058] In this embodiment, if the OSD of the object storage device associated with a segment is a local OSD of the storage node, then the weight of the segment is the first weight; if the OSD associated with the segment is not a local OSD of the storage node, then the weight of the segment is the second weight.

[0059] In this embodiment, the service server connects to different storage nodes in the distributed storage system through different paths. When the service server receives weight response instructions from each storage node, it determines the path set associated with the LUN identifier. This path set includes the weights of each segment associated with the LUN identifier carried in the weight response instructions returned by each storage node. Upon receiving an IO request for a target segment, the service server determines the target storage node based on the target path set associated with the LUN identifier of the target LUN to which the target segment belongs, and performs IO processing on the target storage node. The weight of the target segment returned by the target storage node in the target path set is the first weight.

[0060] This concludes the process. Figure 7 The process is shown below.

[0061] pass Figure 7 As can be seen from the process shown, in this embodiment of the application, by introducing segment weight information to characterize whether the OSD associated with each segment is local to the storage node, the business server is assisted in dynamically determining the optimal storage node for handling IO requests based on the latest segment weight information returned by each storage node. This means that the storage node where the OSD accessed by the IO request is located locally exists. This can fully adapt to the data distribution characteristics of distributed storage itself, such as data segmentation and OSD mapping, so as to give full play to the performance of the distributed storage system, thereby effectively reducing cross-node OSD access of IO requests, reducing node network jump overhead, and thus improving the performance of data processing such as data read and write.

[0062] The following is a summary of the above. Figure 7 The method shown will be described in further detail: In this embodiment, as one example, a storage node receives a hit rate request instruction periodically issued by a business server. The hit rate request instruction carries a LUN identifier. The storage node then returns a hit rate response instruction to the business server, carrying the shortest path hit rate associated with the LUN identifier. This allows the business server to trigger a weight request instruction to be issued through different paths connecting different storage nodes in the distributed storage system if it finds that the shortest path hit rate returned by any storage node is lower than a preset hit rate threshold after receiving the hit rate response instructions from each storage node. The weight request instruction carries the aforementioned LUN identifier.

[0063] In this embodiment, as another embodiment, the storage nodes periodically and proactively report the shortest path hit rate associated with the LUN identifier to the business server. This allows the business server to receive the shortest path hit rates proactively reported by each storage node. If the business server finds that the shortest path hit rate reported by any storage node is lower than a preset hit rate threshold, it will trigger the issuance of a weight request instruction through different paths connecting different storage nodes in the distributed storage system. This weight request instruction carries the aforementioned LUN identifier.

[0064] As for Figure 7 For the specific implementation of each step in the method shown, please refer to the above. Figure 2 The relevant descriptions of the method flow shown are not repeated here.

[0065] To facilitate understanding of the specific implementation process of the fully optimized path read / write method for distributed storage described above, specific examples are given below.

[0066] In this embodiment, as an example, the specific implementation process of the above-mentioned fully optimized path read / write method for distributed storage is as follows: After the business server successfully establishes path connections with different storage nodes in the distributed storage system, it will issue weight request instructions for each specified LUN through different paths connecting different storage nodes in the distributed storage system. The weight request instructions carry the LUN identifier of the specified LUN.

[0067] Upon receiving the aforementioned weight request instruction, any storage node will return a weight response instruction to the service server, carrying the weights of each segment associated with the aforementioned LUN identifier. Specifically, if the OSD associated with a segment is a local OSD of the storage node, the weight of that segment is the first weight; if the OSD associated with a segment is not a local OSD of the storage node, the weight of that segment is the second weight.

[0068] After receiving the weight response instructions returned by each storage node, the business server will determine the set of paths associated with the aforementioned LUN identifier; the set of paths includes the weights of each segment associated with the aforementioned LUN identifier carried in the weight response instructions returned by each storage node.

[0069] Subsequently, when the business server receives an IO request for the target segment, it determines the target storage node based on the set of target paths associated with the LUN identifier of the target LUN to which the target segment belongs. The server then issues the IO request through the path connecting to the target storage node, so that when the storage node receives the IO request, it processes the IO request based on the multiple OSDs to which the target segment is mapped in the distributed storage system. The weight of the target segment returned by the target storage node in the set of target paths is the first weight.

[0070] This completes the processing of the IO request.

[0071] Furthermore, in this embodiment, as an example, the service server can also periodically (e.g., every 12 hours or 24 hours) issue a hit rate request instruction to any specified LUN through different paths connecting different storage nodes in the distributed storage system; the hit rate request instruction carries the LUN identifier of the specified LUN.

[0072] Upon receiving the aforementioned hit rate request instruction, any storage node will return a hit rate response instruction to the business server, carrying the shortest path hit rate associated with the aforementioned LUN identifier.

[0073] After receiving the hit rate response instructions from each storage node, if the business server finds that the shortest path hit rate returned by any storage node is lower than the preset hit rate threshold, it will return to the above steps of issuing weight request instructions by connecting different paths of different storage nodes in the distributed storage system, in order to regain the weight of each segment associated with the above LUN identifier.

[0074] Alternatively, as another embodiment, each storage node can periodically calculate the weight of each segment associated with the LUN identifier of any given LUN, and actively report it to the connected service server.

[0075] After receiving the shortest path hit rate associated with the LUN identifier periodically and proactively reported by each storage node, if the business server finds that the shortest path hit rate returned by any storage node is lower than the preset hit rate threshold, it will return to the step of issuing weight request instructions by connecting different paths of different storage nodes in the distributed storage system, so as to regain the weight of each segment associated with the LUN identifier.

[0076] This concludes the description of the method provided in the embodiments of this application. The apparatus provided in the embodiments of this application will now be described: As an example, this embodiment also provides a fully optimized path read / write device for distributed storage. This device corresponds to... Figure 2 The method flow is shown below. For example, see [link to example]. Figure 8 As shown, the fully optimized path read / write device 800 of this distributed storage is applied to a business server, and includes: The distribution module 801 is used to distribute weight request instructions through different paths connecting different storage nodes in the distributed storage system. The weight request instruction carries a LUN identifier, which is used to instruct any storage node that receives the weight request instruction to return a weight response instruction carrying the weights of each segment associated with the LUN identifier. If the object storage device OSD associated with a segment is the local OSD of the storage node, then the weight of the segment is the first weight. If the OSD associated with the segment is not the local OSD of the storage node, then the weight of the segment is the second weight. The determination module 802 is used to determine the set of paths associated with the LUN identifier in response to the weight response instructions returned by each storage node; the set of paths includes the weights of each segment associated with the LUN identifier carried by the weight response instructions returned by each storage node. The processing module 803 is configured to, when receiving an input / output IO request for a target segment, determine a target storage node based on a set of target paths associated with the LUN identifier of the target LUN to which the target segment belongs, and perform IO processing on the target storage node; the weight of the target segment returned by the target storage node in the set of target paths is a first weight.

[0077] As one embodiment, the device further includes: The acquisition module is used to periodically obtain the shortest path hit rate associated with the LUN identifier by connecting different paths to different storage nodes in the distributed storage system; wherein, the shortest path hit rate associated with the LUN identifier under any storage node is determined based on the number of at least one IO request received by the storage node for the LUN corresponding to the LUN identifier within a specified time period, and the number of times the OSD accessed by each IO request is the local OSD of the storage node; the specified time period refers to the period from the end time of the previous statistical period of the shortest path hit rate to the current time; If the shortest path hit rate returned by any storage node is found to be lower than the preset hit rate threshold, then the step of issuing weight request instructions by connecting different paths of different storage nodes in the distributed storage system is returned.

[0078] As one embodiment, obtaining the shortest path hit rate associated with the LUN identifier includes: Periodically, hit rate request instructions are issued through different paths connecting different storage nodes in the distributed storage system; the hit rate request instruction carries the LUN identifier, which is used to instruct any storage node that receives the hit rate request instruction to return a hit rate response instruction carrying the shortest path hit rate associated with the LUN identifier; Receive the hit rate response instructions returned by each storage node in response to the hit rate request instruction.

[0079] As one embodiment, obtaining the shortest path hit rate associated with the LUN identifier includes: By connecting different paths of different storage nodes in the distributed storage system, the system receives the shortest path hit rate associated with the LUN identifier, which is periodically and actively reported by each storage node.

[0080] As one embodiment, performing IO processing on the target storage node includes: The IO request is sent through the path connecting the target storage node, so that when the storage node receives the IO request, it processes the IO request based on the multiple OSDs to which the target segment is mapped in the distributed storage system.

[0081] This concludes the process. Figure 8 Structural description of the device shown.

[0082] As another embodiment, this embodiment also provides another fully optimized path read / write device for distributed storage. This device corresponds to... Figure 7 The method flow is shown below. For example, see [link to example]. Figure 9 As shown, the fully optimized path read / write device 900 for distributed storage is applied to any storage node in a distributed storage system, and includes: The receiving module 901 is used to receive a weight request instruction sent by the connected service server. The weight request instruction carries a LUN identifier and returns a weight response instruction to the service server, which carries the weights of each segment associated with the LUN identifier. If the object storage device OSD associated with a segment is a local OSD of the storage node, the weight of the segment is the first weight; if the OSD associated with the segment is not a local OSD of the storage node, the weight of the segment is the second weight. The service server connects to different storage nodes in the distributed storage system via different paths. When the service server receives weight response instructions from each storage node, it determines the path set associated with the LUN identifier. The path set includes the weights of each segment associated with the LUN identifier carried in the weight response instructions returned by each storage node. When it receives an input / output IO request for a target segment, it determines the target storage node based on the target path set associated with the LUN identifier of the target LUN to which the target segment belongs, and performs IO processing on the target storage node. The weight of the target segment returned by the target storage node in the target path set is the first weight.

[0083] As one embodiment, the receiving module 901 is further configured to receive a hit rate request instruction periodically issued by the service server; the hit rate request instruction carries the LUN identifier; The service server returns a hit rate response instruction carrying the shortest path hit rate associated with the LUN identifier. This allows the service server to trigger a weight request instruction to be sent through different paths connecting different storage nodes in the distributed storage system if it finds that the shortest path hit rate returned by any storage node is lower than a preset hit rate threshold after receiving the hit rate response instructions from each storage node.

[0084] As one embodiment, the device further includes: The reporting module is used to periodically and proactively report the shortest path hit rate associated with the LUN identifier to the business server. This allows the business server to receive the shortest path hit rates proactively reported by each storage node. If the business server finds that the shortest path hit rate reported by any storage node is lower than a preset hit rate threshold, it will trigger a weight request instruction to be sent through different paths connecting different storage nodes in the distributed storage system.

[0085] This concludes the process. Figure 9 Structural description of the device shown.

[0086] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0087] Please see Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided as an exemplary embodiment of this application. The electronic device includes a processor and a computer-readable storage medium; the computer-readable storage medium stores a plurality of computer program instructions, which, when executed by the processor, implement the method disclosed in the above example of this application. Depending on the actual function of the electronic device, other hardware may also be included, which will not be elaborated further.

[0088] Based on the same concept as the above method, this application also provides a computer-readable storage medium storing a plurality of computer program instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0089] For example, the aforementioned computer-readable storage medium can be any electronic, magnetic, optical, or other physical storage device, such as: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid-state drive, any type of storage disk (such as optical disc, DVD, etc.), or similar storage media, or combinations thereof.

[0090] The above are merely preferred embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A fully optimized path read / write method for distributed storage, characterized in that, This method is applied to a business server that connects to different storage nodes in a distributed storage system via different paths. The method includes: Weight request instructions are issued by connecting different paths of different storage nodes in the distributed storage system. The weight request instruction carries a logical unit number (LUN) identifier, which is used to instruct any storage node that receives the weight request instruction to return a weight response instruction carrying the weights of each segment associated with the LUN identifier. If the object storage device OSD associated with a segment is the local OSD of the storage node, then the weight of the segment is the first weight. If the OSD associated with the segment is not the local OSD of the storage node, then the weight of the segment is the second weight. For each storage node's returned weight response instruction, determine the path set associated with the LUN identifier; the path set includes the weight of each segment associated with the LUN identifier carried in the weight response instruction returned by each storage node. When an input / output IO request for a target segment is received, a target storage node is determined based on the target path set associated with the LUN identifier of the target LUN to which the target segment belongs, and IO processing is performed on the target storage node; the weight of the target segment returned by the target storage node in the target path set is the first weight.

2. The method according to claim 1, characterized in that, The method further includes: The shortest path hit rate associated with the LUN identifier is obtained periodically by connecting different paths of different storage nodes in the distributed storage system. The shortest path hit rate associated with the LUN identifier under any storage node is determined based on the number of at least one IO request received by the storage node for the LUN corresponding to the LUN identifier within a specified time period, and the number of times the OSD accessed by each IO request is a local OSD of that storage node. The specified time period refers to the period from the end of the previous statistical period for the shortest path hit rate to the current time. If the shortest path hit rate returned by any storage node is found to be lower than the preset hit rate threshold, then the step of issuing weight request instructions by connecting different paths of different storage nodes in the distributed storage system is returned.

3. The method according to claim 2, characterized in that, The process of obtaining the shortest path hit rate associated with the LUN identifier includes: Periodically, hit rate request instructions are issued through different paths connecting different storage nodes in the distributed storage system; the hit rate request instruction carries the LUN identifier, which is used to instruct any storage node that receives the hit rate request instruction to return a hit rate response instruction carrying the shortest path hit rate associated with the LUN identifier; Receive the hit rate response instructions returned by each storage node in response to the hit rate request instruction.

4. The method according to claim 2, characterized in that, The process of obtaining the shortest path hit rate associated with the LUN identifier includes: By connecting different paths of different storage nodes in the distributed storage system, the system receives the shortest path hit rate associated with the LUN identifier, which is periodically and actively reported by each storage node.

5. The method according to claim 1, characterized in that, The step of performing IO processing on the target storage node includes: The IO request is sent through the path connecting the target storage node, so that when the storage node receives the IO request, it processes the IO request based on the multiple OSDs to which the target segment is mapped in the distributed storage system.

6. A fully optimized path read / write method for distributed storage, characterized in that, This method is applied to any storage node in a distributed storage system, and the method includes: The system receives a weight request instruction from the connected service server. The weight request instruction carries a logical unit number (LUN) identifier. The system then returns a weight response instruction to the service server, which carries the weights of each segment associated with the LUN identifier. If the object storage device OSD associated with a segment is a local OSD of this storage node, the weight of the segment is the first weight. If the OSD associated with the segment is not a local OSD of this storage node, the weight of the segment is the second weight. The service server connects to different storage nodes in the distributed storage system via different paths. When the service server receives weight response instructions from each storage node, it determines the path set associated with the LUN identifier. The path set includes the weights of each segment associated with the LUN identifier carried in the weight response instructions returned by each storage node. When it receives an input / output IO request for a target segment, it determines the target storage node based on the target path set associated with the LUN identifier of the target LUN to which the target segment belongs, and performs IO processing on the target storage node. The weight of the target segment returned by the target storage node in the target path set is the first weight.

7. The method according to claim 6, characterized in that, The method further includes: Receive hit rate request instructions periodically issued by the service server; the hit rate request instructions carry the LUN identifier; The service server returns a hit rate response instruction carrying the shortest path hit rate associated with the LUN identifier. This allows the service server to trigger a weight request instruction to be sent through different paths connecting different storage nodes in the distributed storage system if it finds that the shortest path hit rate returned by any storage node is lower than a preset hit rate threshold after receiving the hit rate response instructions from each storage node.

8. The method according to claim 6, characterized in that, The method further includes: The system periodically and proactively reports the shortest path hit rate associated with the LUN identifier to the business server. This allows the business server to receive the shortest path hit rates proactively reported by each storage node. If the business server finds that the shortest path hit rate reported by any storage node is lower than a preset hit rate threshold, it will trigger a weight request instruction to be issued through different paths connecting different storage nodes in the distributed storage system.

9. An electronic device, characterized in that, The electronic device includes: Processor; and A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 5 or 6 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 5 or 6 to 8.

Images