Distributed storage system processing method and device

Abstract

The invention relates to a distributed storage system processing method and device, and the method comprises the steps: obtaining a first node number of a target node pool and a second node number of management nodes in a distributed storage system under the condition that the security level of the target node pool in the distributed storage system needs to be converted from the node security level to the cabinet security level; and under the condition that the number of the first nodes is greater than or equal to the number of the second nodes, adjusting the distribution condition of the management nodes in the distributed storage system according to the number of the second nodes, the number of the cabinets in the target node pool and the number of third nodes of the management nodes deployed by the cabinets in the target node pool, therefore, the management nodes are distributed on the cabinets of the target node pool in a balanced manner. In this way, when the distributed storage system has the data security level upgrading requirement, upgrading of the data security level can be achieved.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a distributed storage system processing method and device. Background Art

[0002] Currently, distributed storage systems support two levels of data security, including node-level security and cabinet-level security.

[0003] In traditional technology, the data security level is usually determined in advance based on the number of deployed nodes and the reliability requirements of the distributed storage system. The placement of each node in the cabinet and the number of management nodes distributed in each cabinet, such as the number of ZooKeeper (ZK) nodes, are planned based on the determined data security level.

[0004] When the reliability of a distributed storage system needs to be improved or the number of nodes needs to be expanded significantly, the data security level needs to be upgraded from node-level security to cabinet-level security. Traditional technologies do not provide a feasible solution for upgrading the data security level. Summary of the invention

[0005] Based on this, it is necessary to provide a distributed storage system processing method and device to address the above technical issues, which can achieve data security level upgrade when the distributed storage system has a data security level upgrade requirement.

[0006] In a first aspect, the present application provides a distributed storage system processing method, the method comprising:

[0007] In a case where the security level of a target node pool in the distributed storage system needs to be converted from a node security level to a cabinet security level, obtaining a first number of nodes in the target node pool and a second number of management nodes in the distributed storage system;

[0008] When the number of the first nodes is greater than or equal to the number of the second nodes, the distribution of management nodes in the distributed storage system is adjusted according to the number of the second nodes, the number of cabinets in the target node pool, and the third number of management nodes deployed in each cabinet in the target node pool, so that the management nodes are evenly distributed on each cabinet in the target node pool.

[0009] In an embodiment of the present application, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, the first node number of the target node pool and the second node number of the management node in the distributed storage system are obtained; when the first node number is greater than or equal to the second node number, it means that all management nodes in the distributed storage system can be deployed in the target node pool; and then according to the second node number, the number of cabinets in the target node pool and the third node number of the management nodes deployed in each cabinet in the target node pool, the distribution of the management nodes in the distributed storage system is adjusted so that the management nodes are evenly distributed on each cabinet in the target node pool. In this way, when the distributed storage system has a data security level upgrade demand, the data security level can be upgraded so that the distribution of the management nodes in the distributed storage system meets the cabinet security level; and the management nodes are evenly distributed on each cabinet in the target node pool, even if an individual cabinet fails, it will only affect the normal operation of the management node on the failed cabinet, and will not cause the cluster to fail to operate normally.

[0010] In one of the embodiments, adjusting the distribution of management nodes in the distributed storage system according to the number of the second nodes, the number of cabinets in the target node pool, and the number of third nodes of management nodes deployed in each cabinet in the target node pool includes:

[0011] Determine the maximum and minimum number of the management nodes to be deployed in each cabinet in the target node pool according to the number of the second nodes and the number of cabinets in the target node pool;

[0012] The distribution of the management nodes in the distributed storage system is adjusted according to the maximum number, the minimum number, and the third node number of the management nodes deployed in each cabinet in the target node pool.

[0013] In an embodiment of the present application, by determining the maximum and minimum numbers of management nodes distributed on each cabinet and adjusting the number of management nodes deployed on each cabinet, the number of management nodes deployed on each cabinet is within the range of the minimum and maximum numbers, thereby achieving balanced distribution of management nodes on each cabinet of the target node pool.

[0014] In one of the embodiments, adjusting the distribution of management nodes in the distributed storage system according to the maximum number, the minimum number, and the third number of management nodes deployed in each cabinet in the target node pool includes:

[0015] In the case where the distribution situation is that all the management nodes in the distributed storage system are located in the target node pool, adjusting the distribution situation of the management nodes in the distributed storage system according to the maximum number, the minimum number, and the third node number of the management nodes deployed in each cabinet in the target node pool;

[0016] In the case where the distribution situation is that not all management nodes in the distributed storage system are located in the target node pool, after the management nodes located in other node pools in the distributed storage system are converted into non-management nodes, the distribution situation of the management nodes in the distributed storage system is adjusted according to the maximum number, the minimum number and the third node number of the management nodes deployed in each cabinet in the target node pool.

[0017] In an embodiment of the present application, depending on whether all management nodes are distributed in the target nodes, intra-pool adjustment or inter-pool adjustment is selected for the management nodes to achieve balanced distribution of the management nodes on each cabinet of the target node pool.

[0018] In one of the embodiments, adjusting the distribution of management nodes in the distributed storage system according to the maximum number, the minimum number, and the third number of management nodes deployed in each cabinet in the target node pool includes:

[0019] From the target node pool, select a first cabinet whose number of third nodes is greater than the maximum number and a second cabinet whose number of third nodes is less than the minimum number;

[0020] Convert a first number of management nodes in each first cabinet into non-management nodes; wherein the first number corresponding to each first cabinet is the difference between the third number of management nodes deployed in the first cabinet and the maximum number;

[0021] Convert a second number of non-managed nodes in each second cabinet into managed nodes; wherein the second number corresponding to each second cabinet is a difference between the minimum number and a third number of managed nodes deployed in the second cabinet.

[0022] In an embodiment of the present application, by adding management nodes to cabinets where the number of deployed management nodes is less than the minimum number, and removing redundant management nodes from cabinets where the number of deployed management nodes is greater than the maximum number, the number of management nodes deployed in each cabinet is within the range of the minimum number and the maximum number, thereby achieving a balanced distribution of management nodes on each cabinet in the target node pool.

[0023] In one of the embodiments, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, the method further includes:

[0024] Obtain each disk selection array of the disk pool associated with the target node pool, wherein each disk selection array includes a disk identifier of a disk for storing original business data and a disk identifier of a disk for storing duplicate business data of the original business data; and disks corresponding to different disk identifiers contained in each disk selection array are deployed on different cabinets;

[0025] According to the first data storage times of the disks corresponding to the different disk identifiers in the data storage scenarios based on the respective disk selection arrays, a source-side disk selection array and a target-side disk selection array are selected from the respective disk selection arrays;

[0026] The data of each disk corresponding to the source-side disk selection array is migrated to each disk corresponding to the target-side disk selection array.

[0027] In an embodiment of the present application, by determining a source-side disk selection array with a larger data storage capacity and a target-side disk selection array with a smaller data storage capacity according to the first data storage times of disks corresponding to different disk identifiers in data storage scenarios based on each disk selection array, and migrating data in the source-side disk selection array to the target-side disk selection array, balanced data storage in each cabinet is achieved, the load of each cabinet is balanced, and the business processing speed and utilization efficiency of each cabinet are improved.

[0028] In one of the embodiments, selecting a source-side disk selection array and a target-side disk selection array from each disk selection array according to the first data storage times of disks corresponding to different disk identifiers in data storage scenarios based on each disk selection array includes:

[0029] Determine, according to the first data storage times of the disks corresponding to the different disk identifiers in the data storage scenarios based on the disk selection arrays, the second data storage times of the cabinets where the disks corresponding to the different disk identifiers are located in the data storage scenarios based on the disk selection arrays;

[0030] The cabinet corresponding to the largest second data storage number is used as the source cabinet;

[0031] Use the disk corresponding to the largest first data storage number in the source cabinet as the source disk;

[0032] Selecting a source disk selection array from the disk selection array containing the disk identifier of the source disk;

[0033] A target-side disk selection array is selected from the disk selection array that does not include the disk identifiers of the disks in the source-side cabinet.

[0034] In the embodiment of the present application, by determining the source-side disk selection array with a larger data storage capacity and the target-side disk selection array with a smaller data storage capacity based on the first data storage times of the disks corresponding to different disk identifiers in the data storage scenario based on each disk selection array and the second data storage times of the cabinets where the disks corresponding to the different disk identifiers are located, and migrating the data in the source-side disk selection array to the target-side disk selection array, the data storage in each cabinet is balanced, the load of each cabinet is balanced, and the business processing speed and utilization efficiency of each cabinet are improved.

[0035] In one of the embodiments, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, the method further includes:

[0036] Creating a sorted array; wherein the number of elements in the sorted array is the number of the first nodes;

[0037] The node identifiers of the nodes in the target node pool are written into the sorted array; wherein the nodes corresponding to two adjacent node identifiers in the sorted array are located in adjacent cabinets, and the cabinet where the node corresponding to each node identifier is located is the cabinet in the target node pool where the cabinets corresponding to the written node identifiers are located have the least number of appearances; the written node identifier is the node identifier before the node identifier in the sorted array;

[0038] Generate a node domain linked list according to the arrangement order of each node identifier in the sorted array;

[0039] A node view is generated according to the node domain linked list and data redundancy; wherein the node view is used to indicate the node identifier of the node storing the original log data and the node identifier of the node storing the copy log data of the original log data.

[0040] In an embodiment of the present application, each node in the node view generated according to the node domain linked list and redundancy is deployed on a different cabinet, so that the nodes storing the same log data are deployed on different cabinets, realizing cabinet security level storage of log data and improving the security of data storage.

[0041] In one embodiment, the method further comprises:

[0042] In the case where there are abnormal nodes in the target node pool, node identifiers of the abnormal nodes in the sorted array are removed; wherein the abnormal nodes include faulty nodes and / or deleted nodes;

[0043] Generate a new node domain linked list according to the sorting order of each node identifier in the sorted array after elimination processing;

[0044] According to the new node domain linked list, the node view is updated to obtain a new node view.

[0045] In an embodiment of the present application, when there are abnormal nodes in the target node pool, the node identifier of the abnormal node can be removed from the sorted array, and a new node view can be further generated based on the sorting order of each node identifier in the sorted array after the removal process, thereby achieving cabinet security level storage of log data even in the presence of abnormal nodes.

[0046] In one embodiment, the method further comprises:

[0047] In the case where there is a newly added node in the target node pool, if there is a target view in which the cabinets to which the adjacent nodes belong are not adjacent in the new node view, determining whether the cabinet to which the newly added node belongs is adjacent to the cabinet to which any node in the target view belongs;

[0048] If the cabinet to which the newly added node belongs is adjacent to the cabinet to which the target node in the target view belongs, the node identifier of the newly added node is written into the node identifier of the target node in the sorted array to obtain a new sorted array;

[0049] If the cabinet to which the newly added node belongs is not adjacent to the cabinet to which any node in the target view belongs, the node identifier of the newly added node is written into the tail of the sorted array to obtain a new sorted array;

[0050] Generate a new node domain linked list according to the sorting order of each node identifier in the new sorting array;

[0051] The new node view is updated according to the new node domain linked list.

[0052] In an embodiment of the present application, by determining whether the newly added node belongs to the same cabinet as the abnormal node, and then determining the position in the sorted array where the new node identifier is written, it is achieved that in the presence of a new node, the nodes in the generated node view can still be deployed on different cabinets, thereby achieving cabinet security level storage of log data.

[0053] In a second aspect, the present application further provides a distributed storage system processing device, the device comprising:

[0054] A first acquisition module, configured to acquire a first number of nodes in the target node pool and a second number of nodes in the management node in the distributed storage system when the security level of the target node pool in the distributed storage system needs to be converted from a node security level to a cabinet security level;

[0055] An adjustment module is used to adjust the distribution of management nodes in the distributed storage system according to the second number of nodes, the number of cabinets in the target node pool, and the third number of management nodes deployed in each cabinet in the target node pool when the number of the first nodes is greater than or equal to the number of the second nodes, so that the management nodes are evenly distributed on each cabinet in the target node pool.

[0056] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any one of the distributed storage system processing methods described in the first aspect.

[0057] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the distributed storage system processing methods described in the first aspect is implemented.

[0058] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements any one of the distributed storage system processing methods described in the first aspect.

[0059] The above-mentioned distributed storage system processing method and device, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, obtain the first node number of the target node pool and the second node number of the management node in the distributed storage system; when the first node number is greater than or equal to the second node number, it means that all management nodes in the distributed storage system can be deployed in the target node pool; and then according to the second node number, the cabinet number of the target node pool and the third node number of the management nodes deployed in each cabinet in the target node pool, the distribution of the management nodes in the distributed storage system is adjusted so that the management nodes are evenly distributed on each cabinet in the target node pool. In this way, when the distributed storage system has a data security level upgrade requirement, the data security level can be upgraded so that the distribution of the management nodes in the distributed storage system meets the cabinet security level; and the management nodes are evenly distributed on each cabinet in the target node pool, even if an individual cabinet fails, it will only affect the normal operation of the management node on the failed cabinet, and will not cause the cluster to fail to operate normally. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of a process flow of a distributed storage system processing method in one embodiment;

[0061] Figure 2A schematic diagram of a process for adjusting the distribution of management nodes in an embodiment;

[0062] Figure 3 A schematic diagram of a flow chart for adjusting the distribution of management nodes in another embodiment;

[0063] Figure 4 A schematic diagram of a process for adjusting stored business data in one embodiment;

[0064] Figure 5 A schematic diagram of a process for determining a source-side disk selection array and a target-side disk selection array in one embodiment;

[0065] Figure 6 A schematic diagram of a process for generating a node view in an embodiment;

[0066] Figure 7 is a schematic diagram of a node view in one embodiment;

[0067] Figure 8 A schematic diagram of a process for generating a new node view in an embodiment;

[0068] Fig. 9 A schematic diagram of a new node view generated in one embodiment;

[0069] Fig.10 A schematic diagram of a process for generating a node view in another embodiment;

[0070] Fig.11 It is a flowchart of a distributed storage system processing method in another embodiment;

[0071] Fig.12 is a structural block diagram of a distributed storage system processing device in one embodiment;

[0072] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0074] The distributed storage system processing method provided in the embodiment of the present application can be applied to the application scenario of upgrading the data security level of the distributed storage system, and is specifically applied to the application scenario of upgrading the data security level of the distributed storage system from the node security level to the cabinet security level. The method provided in the embodiment of the present application can be executed by a management node in the distributed storage system, and the management node can be a server.

[0075] In one embodiment, Figure 1 As shown, a distributed storage system processing method is provided, and the method is applied to a management node of a distributed storage system as an example for description, including the following steps:

[0076] S101, when it is necessary to convert the security level of a target node pool in a distributed storage system from a node security level to a cabinet security level, obtain a first node number of the target node pool and a second node number of management nodes in the distributed storage system.

[0077] For example, the node security level can be understood as the original data and the replica data are stored on different nodes, and the nodes are deployed on the same cabinet; if the cabinet fails, all nodes on the cabinet cannot be used. The cabinet security level can be understood as the original data and the replica data are stored on different nodes, and the nodes are deployed on different cabinets. If the cabinet where one copy of the data is located fails, the data on the failed cabinet can be restored through the replica data deployed on other cabinets.

[0078] The target node pool is a node pool in the distributed storage system that needs to be converted from the node security level to the cabinet security level. When the security level of the target node pool in the distributed storage system needs to be converted from the node security level to the cabinet security level, each node in the target node pool can be deployed in each cabinet in advance and each cabinet can be placed in a suitable position. Then, in the management interface of the distributed storage system, the data security level of the target node pool can be upgraded from the node security level to the cabinet security level.

[0079] Furthermore, a first node number of nodes included in the target node pool and a second node number of management nodes in the distributed storage system may be obtained, wherein the management node may be a (ZooKeeper, ZK) node.

[0080] S102, when the number of first nodes is greater than or equal to the number of second nodes, the distribution of management nodes in the distributed storage system is adjusted according to the number of second nodes, the number of cabinets in the target node pool, and the third number of management nodes deployed in each cabinet in the target node pool, so that the management nodes are evenly distributed on each cabinet in the target node pool.

[0081] Furthermore, it is determined whether the number of first nodes is greater than or equal to the number of second nodes. When the number of first nodes is greater than or equal to the number of second nodes, it means that all management nodes in the distributed storage system can be deployed in the target node pool. Therefore, the distribution of management nodes in the distributed storage system can be adjusted according to the number of second nodes, the number of cabinets in the target node pool and the third number of management nodes deployed in each cabinet in the target node pool, so that the management nodes are evenly distributed on each cabinet in the target node pool, that is, on the one hand, the distribution of management nodes meets the cabinet safety level, and on the other hand, the management nodes deployed on each cabinet in the target node pool are as evenly distributed as possible.

[0082] It should be noted that if more than half of the management nodes in the distributed storage system fail, the distributed storage system will not work properly. If the management nodes are unevenly distributed on each cabinet, there may be a cabinet with more than or equal to the majority of management nodes. In this case, if the cabinet fails, the entire distributed storage system will not work properly. This situation can be avoided by evenly distributing the management nodes on each cabinet of the target node pool.

[0083] The above-mentioned distributed storage system processing method, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, obtains the first node number of the target node pool and the second node number of the management node in the distributed storage system; when the first node number is greater than or equal to the second node number, it means that all management nodes in the distributed storage system can be deployed in the target node pool; and then according to the second node number, the number of cabinets in the target node pool and the third node number of the management nodes deployed in each cabinet in the target node pool, the distribution of the management nodes in the distributed storage system is adjusted so that the management nodes are evenly distributed on each cabinet in the target node pool. In this way, when the distributed storage system has a data security level upgrade requirement, the data security level can be upgraded so that the distribution of the management nodes in the distributed storage system meets the cabinet security level; and the management nodes are evenly distributed on each cabinet in the target node pool, even if an individual cabinet fails, it will only affect the normal operation of the management node on the failed cabinet, and will not cause the cluster to fail to operate normally.

[0084] In some optional implementations, the present application embodiment provides an optional implementation of the above S102. Figure 2 , Figure 2 A schematic diagram of a process for adjusting the distribution of management nodes is provided, which specifically includes the following steps:

[0085] S201, determining the maximum number and minimum number of management nodes to be deployed in each cabinet in the target node pool according to the number of second nodes and the number of cabinets in the target node pool.

[0086] Exemplarily, in order to evenly distribute the management nodes on the cabinets in the target node pool, the ratio of the number of first nodes to the number of cabinets in the target node pool can be rounded up to obtain the maximum number; the ratio of the number of first nodes to the number of cabinets in the target node pool can be rounded down to obtain the minimum number. For example, if the total number of management nodes is 5 and the number of cabinets in the target node pool is 3, the maximum number is 5 / 3 rounded up, that is, the maximum number is 2, and the minimum number is 5 / 3 rounded down, that is, the minimum number is 1.

[0087] S202: Adjust the distribution of management nodes in the distributed storage system according to the maximum number, the minimum number, and the number of third nodes of the management nodes deployed in each cabinet in the target node pool.

[0088] Furthermore, the distribution of management nodes in the distributed storage system can be adjusted according to the maximum number, the minimum number, and the number of third nodes of management nodes deployed in each cabinet in the target node pool, so that the number of management nodes deployed on each cabinet is within the range of the minimum number and the maximum number. For example, if the number of third nodes of management nodes deployed in a cabinet is less than the minimum number, it is necessary to increase the number of management nodes deployed in the cabinet; if the number of third nodes of management nodes deployed in a cabinet is greater than the maximum number, the number of management nodes deployed in the cabinet is reduced; and finally, the number of management nodes deployed on each cabinet is within the range of the minimum number and the maximum number.

[0089] In an embodiment of the present application, by determining the maximum and minimum numbers of management nodes distributed on each cabinet and adjusting the number of management nodes deployed on each cabinet, the number of management nodes deployed on each cabinet is within the range of the minimum and maximum numbers, thereby achieving balanced distribution of management nodes on each cabinet of the target node pool.

[0090] It should be noted that all management nodes in the distributed storage system may be deployed in the target node pool, so that the distribution of management nodes in the target node pool can be adjusted within the pool; that is, when the distribution situation is that all management nodes in the distributed storage system are located in the target node pool, the distribution of management nodes in the distributed storage system can be adjusted according to the maximum number, minimum number and third node number of management nodes deployed in each cabinet in the target node pool.

[0091] In the case where the distribution situation is that not all management nodes in the distributed storage system are located in the target node pool, it is necessary to adjust the management nodes in each node pool where the management nodes are deployed between pools. That is, after the management nodes located in other node pools in the distributed storage system are converted to non-management nodes, the distribution of management nodes in the distributed storage system can be adjusted according to the maximum number, minimum number, and number of third nodes of management nodes deployed in each cabinet in the target node pool. It should be noted that the number of management nodes deployed in other node pools needs to be equal to the number of newly added management nodes in the target node pool, so that all management nodes in the distributed storage system can be deployed in the target node pool.

[0092] It is understandable that both intra-pool adjustment and inter-pool adjustment require that all management nodes be evenly distributed on the cabinets of the target node pool. The difference is that inter-pool migration requires that management nodes deployed outside the target node pool be converted to non-management nodes.

[0093] In an embodiment of the present application, depending on whether all management nodes are distributed in the target nodes, intra-pool adjustment or inter-pool adjustment is selected for the management nodes to achieve balanced distribution of the management nodes on each cabinet of the target node pool.

[0094] Furthermore, an embodiment of the present application provides another specific implementable method for adjusting the distribution of management nodes.

[0095] See also Figure 3 , Figure 3 Another flow chart for adjusting the distribution of management nodes is provided, which specifically includes the following steps:

[0096] S301 , selecting, from a target node pool, a first cabinet whose number of third nodes is greater than a maximum number and a second cabinet whose number of third nodes is less than a minimum number.

[0097] Exemplarily, from the target node pool, a first cabinet whose number of third nodes is greater than the maximum number is selected, that is, the first cabinet is a cabinet whose number of nodes of management nodes deployed in the target node pool is greater than the maximum number. And from the target node pool, a second cabinet whose number of third nodes is less than the minimum number is selected, that is, the second cabinet is a cabinet whose number of nodes of management nodes deployed in the target node pool is less than the minimum number.

[0098] S302: Convert a first number of managed nodes in each first cabinet into non-managed nodes.

[0099] Exemplarily, the first number corresponding to each first cabinet is the difference between the third number of management nodes deployed in the first cabinet and the maximum number. That is, it is determined that the number of management nodes deployed in each cabinet is greater than the first number of the maximum number, and then the first number of management nodes is selected in each first cabinet, and the first number of management nodes is converted into non-management nodes.

[0100] Optionally, after selecting a first number of management nodes in each first cabinet, the node identifiers of the selected management nodes can be placed in a list to be scaled down. After determining all the node identifiers that need to be converted from management nodes to non-management nodes, the management services in the management nodes corresponding to each node identifier can be uninstalled according to the node identifiers in the list to be scaled down, so that the management nodes corresponding to each node identifier are converted into non-management nodes.

[0101] S303: Convert a second number of non-management nodes in each second cabinet into management nodes.

[0102] Furthermore, the second number corresponding to each second cabinet is the difference between the minimum number and the third number of management nodes deployed in the second cabinet. That is, it is determined that the number of management nodes deployed in each cabinet is less than the second number of the minimum number, and then the second number of non-management nodes is selected in each second cabinet, and the second number of non-management nodes is converted into management nodes.

[0103] Optionally, after selecting a second number of management nodes in each second cabinet, the node identifiers of the selected non-management nodes can be placed in the list to be expanded. After determining all the node identifiers that need to be converted from non-management nodes to management nodes, management services are deployed for the non-management nodes corresponding to each node identifier based on the node identifiers in the list to be expanded, so that the non-management nodes corresponding to each node identifier are converted into management nodes.

[0104] Exemplarily, if the number of third nodes of the management nodes deployed in a cabinet in the target node pool is the maximum number or the minimum number, the number of management nodes deployed in the cabinet may not be adjusted.

[0105] In an embodiment of the present application, by adding management nodes to cabinets where the number of deployed management nodes is less than the minimum number, and removing redundant management nodes from cabinets where the number of deployed management nodes is greater than the maximum number, the number of management nodes deployed in each cabinet is within the range of the minimum number and the maximum number, thereby achieving a balanced distribution of management nodes on each cabinet in the target node pool.

[0106] In some optional implementations, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, the data stored in the target node pool can also be adjusted so that the stored data is evenly distributed in each cabinet to balance the load of each cabinet, thereby improving the business processing speed and utilization efficiency of each cabinet.

[0107] For example, see Figure 4 , Figure 4 A schematic diagram of a process for adjusting stored business data is provided, which specifically includes the following steps:

[0108] S401, obtaining each disk selection array of the disk pool associated with the target node pool.

[0109] Exemplarily, each disk selection array may include disk identifiers of disks used to store original business data and disk identifiers of disks used to store copy business data of the original business data; the disks corresponding to different disk identifiers contained in each disk selection array are deployed on different cabinets.

[0110] It should be noted that the target node pool can be associated with multiple disk pools, each disk pool can contain non-identical disks, and the disks are deployed on each node in the target node pool. The disk selection array corresponding to the disk pool can be determined based on the number of disks in the disk pool and the data redundancy; wherein the data redundancy is the number of backups of the original data and the replica data. For example, if the disk pool includes 6 disks and the data redundancy is 3, that is, the number of backups of the original data and the replica data is 3, then each disk selection array includes the disk IDs of 3 disks, and the disks corresponding to each disk ID are deployed in different cabinets. In this way, the original data and the replica data can be stored on different cabinets, and the data security level can be achieved at the cabinet security level.

[0111] Based on this, each disk selection array of the disk pool associated with the node pool can be obtained. The disks corresponding to the disk identifiers included in each disk selection array are disks in the disk pool and are deployed on different cabinets.

[0112] S402: Select a source-side disk selection array and a target-side disk selection array from each disk selection array according to the first data storage times of disks corresponding to different disk identifiers in data storage scenarios based on each disk selection array.

[0113] It should be noted that different disk identifiers may appear in different disk selection arrays, and the disks corresponding to the disk identifiers in the same disk selection array usually store more than one copy of the business data, thereby determining the first data storage times of the disks corresponding to different disk identifiers in the data storage scenarios based on each disk selection array, wherein the data storage scenarios based on each disk selection array refer to the scenarios where the original business data and the copy business data are stored according to the disk identifiers in the disk selection array. The first data storage times of the disks corresponding to different disk identifiers in the data storage scenarios based on each disk selection array can be understood as the total data storage times of the disks corresponding to different disk identifiers in the scenarios where the data is stored according to the disk identifiers in the disk selection array.

[0114] Exemplarily, assume that the disk selection arrays corresponding to the disk pool are {1,2,3}, {1,4,6}, {1,6,9}, {5,7,9}, {2,7,9}, and {3,4,6}, where 1-9 represent disk IDs, and the disks corresponding to disk ID 1, disk ID 5, and disk ID 8 are deployed on the same cabinet, the disks corresponding to disk ID 2 and disk ID 6 are deployed on the same cabinet, the disks corresponding to disk ID 3 and disk ID 7 are deployed on the same cabinet, the disks corresponding to disk ID 4 and disk ID 9 are deployed on the same cabinet, and the disks corresponding to disk ID 1, disk ID 2, disk ID 3, and disk ID 4 are deployed on different cabinets. Assume that in a scenario where data is stored based on each disk selection array, the number of times data is stored using the disk selection arrays {1,2,3}, {1,4,6}, {1,6,9}, {5,7,9}, {2,7,9}, and {3,4,6} are 2 times, 1 time, 1 time, 2 times, 0 times, and 0 times, respectively. Then the first data storage times of the disks corresponding to different disk identifiers in the data storage scenarios based on each disk selection array can be understood as follows: the first data storage times of the disks corresponding to disk identifier 1 in the data storage scenarios based on each disk selection array are 4, the first data storage times of the disks corresponding to disk identifier 2 in the data storage scenarios based on each disk selection array are 2, the first data storage times of the disks corresponding to disk identifier 3 in the data storage scenarios based on each disk selection array are 2, the first data storage times of the disks corresponding to disk identifier 4 in the data storage scenarios based on each disk selection array are 1, the first data storage times of the disks corresponding to disk identifier 5 in the data storage scenarios based on each disk selection array are 2, the first data storage times of the disks corresponding to disk identifier 6 in the data storage scenarios based on each disk selection array are 2, the first data storage times of the disks corresponding to disk identifier 7 in the data storage scenarios based on each disk selection array are 2, the first data storage times of the disks corresponding to disk identifier 8 in the data storage scenarios based on each disk selection array are 0, and the first data storage times of the disks corresponding to disk identifier 9 in the data storage scenarios based on each disk selection array are 3.

[0115] Then, according to the first data storage times of the disks corresponding to different disk identifiers in the data storage scenario based on each disk selection array, a source disk selection array and a target disk selection array can be selected from each disk selection array. For example, the disk selection array containing the disk identifier with the largest number of first data storage times can be used as the source, and the disk selection array containing the disk identifier with the smallest number of first data storage times can be used as the target.

[0116] S403, migrating the data of each disk corresponding to the disk selection array at the source end to each disk corresponding to the disk selection array at the target end.

[0117] Furthermore, the data of each disk corresponding to the source-side disk selection array can be migrated to each disk corresponding to the target-side disk selection array, so as to achieve more balanced data storage in each cabinet.

[0118] In the embodiment of the present application, by determining a source-side disk selection array with a larger data storage capacity and a target-side disk selection array with a smaller data storage capacity, and migrating the data in the source-side disk selection array to the target-side disk selection array, the data storage in each cabinet is balanced, the load of each cabinet is balanced, and the business processing speed and utilization efficiency of each cabinet are improved.

[0119] Further, see Figure 5 , Figure 5 A schematic diagram of a process for determining a source end disk selection array and a target end disk selection array is provided, which specifically includes the following steps:

[0120] S501, determining a second data storage number of a cabinet where disks corresponding to different disk identifiers are located in a data storage scenario based on each disk selection array according to a first data storage number of disks corresponding to different disk identifiers in a data storage scenario based on each disk selection array.

[0121] For example, in a scenario where data storage is performed based on each disk selection array, the second data storage times of the cabinets where disks corresponding to different disk identifiers are located can be understood as: the sum of the total data storage times of the disks corresponding to each disk identifier in the scenario where data storage is performed according to the disk identifiers in the disk selection array.

[0122] Exemplarily, taking the above embodiment as an example, it is assumed that the disk selection arrays corresponding to the disk pool are {1,2,3}, {1,4,6}, {1,6,9}, {5,7,9}, {2,7,9} and {3,4,6}; and the disk corresponding to the disk identifier 1 has a first data storage number of 4 in the data storage scenarios based on each disk selection array, the disk corresponding to the disk identifier 2 has a first data storage number of 2 in the data storage scenarios based on each disk selection array, the disk corresponding to the disk identifier 3 has a first data storage number of 2 in the data storage scenarios based on each disk selection array, and the disk corresponding to the disk identifier 4 has a first data storage number of 4 in the data storage scenarios based on each disk selection array. The first data storage number of the row data storage scenario is 1, the first data storage number of the disk corresponding to the disk identifier 5 in the data storage scenario based on each disk selection array is 2, the first data storage number of the disk corresponding to the disk identifier 6 in the data storage scenario based on each disk selection array is 2, the first data storage number of the disk corresponding to the disk identifier 7 in the data storage scenario based on each disk selection array is 2, the first data storage number of the disk corresponding to the disk identifier 8 in the data storage scenario based on each disk selection array is 0, and the first data storage number of the disk corresponding to the disk identifier 9 in the data storage scenario based on each disk selection array is 3.

[0123] Assume that the disks corresponding to disk ID 1, disk ID 5, and disk ID 8 are deployed in the same cabinet A, the disks corresponding to disk ID 2 and disk ID 6 are deployed in the same cabinet B, the disks corresponding to disk ID 3 and disk ID 7 are deployed in the same cabinet C, and the disks corresponding to disk ID 4 and disk ID 9 are deployed in the same cabinet D. Then, in the data storage scenario based on each disk selection array, the second data storage times of cabinet A where the disks corresponding to disk ID 1, disk ID 5, and disk ID 8 are located is 4+2+0; in the data storage scenario based on each disk selection array, the second data storage times of cabinet B where the disks corresponding to disk ID 2 and disk ID 6 are located is 2+2; in the data storage scenario based on each disk selection array, the second data storage times of cabinet C where the disks corresponding to disk ID 3 and disk ID 7 are located is 2+2; in the data storage scenario based on each disk selection array, the second data storage times of cabinet D where the disks corresponding to disk ID 4 and disk ID 9 are located is 1+3.

[0124] S502: Use the cabinet corresponding to the maximum second data storage times as the source cabinet.

[0125] Illustratively, in the above embodiment, the cabinet corresponding to the maximum second data storage number is cabinet A, and cabinet A is used as the source cabinet.

[0126] S503: Use the disk corresponding to the largest first data storage number in the source cabinet as the source disk.

[0127] Furthermore, the disk corresponding to the largest first data storage number in the source cabinet A is disk 1, and disk 1 is used as the source disk.

[0128] S504, selecting a source disk selection array from the disk selection array containing the disk identifier of the source disk.

[0129] Furthermore, the disk selection arrays containing the disk identifiers of the source disks are {1,2,3}, {1,4,6}, and {1,6,9}, and any one of {1,2,3}, {1,4,6}, and {1,6,9} can be selected as the source disk selection array. For example, {1,2,3} is selected as the source disk selection array.

[0130] S505 , selecting a target-side disk selection array from the disk selection array that does not include the disk identifiers of the disks in the source-side cabinet.

[0131] For example, the disk selection arrays that do not include the disk identifiers of the disks in the source cabinet are {2,7,9} and {3,4,6}, and any of {2,7,9} and {3,4,6} can be used as the target disk selection array to migrate the data of the disks corresponding to the source disk selection array to the disks corresponding to the target disk selection array, thereby balancing the load of data storage on the source cabinet.

[0132] In an embodiment of the present application, by determining a source-side disk selection array with a larger data storage capacity and a target-side disk selection array with a smaller data storage capacity, and migrating the data in the source-side disk selection array to the target-side disk selection array, the data storage in each cabinet is balanced, the load of each cabinet is balanced, and the business processing speed and utilization efficiency of each cabinet are improved.

[0133] In some optional implementations, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, the nodes storing the log data can also be updated so that the nodes storing the log data are also deployed in different cabinets, thereby achieving that the log data is also stored according to the cabinet security level.

[0134] For example, see Figure 6 , Figure 6 A schematic diagram of a process for generating a node view is provided, which specifically includes the following steps:

[0135] S601, create a sorted array.

[0136] Exemplarily, a sorted array having the same number of elements as the number of nodes in the target node pool may be created, that is, the number of elements in the sorted array is the number of first nodes; the sorted array is used to store the node identifier of each node in the target node pool.

[0137] S602, writing the node identifiers of the nodes in the target node pool into a sorted array.

[0138] Exemplarily, you can first select any node in the target node pool, and write the node identifier of the node into the first element of the sorted array. Then determine whether there are any nodes that have not been written into the sorted array. If so, first determine whether there are any nodes that have not been written into the sorted array in the adjacent cabinets adjacent to the cabinet to which the node corresponding to the last node identifier written into the sorted array belongs. If there are nodes that have not been written into the sorted array in the adjacent cabinets, then select the cabinet with the smallest degree of duplication of the cabinets to which the nodes corresponding to the node identifiers written into the sorted array belong from the adjacent cabinets, and write the node identifier of any node in the cabinet that has not been written into the sorted array after the last node identifier in the sorted array. Until there are no node identifiers that have not been written into the sorted array, the writing of all node identifiers is completed.

[0139] By adopting this method of writing into the sorted array, the nodes corresponding to two adjacent node identifiers in the sorted array are located in adjacent cabinets, and the cabinet where the node corresponding to each node identifier is located is the cabinet in the target node pool where the nodes corresponding to the written node identifier are located appear the least number of times; the written node identifier is the node identifier before the node identifier in the sorted array.

[0140] S603, generating a node domain linked list according to the arrangement order of each node identifier in the sorting array.

[0141] Furthermore, a node domain linked list may be generated according to the arrangement order of the node identifiers in the sorted array. For example, if the arrangement order of the node identifiers in the sorted array is 1, 6, 4, 2, 5, and 3, the order of the generated node domain linked list is 1->6->4->2->5->3.

[0142] S604, generating a node view according to the node domain linked list and data redundancy.

[0143] Furthermore, the data redundancy is the total number of original log data and duplicate log data of the same log data. Thus, a node view can be generated according to the node domain linked list and the data redundancy. The node view is used to indicate the node identifier of the node storing the original log data and the node identifier of the node storing the duplicate log data of the original log data.

[0144] For example, see Figure 7 , Figure 7A schematic diagram of a node view is provided. The node view is generated based on the node domain linked list. That is, node 1 and node 2 are deployed in cabinet cab1, node 3 and node 4 are deployed in cabinet cab2, and node 5 and node 6 are deployed in cabinet cab3. The node domain linked list generated in the above manner can make each node in each node view deployed on a different cabinet, and then make the original data and copy data of the log data stored on different cabinets, realizing the cabinet security level of the data.

[0145] In an embodiment of the present application, each node in the node view generated according to the node domain linked list and redundancy is deployed on a different cabinet, so that the nodes storing the same log data are deployed on different cabinets, realizing cabinet security level storage of log data and improving the security of data storage.

[0146] In some optional implementations, a faulty node may appear in the target node pool, or a node may be removed. In this case, it is necessary to regenerate the node domain linked list and regenerate the node view based on the newly generated node domain linked list to indicate the node identifier of the node used to store the original log data and the node identifier of the node storing the copy log data of the original log data.

[0147] For example, see Figure 8 , Figure 8 A schematic diagram of a process for generating a new node view is provided, which specifically includes the following steps:

[0148] S801: When there are abnormal nodes in the target node pool, node identifiers of the abnormal nodes in the sorting array are removed.

[0149] Exemplarily, when there are abnormal nodes in the target node pool, the node identifiers of the abnormal nodes in the sorting array may be removed, and the arrangement order of other node identifiers remains unchanged. The abnormal nodes include faulty nodes or deleted nodes, or faulty nodes and deleted nodes.

[0150] S802, generating a new node domain linked list according to the sorting order of each node identifier in the sorted array after the elimination process.

[0151] Exemplarily, a new node domain linked list is generated according to the sorting order of each node identifier in the sorted array after elimination processing. For example, if node 3 is an abnormal node, the new node domain linked list is 1->6->4->2->5.

[0152] S803: Update the node view according to the new node domain linked list to obtain a new node view.

[0153] Furthermore, the node view can be updated according to the new node domain linked list to obtain a new node view. For example, the generated new node view is as follows: Fig. 9 As shown, Fig. 9 It shows the situation that node 3 is a faulty node, and the node views of the remaining nodes are generated according to the new node view and data redundancy.

[0154] In an embodiment of the present application, when there are abnormal nodes in the target node pool, the node identifier of the abnormal node can be removed from the sorted array, and a new node view can be further generated based on the sorting order of each node identifier in the sorted array after the removal process, thereby achieving cabinet security level storage of log data even in the presence of abnormal nodes.

[0155] Furthermore, if a newly added node exists after an abnormal node exists in the target node pool, the newly added node may be a faulty node that has been added back, or may be a newly added node that has never appeared before.

[0156] Based on this, see Fig.10 , Fig.10 Another process diagram for generating a node view is provided, which specifically includes the following steps:

[0157] S1001, when there is a newly added node in the target node pool, if there is a target view in which the cabinets to which the adjacent nodes belong are not adjacent in the new node view, determine whether the cabinet to which the newly added node belongs is adjacent to the cabinet to which any node in the target view belongs.

[0158] It should be noted that if there have been no abnormal nodes in the target node pool, the cabinets to which adjacent nodes belong in the node view should be adjacent. Only when there have been abnormal nodes and the node identifiers of the nodes have been removed from the sorted array will the cabinets to which adjacent nodes belong in the node view be non-adjacent. Therefore, when there are new nodes in the target node pool, it can be first determined whether there is a target view in which the cabinets to which adjacent nodes belong are non-adjacent in the new node view. If so, it is determined whether the cabinet to which the new node belongs is adjacent to the cabinet to which any node in the target view belongs.

[0159] S1002: If the cabinet to which the newly added node belongs is adjacent to the cabinet to which the target node in the target view belongs, write the node identifier of the newly added node into the node identifier of the target node in the sorted array to obtain a new sorted array.

[0160] Furthermore, if the cabinet to which the newly added node belongs is adjacent to the cabinet to which the target node in the target view belongs, indicating that the newly added node is the cabinet where the abnormal node is located, the node identifier of the newly added node can be written after the node identifier of the target node in the sorted array to obtain a new sorted array.

[0161] S1003: If the cabinet to which the newly added node belongs is not adjacent to the cabinet to which any node in the target view belongs, write the node identifier of the newly added node into the tail of the sorted array to obtain a new sorted array.

[0162] Optionally, if the cabinet to which the newly added node belongs is not adjacent to the cabinet to which any node in the target view belongs, indicating that the newly added node is not the cabinet where the abnormal node is located, the node identifier of the newly added node is written to the end of the sorted array to obtain a new sorted array.

[0163] S1004, generating a new node domain linked list according to the sorting order of each node identifier in the new sorting array.

[0164] Exemplarily, the node identifier of the newly added node may be added after the node identifier of the target node, or added to the end of the sorted array, and a new node domain linked list may be generated according to the sorting order of the node identifiers in the new sorted array.

[0165] S1005: Update the new node view according to the new node domain linked list.

[0166] Furthermore, the new node view can be updated according to the new node and the linked list, so that the new node view can still store the log data according to the cabinet security level.

[0167] In an embodiment of the present application, by determining whether the newly added node belongs to the same cabinet as the abnormal node, and then determining the position in the sorted array where the new node identifier is written, it is achieved that in the presence of a new node, the nodes in the generated node view can still be deployed on different cabinets, thereby achieving cabinet security level storage of log data.

[0168] Illustratively, an embodiment of the present application provides that in the case where the security level of a target node pool in a distributed storage system needs to be converted from a node security level to a cabinet security level, adjustment of management nodes, migration of storage data, and update of node views can be performed simultaneously to complete the upgrade of the data security level of the target node pool from the node security level to the cabinet security level.

[0169] Based on this, see Fig.11 , Fig.11 A flow chart of another distributed storage system processing method is provided, which specifically includes the following steps:

[0170] S11011, obtaining the first node quantity of the target node pool and the second node quantity of the management node in the distributed storage system.

[0171] S11012: When the number of first nodes is greater than or equal to the number of second nodes, determine the maximum number and minimum number of management nodes deployed in each cabinet in the target node pool according to the number of second nodes and the number of cabinets in the target node pool.

[0172] S11013: Adjust the distribution of management nodes in the distributed storage system according to the maximum number, the minimum number, and the number of third nodes of the management nodes deployed in each cabinet in the target node pool.

[0173] S11021, obtaining each disk selection array of the disk pool associated with the target node pool.

[0174] S11022, according to the first data storage times of the disks corresponding to the different disk identifiers in the data storage scenarios based on the disk selection arrays, determine the second data storage times of the cabinets where the disks corresponding to the different disk identifiers are located in the data storage scenarios based on the disk selection arrays.

[0175] S11023: Use the cabinet corresponding to the maximum second data storage times as the source cabinet.

[0176] S11024: Use the disk corresponding to the largest first data storage number in the source cabinet as the source disk.

[0177] S11025, select a source disk selection array from the disk selection array containing the disk identifier of the source disk.

[0178] S11026: Select a target-side disk selection array from the disk selection array that does not include the disk identifiers of the disks in the source-side cabinet.

[0179] S11027, migrate the data of each disk corresponding to the disk selection array on the source side to each disk corresponding to the disk selection array on the target side.

[0180] S11031, create a sorted array.

[0181] S11032, write the node identifier of the node in the target node pool into the sorted array.

[0182] S11033, generating a node domain linked list according to the arrangement order of each node identifier in the sorting array.

[0183] S11034, generate a node view according to the node domain linked list and data redundancy.

[0184] S11035: When there are abnormal nodes in the target node pool, the node identifiers of the abnormal nodes in the sorting array are removed.

[0185] S11036, generating a new node domain linked list according to the sorting order of each node identifier in the sorted array after the elimination process.

[0186] S11037, update the node view according to the new node domain linked list to obtain a new node view.

[0187] S11038, when there is a newly added node in the target node pool, if there is a target view in which the cabinets to which the adjacent nodes belong are not adjacent in the new node view, determine whether the cabinet to which the newly added node belongs is adjacent to the cabinet to which any node in the target view belongs.

[0188] S11039, if the cabinet to which the newly added node belongs is adjacent to the cabinet to which the target node in the target view belongs, then the node identifier of the newly added node is written into the node identifier of the target node in the sorted array to obtain a new sorted array; if the cabinet to which the newly added node belongs is not adjacent to the cabinet to which any node in the target view belongs, then the node identifier of the newly added node is written into the tail of the sorted array to obtain a new sorted array.

[0189] S110310, generating a new node domain linked list according to the sorting order of each node identifier in the new sorting array.

[0190] S110311, update the new node view according to the new node domain linked list.

[0191] The specific process of the above S110101 to S110311 can be found in the description of the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0192] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0193] Based on the same inventive concept, the embodiment of the present application also provides a distributed storage system processing device for implementing the above-mentioned distributed storage system processing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the one or more distributed storage system processing device embodiments provided below can refer to the above-mentioned limitations on the distributed storage system processing method, and will not be repeated here.

[0194] In one embodiment, Fig.12 As shown, a distributed storage system processing device is provided, including:

[0195] A first acquisition module 10 is used to acquire a first number of nodes in the target node pool and a second number of nodes of the management node in the distributed storage system when the security level of the target node pool in the distributed storage system needs to be converted from a node security level to a cabinet security level;

[0196] The adjustment module 20 is used to adjust the distribution of management nodes in the distributed storage system according to the number of second nodes, the number of cabinets in the target node pool, and the number of third nodes of management nodes deployed in each cabinet in the target node pool when the number of first nodes is greater than or equal to the number of second nodes, so that the management nodes are evenly distributed on each cabinet in the target node pool.

[0197] The above-mentioned distributed storage system processing device, when it is necessary to convert the security level of the target node pool in the distributed storage system from the node security level to the cabinet security level, obtains the first node number of the target node pool and the second node number of the management node in the distributed storage system; when the first node number is greater than or equal to the second node number, it means that all management nodes in the distributed storage system can be deployed in the target node pool; and then according to the second node number, the cabinet number of the target node pool and the third node number of the management nodes deployed in each cabinet in the target node pool, the distribution of the management nodes in the distributed storage system is adjusted so that the management nodes are evenly distributed on each cabinet in the target node pool. In this way, when the distributed storage system has a data security level upgrade requirement, the data security level can be upgraded so that the distribution of the management nodes in the distributed storage system meets the cabinet security level; and the management nodes are evenly distributed on each cabinet in the target node pool, even if an individual cabinet fails, it will only affect the normal operation of the management node on the failed cabinet, and will not cause the cluster to fail to operate normally.

[0198] In one embodiment, the adjustment module 20 specifically includes:

[0199] A determination unit, used to determine the maximum number and minimum number of management nodes deployed in each cabinet in the target node pool according to the number of second nodes and the number of cabinets in the target node pool;

[0200] The adjustment unit is used to adjust the distribution of management nodes in the distributed storage system according to the maximum number, the minimum number and the number of third nodes of the management nodes deployed in each cabinet in the target node pool.

[0201] In one embodiment, the adjustment unit specifically includes:

[0202] A first adjustment subunit is used to adjust the distribution of management nodes in the distributed storage system according to the maximum number, the minimum number, and the third node number of management nodes deployed in each cabinet in the target node pool when the distribution situation is that all management nodes in the distributed storage system are located in the target node pool;

[0203] The second adjustment sub-unit is used to adjust the distribution of management nodes in the distributed storage system according to the maximum number, the minimum number and the third number of management nodes deployed in each cabinet in the target node pool after converting the management nodes located in other node pools in the distributed storage system into non-management nodes when the distribution situation is that not all management nodes in the distributed storage system are located in the target node pool.

[0204] In one embodiment, the second adjustment subunit is specifically used for:

[0205] From the target node pool, select a first cabinet in which the number of third nodes is greater than the maximum number, and a second cabinet in which the number of third nodes is less than the minimum number; convert a first number of management nodes in each first cabinet into non-management nodes; wherein the first number corresponding to each first cabinet is the difference between the third number of management nodes deployed in the first cabinet and the maximum number; convert a second number of non-management nodes in each second cabinet into management nodes; wherein the second number corresponding to each second cabinet is the difference between the minimum number and the third number of management nodes deployed in the second cabinet.

[0206] In one embodiment, the device further comprises:

[0207] A second acquisition module, in the case where the security level of a target node pool in a distributed storage system needs to be converted from a node security level to a cabinet security level, the second acquisition module is used to acquire each disk selection array of a disk pool associated with the target node pool, wherein each disk selection array includes a disk identifier of a disk for storing original business data and a disk identifier of a disk for storing a copy of the original business data; and disks corresponding to different disk identifiers contained in each disk selection array are deployed on different cabinets;

[0208] A selection module is used to select a source-side disk selection array and a target-side disk selection array from each disk selection array according to the first data storage times of disks corresponding to different disk identifiers in the data storage scenario based on each disk selection array;

[0209] The migration module is used to migrate the data of each disk corresponding to the source end disk selection array to each disk corresponding to the target end disk selection array.

[0210] In one embodiment, the selection module is specifically used to:

[0211] Determine the second data storage number of the cabinet where the disks corresponding to different disk identifiers are located in the data storage scenario based on each disk selection array according to the first data storage number of the disks corresponding to different disk identifiers in the data storage scenario based on each disk selection array; take the cabinet corresponding to the largest second data storage number as the source cabinet; take the disk corresponding to the largest first data storage number in the source cabinet as the source disk; select the source disk selection array from the disk selection array that includes the disk identifiers of the source disks; select the target disk selection array from the disk selection array that does not include the disk identifiers of each disk in the source cabinet.

[0212] In one embodiment, the device further comprises:

[0213] A creation module, where, when it is necessary to convert the security level of a target node pool in a distributed storage system from a node security level to a cabinet security level, the creation module is used to create a sorted array; wherein the number of elements in the sorted array is the number of first nodes;

[0214] A writing module is used to write the node identifiers of the nodes in the target node pool into the sorted array; wherein the nodes corresponding to two adjacent node identifiers in the sorted array are located in adjacent cabinets, and the cabinet where the node corresponding to each node identifier is located is the cabinet in the target node pool where the cabinets corresponding to the written node identifier are located have the least number of appearances; the written node identifier is the node identifier before the node identifier in the sorted array;

[0215] A generation module is used to generate a node domain linked list according to the arrangement order of each node identifier in the sorted array; generate a node view according to the node domain linked list and data redundancy; wherein the node view is used to indicate the node identifier of the node storing the original log data and the node identifier of the node storing the copy log data of the original log data.

[0216] In one embodiment, the generating module is further configured to:

[0217] When there are abnormal nodes in the target node pool, the node identifiers of the abnormal nodes in the sorted array are removed; wherein the abnormal nodes include faulty nodes and / or deleted nodes; a new node domain linked list is generated according to the sorting order of each node identifier in the sorted array after the removal process; and the node view is updated according to the new node domain linked list to obtain a new node view.

[0218] In one embodiment, the generating module is further configured to:

[0219] In the case where there is a newly added node in the target node pool, if there is a target view in which the cabinets to which the adjacent nodes belong are not adjacent in the new node view, determine whether the cabinet to which the newly added node belongs is adjacent to the cabinet to which any node in the target view belongs; if the cabinet to which the newly added node belongs is adjacent to the cabinet to which the target node in the target view belongs, write the node identifier of the newly added node into the node identifier of the target node in the sorted array to obtain a new sorted array; if the cabinet to which the newly added node belongs is not adjacent to the cabinet to which any node in the target view belongs, write the node identifier of the newly added node to the end of the sorted array to obtain a new sorted array; generate a new node domain linked list according to the sorting order of the node identifiers in the new sorted array; and update the new node view according to the new node domain linked list.

[0220] Each module in the above-mentioned distributed storage system processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0221] In one embodiment, a computer device is provided. The computer device may be a management node (server). The internal structure diagram thereof may be as follows: Fig.13 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data in a node pool. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a distributed storage system processing method is implemented.

[0222] Those skilled in the art will understand that Fig.13The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0223] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the distributed storage system processing method described in any of the above embodiments are implemented.

[0224] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the distributed storage system processing method described in any of the above embodiments are implemented.

[0225] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the distributed storage system processing method described in any of the above embodiments.

[0226] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0227] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0228] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0229] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A distributed storage system processing method, characterized in that: The method comprises: In a case where the security level of a target node pool in the distributed storage system needs to be converted from a node security level to a cabinet security level, obtaining a first number of nodes in the target node pool and a second number of management nodes in the distributed storage system; When the number of the first nodes is greater than or equal to the number of the second nodes, the distribution of management nodes in the distributed storage system is adjusted according to the number of the second nodes, the number of cabinets in the target node pool, and the third number of management nodes deployed in each cabinet in the target node pool, so that the management nodes are evenly distributed on each cabinet in the target node pool.

2. The method according to claim 1, characterized in that The adjusting the distribution of the management nodes in the distributed storage system according to the second node quantity, the cabinet quantity of the target node pool, and the third node quantity of the management nodes deployed in each cabinet in the target node pool includes: Determine the maximum and minimum number of the management nodes to be deployed in each cabinet in the target node pool according to the number of the second nodes and the number of cabinets in the target node pool; The distribution of the management nodes in the distributed storage system is adjusted according to the maximum number, the minimum number, and the third node number of the management nodes deployed in each cabinet in the target node pool.

3. The method according to claim 2, characterized in that The adjusting the distribution of the management nodes in the distributed storage system according to the maximum number, the minimum number, and the third number of management nodes deployed in each cabinet in the target node pool includes: In the case where the distribution situation is that all the management nodes in the distributed storage system are located in the target node pool, adjusting the distribution situation of the management nodes in the distributed storage system according to the maximum number, the minimum number, and the third node number of the management nodes deployed in each cabinet in the target node pool; In the case where the distribution situation is that not all management nodes in the distributed storage system are located in the target node pool, after the management nodes located in other node pools in the distributed storage system are converted into non-management nodes, the distribution situation of the management nodes in the distributed storage system is adjusted according to the maximum number, the minimum number and the third node number of the management nodes deployed in each cabinet in the target node pool.

4. The method according to claim 3, characterized in that The adjusting the distribution of the management nodes in the distributed storage system according to the maximum number, the minimum number, and the third number of management nodes deployed in each cabinet in the target node pool includes: From the target node pool, select a first cabinet whose number of third nodes is greater than the maximum number and a second cabinet whose number of third nodes is less than the minimum number; Convert a first number of management nodes in each first cabinet into non-management nodes; wherein the first number corresponding to each first cabinet is the difference between the third number of management nodes deployed in the first cabinet and the maximum number; Convert a second number of non-managed nodes in each second cabinet into managed nodes; wherein the second number corresponding to each second cabinet is a difference between the minimum number and a third number of managed nodes deployed in the second cabinet.

5. The method according to claim 1, characterized in that In the case where the security level of the target node pool in the distributed storage system needs to be converted from the node security level to the cabinet security level, the method further includes: Obtain each disk selection array of the disk pool associated with the target node pool, wherein each disk selection array includes a disk identifier of a disk for storing original business data and a disk identifier of a disk for storing duplicate business data of the original business data; and disks corresponding to different disk identifiers contained in each disk selection array are deployed on different cabinets; According to the first data storage times of the disks corresponding to the different disk identifiers in the data storage scenarios based on the respective disk selection arrays, a source-side disk selection array and a target-side disk selection array are selected from the respective disk selection arrays; The data of each disk corresponding to the source-side disk selection array is migrated to each disk corresponding to the target-side disk selection array.

6. The method according to claim 5, characterized in that The method of selecting a source-end disk selection array and a target-end disk selection array from each disk selection array according to the first data storage times of disks corresponding to different disk identifiers in the data storage scenario based on each disk selection array comprises: Determine, according to the first data storage times of the disks corresponding to the different disk identifiers in the data storage scenarios based on the disk selection arrays, the second data storage times of the cabinets where the disks corresponding to the different disk identifiers are located in the data storage scenarios based on the disk selection arrays; The cabinet corresponding to the largest second data storage number is used as the source cabinet; Use the disk corresponding to the largest first data storage number in the source cabinet as the source disk; Selecting a source disk selection array from the disk selection array containing the disk identifier of the source disk; A target-side disk selection array is selected from the disk selection array that does not include the disk identifiers of the disks in the source-side cabinet.

7. The method according to claim 1, characterized in that In the case where the security level of the target node pool in the distributed storage system needs to be converted from the node security level to the cabinet security level, the method further includes: Creating a sorted array; wherein the number of elements in the sorted array is the number of the first nodes; The node identifiers of the nodes in the target node pool are written into the sorted array; wherein the nodes corresponding to two adjacent node identifiers in the sorted array are located in adjacent cabinets, and the cabinet where the node corresponding to each node identifier is located is the cabinet in the target node pool where the cabinets corresponding to the written node identifiers are located have the least number of appearances; the written node identifier is the node identifier before the node identifier in the sorted array; Generate a node domain linked list according to the arrangement order of each node identifier in the sorted array; A node view is generated according to the node domain linked list and data redundancy; wherein the node view is used to indicate the node identifier of the node storing the original log data and the node identifier of the node storing the copy log data of the original log data.

8. The method according to claim 7, characterized in that The method further comprises: In the case where there are abnormal nodes in the target node pool, node identifiers of the abnormal nodes in the sorted array are removed; wherein the abnormal nodes include faulty nodes and / or deleted nodes; Generate a new node domain linked list according to the sorting order of each node identifier in the sorted array after elimination processing; According to the new node domain linked list, the node view is updated to obtain a new node view.

9. The method according to claim 8, characterized in that The method further comprises: In the case where there is a newly added node in the target node pool, if there is a target view in which the cabinets to which the adjacent nodes belong are not adjacent in the new node view, determining whether the cabinet to which the newly added node belongs is adjacent to the cabinet to which any node in the target view belongs; If the cabinet to which the newly added node belongs is adjacent to the cabinet to which the target node in the target view belongs, the node identifier of the newly added node is written into the node identifier of the target node in the sorted array to obtain a new sorted array; If the cabinet to which the newly added node belongs is not adjacent to the cabinet to which any node in the target view belongs, the node identifier of the newly added node is written into the tail of the sorted array to obtain a new sorted array; Generate a new node domain linked list according to the sorting order of each node identifier in the new sorting array; The new node view is updated according to the new node domain linked list.

10. A distributed storage system processing device, characterized in that: The device comprises: A first acquisition module, configured to acquire a first number of nodes in the target node pool and a second number of nodes of the management node in the distributed storage system when the security level of the target node pool in the distributed storage system needs to be converted from a node security level to a cabinet security level; An adjustment module is used to adjust the distribution of management nodes in the distributed storage system according to the second number of nodes, the number of cabinets in the target node pool, and the third number of management nodes deployed in each cabinet in the target node pool when the number of the first nodes is greater than or equal to the number of the second nodes, so that the management nodes are evenly distributed on each cabinet in the target node pool.

Images