Data node management method and device, computer equipment and readable storage medium

Abstract

The invention relates to a data node management method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: determining a plurality of namespaces of a disk; the plurality of namespaces are obtained by dividing the disk, and the plurality of namespaces are storage units with data isolation; allocating a processor core according to each namespace, and binding each namespace with the allocated corresponding processor core to obtain the processor core bound with each namespace; and according to the processor core bound with each namespace, managing the data node corresponding to each namespace. By adopting the method, the processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a data node management method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] As one of the most important components in a distributed cluster, the data node is responsible for storing and managing data. In a production or test environment, the data node service can be deployed on a HDD hard disk to store actual business data.

[0003] As the cluster usage time increases, the amount of data stored on each HDD data node will also increase. During the data storage process, the merging characteristics of RocksDB storage itself will cause very large jitters on read and write performance, which will not only occupy the read and write bandwidth of the SSD disk, but also increase the data read and write latency on the HDD disk, causing upper-layer business jams and affecting user experience. Summary of the invention

[0004] Based on this, it is necessary to provide a data node management method, device, computer equipment, computer-readable storage medium and computer program product that can improve processing efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a data node management method, including: a data node management method, the method comprising:

[0006] Determine multiple namespaces of a disk; the multiple namespaces are obtained by dividing the disk, and the multiple namespaces are storage units for data isolation;

[0007] Allocating a processor core according to each of the namespaces, binding each of the namespaces with the corresponding allocated processor core, and obtaining the processor core bound to each of the namespaces;

[0008] According to the processor core bound to each namespace, the data nodes corresponding to each namespace are managed.

[0009] In one embodiment, binding each of the namespaces to a corresponding allocated processor core to obtain the processor core to which each of the namespaces is bound includes:

[0010] Binding each of the namespaces to a corresponding allocated processor core through affinity parameters to obtain a processor core to which each of the namespaces is bound;

[0011] The affinity parameter indicates the binding relationship between the interrupt request of each namespace and the processor core.

[0012] In one of the embodiments, the affinity parameters include preset interrupt affinity parameters and parameters corresponding to the script;

[0013] The step of binding each of the namespaces to a corresponding allocated processor core by using affinity parameters to obtain a processor core to which each of the namespaces is bound includes:

[0014] Disabling the disk affinity between the disk and the processor core by using the preset interrupt affinity parameter; the disk affinity indicates that the interrupt request of the disk is processed by the processor core to which the disk is bound;

[0015] Through the parameters corresponding to the script, the space affinity of each namespace is bound to the corresponding allocated processor core to obtain the processor core bound to each namespace; the space affinity represents the interrupt request of each namespace, which is processed by the processor core bound to each namespace.

[0016] In one of the embodiments, the namespace includes an index namespace for storing metadata and a data namespace for storing logs and data;

[0017] The determining of multiple namespaces of the disk includes:

[0018] Obtain the number of namespaces, the number of index namespaces, and the number of virtual nodes of the mechanical hard disk of the server; the number of namespaces of the server indicates the number of namespaces obtained by dividing each disk of the server;

[0019] Determine the index namespace according to the number of the index namespaces;

[0020] The difference between the number of namespaces of the server and the number of index namespaces is determined, and the data namespace bound to the virtual node is determined according to the namespaces with the difference and the number of virtual nodes.

[0021] In one embodiment, the disk is a solid state drive, the namespace includes an index namespace obtained by dividing the solid state drive, and a data namespace for storing logs and data, and the solid state drive has a non-volatile memory host controller interface;

[0022] The managing the data nodes corresponding to each namespace according to the processor core bound to each namespace includes:

[0023] Get the drive letter for reconstruction;

[0024] According to the processor core bound to the index namespace indicated by the drive letter for reconstruction, formatting the index namespace indicated by the drive letter for reconstruction and reconstructing the data nodes; and / or,

[0025] When the number of the data namespace represented by the drive letter for reconstruction and the number of the mechanical hard disk disks represented by the drive letter for reconstruction are inconsistent, skip the physical disks that do not exist in the mechanical hard disk; when the numbers are consistent, detect the virtual nodes of the mechanical hard disk, skip the existing virtual nodes, and create the virtual nodes according to the number of virtual nodes corresponding to the mechanical hard disk; bind the virtual nodes to the data namespace.

[0026] In one embodiment, the method further comprises:

[0027] When the number of data nodes changes, the parameters of the object storage cluster are adjusted according to the preset parameters corresponding to the changed number of data nodes;

[0028] The object storage cluster includes the solid state drive and the mechanical hard drive.

[0029] In one embodiment, managing the data nodes corresponding to each namespace according to the processor core bound to each namespace includes:

[0030] Get the drive letter for repair;

[0031] If the physical volume indicated by the repair drive letter does not exist, format the disk and return to the step of determining the multiple namespaces of the disk;

[0032] If the physical volume indicated by the repair drive letter exists, detecting whether the process of the data node indicated by the repair drive letter exists;

[0033] If the process does not exist, the physical volume data and the stored data corresponding to the data node indicated by the repair drive letter are cleared, and the process returns to the step of determining the multiple namespaces of the disk.

[0034] In a second aspect, the present application also provides a data node management device, including:

[0035] A determination module, used to determine multiple namespaces of a disk; the multiple namespaces are obtained by dividing the disk, and the multiple namespaces are storage units for data isolation;

[0036] A binding module, configured to allocate a processor core according to each of the namespaces, bind each of the namespaces with the allocated corresponding processor core, and obtain the processor core bound to each of the namespaces;

[0037] The management module is used to manage the data nodes corresponding to each namespace according to the processor core bound to each namespace.

[0038] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of managing the data nodes in any of the above embodiments are implemented.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of managing the data nodes in any of the above embodiments are implemented.

[0040] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of managing the data nodes in any of the above embodiments are implemented.

[0041] The above-mentioned data node management method, device, computer equipment, computer-readable storage medium and computer program product determine multiple namespaces of the disk, and the multiple namespaces are obtained by dividing the disk, and the multiple namespaces are storage units for data isolation; the data resources between the namespaces are isolated, and the read or write operations will not affect each other, providing better performance isolation, ensuring the stability and low latency of the application, and even in storage systems of different sizes, the number of data nodes can be flexibly adjusted for different types of application reading and writing, improving the overall expansion capability of the system, and can also effectively utilize storage hardware and reduce resource waste. Furthermore, according to each of the namespaces, a processor core is allocated, and each of the namespaces is bound to the corresponding processor core allocated to obtain the processor core bound to each of the namespaces; the granularity of disk resource isolation can be refined, and storage resources can be flexibly allocated to different applications or services, and then according to the processor core bound to each of the namespaces, the traditional data node deployment process is changed, the performance of the disk can be efficiently exerted, and the data nodes corresponding to each of the namespaces can be finely managed. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 An application environment diagram of a data node management method in one embodiment;

[0044] Figure 2 A schematic diagram of a flow chart of a method for managing a data node in an embodiment;

[0045] Figure 3 A flowchart of the steps of binding a processor core in one embodiment;

[0046] Figure 4 A schematic diagram of a specific flow chart of the core steps of a processor in an embodiment;

[0047] Figure 5 A schematic diagram of a process of rebuilding a data node in a namespace in one embodiment;

[0048] Figure 6 A schematic diagram of a process of repairing a data node in a namespace in one embodiment;

[0049] Figure 7 It is a schematic diagram of the comparison effect of 4K object reading and writing speed in one embodiment;

[0050] Figure 8 This is a schematic diagram showing the comparison effect of the read and write speed of a 4M object in one embodiment;

[0051] Fig. 9 is a structural block diagram of a data node management device in one embodiment;

[0052] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The data node management method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0055] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0056] In an exemplary embodiment, Figure 2 As shown, a data node management method is provided, and the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 202 to 206. Among them:

[0057] Step 202, determining multiple namespaces of the disk; the multiple namespaces are obtained by dividing the disk, and the multiple namespaces are storage units for data isolation.

[0058] A disk is a non-volatile memory that has the function of storing data. The disk can be a solid-state drive (SSD), which stores data based on flash memory, and flash memory stores data through charge, and has a higher processing efficiency than a mechanical hard disk (HDD). When divided into multiple namespaces, the processing efficiency can still be guaranteed. For example, a solid-state drive can support a non-volatile memory host controller interface (NVME), which is a new technology designed specifically for solid-state drives. It uses the PCIe bus to communicate directly with the CPU, thereby achieving higher speeds, lower latency, and more parallelism.

[0059] A namespace is a storage unit that sets data isolation through software code; a namespace is a logical storage space, and the disk space corresponding to each namespace is relatively independent and can be managed and accessed independently. Optionally, each namespace has its own logical block address (LBA) space so that each namespace can be accessed independently. Optionally, each namespace also has its own namespace identifier, capacity, and access control attributes to make the namespace more similar to a real device.

[0060] Data isolation is isolation at the software program level, that is, logical isolation. Optionally, the flash memory space is divided into several independent logical spaces, and the address range of each logical block is 0 to N-1, which means that the amount of data that can be stored in each logical space is N, so each logical space is called a namespace. Therefore, the namespace is not a physical isolation of blocks, but an isolation of logical blocks that can be addressed by the host software. A storage unit is a logical space for storing data.

[0061] In one embodiment, determining a plurality of namespaces of a disk includes: determining an address range of each namespace in a solid state disk according to an address of each namespace stored in an address mapping table.

[0062] In one embodiment, determining multiple namespaces of a disk includes: determining the address of a solid state drive contained in each server; determining addresses of multiple namespaces contained in the solid state drive based on the address of the solid state drive; and determining multiple namespaces of the disk based on the addresses of the multiple namespaces.

[0063] Step 204 , allocate a processor core according to each namespace, bind each namespace with the corresponding allocated processor core, and obtain the processor core bound to each namespace.

[0064] A processor core is an independent computing unit in a processor that can execute instructions and process data. Each processor core includes at least an arithmetic logic unit (ALU), which supports arithmetic operations, logical operations, shift operations, and other functions to execute instructions or process data. Optionally, a processor core can be a physical core or a virtual core. A physical core is an actual processor core, while a virtual core allows each physical core to process two or more threads simultaneously, thereby improving concurrent processing capabilities.

[0065] A processor core is a unit used to constitute a processor. Each processor includes multiple processor cores, so that each processor core executes corresponding instructions or processes corresponding data.

[0066] Each disk namespace is assigned one or more processor cores. Due to data isolation between different namespaces, the computing resources required for each namespace are mainly related to the data stored in each namespace, and the speed of calling processor cores across namespaces is relatively slow. Therefore, processor cores are assigned to each namespace to establish the relationship between namespaces and processor cores, so that each namespace uses its own assigned processor for processing rather than cross-core scheduling. As a result, the relationship between memory and processors can be refined for more fine-grained control.

[0067] The processor core bound to each namespace is the processor core used to process the data contained in each namespace. Through binding, the processor cores for data processing in each namespace can be limited, so that the original data isolation of the namespace is extended to the isolation of the data processing process. At this time, the data processing process of each namespace is based on the corresponding processor core and will not rashly use the processor cores bound to other namespaces, avoiding cross-core scheduling, thereby ensuring that the scheduling rate of the processor core is relatively fast. In addition, since the namespace is a logical space, it can be reconstructed. Therefore, through binding, the coupling between the namespace and the processor core is relatively moderate and can be adjusted more flexibly.

[0068] In an optional embodiment, processor cores are allocated according to each namespace, and each namespace is bound to the corresponding allocated processor core to obtain the processor core bound to each namespace, including: according to the number of processor cores corresponding to the disk and the number of namespaces on the disk, the processor cores are evenly distributed to each namespace; each namespace is bound to the corresponding processor core allocated to each namespace to obtain the processor core bound to each namespace.

[0069] In an optional embodiment, processor cores are allocated to each namespace, and each namespace is bound to the corresponding processor core to obtain the processor core bound to each namespace, including: allocating processor cores to namespaces of each priority in turn according to the priority order of each namespace; binding each namespace to the processor core of each namespace at the corresponding priority to obtain the processor core bound to each namespace. Among them, the higher the priority order of the namespace, the earlier the processor core is allocated, and the lower the priority order of the namespace, the later the processor core is allocated; and / or, the higher the priority order of the namespace, the more processor cores are allocated, and the lower the priority order of the namespace, the fewer processor cores are allocated.

[0070] Step 206 , managing the data nodes corresponding to each namespace according to the processor core bound to each namespace.

[0071] Data nodes are one of the most important components in a distributed cluster, and their main responsibility is to store and manage data. In a production or test environment, data node services can be deployed on mechanical hard disks to store actual business data; if customers have high performance requirements, they can choose to deploy data nodes on solid-state drives or NVME solid-state drives to create cache pools or metadata pools. In addition, the HDD data nodes that store data also need to be bound to the write-ahead log (WAL) and the data in the database for storage, in order to improve the data read and write speed and ensure the consistency of the restored data.

[0072] The correspondence between namespace and data node is that namespace is used to determine the address of disk storage data, and data node is a logical node that exists based on this address. Optionally, data node is an object storage device (OSD) that stores data at the address indicated by the namespace. It is a data node in the Ceph cluster and is responsible for storing and managing data. Each data node is responsible for managing a part of the data and providing data read, write and replication functions.

[0073] In an optional embodiment, managing the data nodes corresponding to each namespace according to the processor core bound to each namespace includes: creating or rebuilding the data nodes corresponding to each namespace according to the processor core bound to each namespace.

[0074] In the above data node management method, multiple namespaces of the disk are determined, and the multiple namespaces are obtained by dividing the disk, and the multiple namespaces are storage units for data isolation; the data resources between the namespaces are isolated, and the read or write operations will not affect each other, providing better performance isolation, ensuring the stability and low latency of the application, and even in storage systems of different sizes, the number of data nodes can be flexibly adjusted for different types of application reading and writing, improving the overall expansion capability of the system, and can also effectively utilize storage hardware and reduce resource waste. Then, according to each namespace, a processor core is allocated, and each namespace is bound to the corresponding processor core allocated to obtain the processor core bound to each namespace; the granularity of disk resource isolation can be refined, and storage resources can be flexibly allocated to different applications or services, and then according to the processor core bound to each namespace, the traditional data node deployment process is changed, the performance of the disk can be efficiently exerted, and the data nodes corresponding to each namespace can be finely managed.

[0075] In an exemplary embodiment, the namespace includes an index namespace for storing metadata and a data namespace for storing logs and data.

[0076] Determine multiple namespaces of the disk, including: obtaining the number of namespaces of the server, the number of index namespaces and the number of data nodes of the mechanical hard disk; the number of namespaces of the server indicates the number of namespaces obtained by dividing each disk of the server; determine the index namespace according to the number of index namespaces; determine the difference between the number of namespaces of the server and the number of index namespaces, and determine the data namespace bound to the above virtual node according to the namespace with the difference and the number of virtual nodes.

[0077] The index namespace is used to store metadata, which is not only the description data of the data stored on the disk, but also the index used to query the data. Metadata includes but is not limited to attributes such as file name and file size.

[0078] As the amount of data stored on disks continues to increase, metadata also continues to increase. In this case, if the number of index namespaces is too small, each index namespace will store too much metadata, affecting data reading and data query. Correspondingly, if the number of index namespaces is too large, since each index namespace is read separately and needs to be merged, it will affect the efficiency of metadata reading operations during the indexing process and reduce the performance of object index query. In this case, this problem can be avoided by obtaining the number of index namespaces.

[0079] The data namespace is a namespace used to store logs and data. Optionally, the data namespace includes a log area and a data area, the amount of data that can be stored in the log area is fixed, and the data area is used to store data other than the log area.

[0080] The number of namespaces is the total number of namespaces contained in the server; the number of index namespaces is the total number of index namespaces contained in the server; the difference between the number of namespaces and the number of index namespaces is the difference between the number of namespaces and the number of index namespaces, which is the number of data namespaces.

[0081] The number of virtual nodes of a mechanical hard disk is the number of logical partitions obtained by dividing the mechanical hard disk. The number of virtual nodes can be used to control the distribution of data stored in the mechanical hard disk. Each virtual node is responsible for storing a part of the data. The data can be evenly distributed to different virtual nodes through hash algorithms or other distribution strategies to form a distributed system. Distributed systems include but are not limited to HDFS, Ceph, etc.

[0082] The data namespace is bound to the virtual node, which has many advantages. It can form hierarchical storage and cache acceleration, and can also mix redundancy strategies to ensure that different types of data are properly protected. For example, frequently accessed hot data will be placed on high-performance SSDs, while seldom accessed cold data will be placed on HDDs with larger capacity but slower speeds to achieve hierarchical storage and cache acceleration; hot data can be stored on both SSDs and HDDs, while cold data is only stored on HDDs. Even if a HDD fails, the copy on the SSD can take over immediately to ensure the continuous availability of data, thus forming a hybrid redundancy strategy.

[0083] In one embodiment, obtaining the number of namespaces, the number of index namespaces, and the number of virtual nodes of the mechanical hard disk of the server includes: obtaining the number of namespaces and the number of index namespaces carried by the server instruction, and obtaining the number of virtual nodes of the mechanical hard disk. Thus, two quantities are directly obtained through the instruction, and then the number of virtual nodes is obtained separately to ensure flexibility.

[0084] In one embodiment, obtaining the number of namespaces, the number of index namespaces, and the number of virtual nodes of the mechanical hard disk of the server includes: determining the application scenario of the server; obtaining the number of preset namespaces, the number of preset index namespaces, and the number of virtual nodes of the mechanical hard disk corresponding to the application scenario. Thus, the three numbers are pre-set according to the application scenario to ensure processing efficiency.

[0085] In one embodiment, determining the index namespace according to the number of index namespaces includes: selecting different physical addresses on the same disk as the index namespace according to the number of index namespaces and the space size attribute corresponding to the number of index namespaces. Thus, the creation process of the index namespace precedes the creation process of the data namespace, so that the physical address that can be used by the index namespace is more sufficient, which helps to avoid abnormality of the index namespace caused by insufficient physical addresses.

[0086] In one embodiment, determining the index namespace according to the number of index namespaces includes: creating namespaces according to the number of namespaces of the server; and selecting a preset number of namespaces from the created namespaces as the index namespaces.

[0087] In one embodiment, the data namespace bound to the virtual node of the mechanical hard disk is determined based on the namespace with the quantity difference and the number of virtual nodes, including: selecting a physical address according to the quantity difference and the space size attribute in the physical address that does not contain the index namespace to obtain the namespace to be bound; binding the namespace to be bound to each virtual node of the mechanical hard disk to obtain the data namespace bound to each virtual node of the mechanical hard disk.

[0088] In one embodiment, the data namespace bound to the virtual node is determined based on the difference in the number of namespaces and the number of virtual nodes, including: when creating a namespace according to the number of namespaces of the server, selecting a namespace other than the index namespace from the created namespaces as the namespace to be bound; binding the namespace to be bound to the virtual node of each mechanical hard disk to obtain the data namespace bound to each virtual node of the mechanical hard disk.

[0089] In this embodiment, since the number of server namespaces is defined according to demand, and too many or too few index namespaces will cause the efficiency of the namespace cluster to be low, the number of server namespaces and the number of index namespaces are directly obtained to meet the demand and avoid abnormal problems caused by indexes; then the index namespace is determined according to the number of index namespaces, forming multiple indexes of metadata objects; at the same time, the difference between the number of server namespaces and the number of index namespaces is determined, and the data namespace is determined according to the number of namespaces and virtual nodes of the difference, so as to use more space for data storage as much as possible, and the virtual node of the mechanical hard disk is bound to the data node of the solid-state hard disk, which can form hierarchical storage and cache acceleration, and can also mix redundancy strategies to ensure that different types of data are properly protected. Therefore, the number of server namespaces and the number of index namespaces are used as control variables, and the index namespace is constructed first, and then the data namespace is created. Through this order, the construction process of the namespace cluster is controlled to ensure processing efficiency.

[0090] In an exemplary embodiment, each namespace is bound to a corresponding assigned processor core to obtain the processor core to which each namespace is bound, including: binding each namespace to a corresponding assigned processor core through an affinity parameter to obtain the processor core to which each namespace is bound; wherein the affinity parameter represents the binding relationship between the interrupt request of each namespace and the processor core.

[0091] The interrupt request of a namespace is an interrupt request for the namespace itself or for its corresponding data node. This interrupt request is used to instruct the processor core bound to the namespace to stop the task being executed to handle the urgent task.

[0092] The affinity parameter is a processor core parameter used to set interrupt affinity. Interrupt affinity is used to indicate the interrupt request allocation process, which avoids interrupt processing delays and resource contention by allocating interrupt requests to appropriate CPU cores for processing.

[0093] The binding relationship is the relationship between which interrupt request is processed by which processor core. Optionally, the corresponding binding relationship can be determined through a mapping table, a linked list or a hash table where the affinity parameter is located.

[0094] Optionally, when the processor core bound to each namespace processes the data contained in its corresponding data node, if an interrupt request is detected, the processor core indicated by the affinity parameter stops the task being processed and responds to the interrupt request according to the processor core indicated by the affinity parameter.

[0095] An interrupt request is a request for instructing a corresponding processor core to stop executing the current task. An interrupt request is used to instruct a corresponding processor core to temporarily interrupt the running program and execute an urgent task instead. Interrupt processing can be initiated by a device for controlling a disk, which includes but is not limited to a timer, a network interface, etc.

[0096] In an optional embodiment, each namespace is bound to a corresponding processor core through an affinity parameter to obtain the processor core bound to each namespace, including: forming a binding relationship between each namespace and the corresponding processor core through a value setting in the affinity parameter to obtain the processor core bound to each namespace. Thus, binding relationships in different modes are formed through the value setting of the affinity parameter.

[0097] Exemplarily, when the affinity parameter is a first value, each namespace is bound to the corresponding assigned processor core according to the binding relationship indicated by the first value, so as to obtain the processor core bound to each namespace; when the affinity parameter is a second value, each namespace is bound to the corresponding assigned processor core according to the binding relationship indicated by the second value, so as to obtain the processor core bound to each namespace.

[0098] In this embodiment, the binding relationship between the interrupt request and the processor core is set by affinity parameters, so that when the data node or the namespace corresponding to the data node detects an emergency task, it is processed based on the processor core bound to the namespace. Therefore, the interrupt request processing processes of each namespace interfere with each other less, reducing the contention of processor cores in different namespaces caused by interrupt requests, thereby improving the overall processing efficiency of the disk.

[0099] In an exemplary embodiment, the affinity parameters include preset interrupt affinity parameters and parameters corresponding to the script. Figure 3 As shown, each namespace is bound to a corresponding allocated processor core through affinity parameters, and the processor core bound to each namespace is obtained, including:

[0100] Step 302, by presetting interrupt affinity parameters, the disk affinity between the disk and the processor core is disabled; the disk affinity means that the interrupt request of the disk is processed by the processor core bound to the disk.

[0101] The preset interrupt affinity parameter is an adjustment parameter of the preset interrupt affinity. The preset interrupt affinity parameter is a preset parameter of the processor core, and the disk affinity can be disabled by switching or adjusting the value.

[0102] Disk affinity is the interrupt affinity at the disk level, which means that the interrupt request for each disk can be processed by the processor core bound to the disk. In this case, the processor core used to process any interrupt request may be the processor core bound to each disk, and tasks may be processed across processor cores. Correspondingly, disabling disk affinity prevents the connection between the disk and the interrupt request, and the interrupt request will not act on each namespace of the disk; thus, it not only forces each namespace of the disk to process the interrupt request, but also prevents the formation or effect of the interrupt affinity relationship, further strengthening system isolation, and avoiding the situation of processing tasks across processor cores, so as to further ensure processing efficiency.

[0103] In an optional embodiment, disabling the disk affinity between the disk and the processor core by preset interrupt affinity parameters includes: disabling the disk affinity between the disk and the processor core when the preset interrupt affinity parameter corresponding to the disk is switched from a first value to a second value. Thus, the disk affinity is disabled by switching the value.

[0104] In an optional embodiment, the disk affinity between the disk and the processor core is disabled by preset interrupt affinity parameters, including: when a binding disable instruction triggered by a preset event is detected, the preset interrupt affinity parameter corresponding to the disk is adjusted to 0 to disable the disk affinity between the disk and the processor core. Thus, the preset value is set by the preset event to efficiently disable the disk affinity.

[0105] Step 304, through the parameters corresponding to the script, the space affinity of each namespace is bound to the corresponding processor core, and the processor core bound to each namespace is obtained; the space affinity indicates that the interrupt request of each namespace is processed by the processor core bound to each namespace.

[0106] The parameters corresponding to the script are parameters whose values ​​are generated by running the script and are used to set the space affinity of each namespace. The parameters corresponding to the script are used to set the binding relationship according to each namespace. Optionally, the parameters corresponding to the script can also be used to allocate processor cores of the namespace to facilitate dynamic adjustment of space affinity.

[0107] Space affinity is the interrupt affinity at the namespace level, which means that the interrupt request for each namespace can be processed by the processor core bound to its namespace. In this case, the processor core used to process any interrupt request is the processor core bound to each namespace, so there will be no situation where tasks are processed across processor cores, thereby further ensuring processing efficiency.

[0108] In an optional embodiment, the spatial affinity of each namespace is bound to the corresponding processor core assigned by the script through the parameters corresponding to the script to obtain the processor core bound to each namespace, including: when the data node corresponding to the namespace is rebuilt, according to the number of namespaces after the reconstruction, the target parameter is selected from the preset parameters corresponding to the script; according to the target parameter, the spatial affinity of each namespace is bound to the processor core assigned to each namespace to obtain the processor core bound to each namespace. Thus, the spatial affinity of each namespace and the corresponding processor core changes dynamically with the number of nodes formed to ensure efficiency in different scenarios.

[0109] In an optional embodiment, the space affinity of each namespace is bound to the corresponding processor core assigned through the parameters corresponding to the script to obtain the processor core bound to each namespace, including: generating affinity parameters of each namespace through the script; configuring the binding relationship between the space affinity of each namespace and the corresponding processor core assigned through the affinity parameters of each namespace to form the processor core bound to each namespace. Thus, the script is executed according to each namespace, and is used to configure the corresponding space affinity for each namespace, and a corresponding configuration file or configuration data can be formed for easy traceability.

[0110] Exemplarily, the namespaces in the disk include namespace A and namespace B, the processor cores bound to namespace A are core A1 and core A2; the processor cores bound to namespace B are core B1 and core B2; and the processor cores bound to the disk are core A1, core A2, core B1, and core B2; in this case, by disabling the direct disk affinity between the disk and the processor core to which it is bound, the connection between the disk and the interrupt request can be prevented, the interrupt request will not act on the disk globally, the processing of the interrupt request is forcibly restricted, and the formation or effect of the interrupt affinity relationship can be prevented, so that the interrupt request of namespace A is processed by core A1 and core A2, but not by core B1 and core B2; similarly, the interrupt request of namespace B is processed by core B1 and core B2, but not by core A1 and core A2. As a result, not only will the situation of calling the processor core across disks be reduced, but also the situation of calling the processor core across namespaces A and B will not occur, so as to further ensure the task processing efficiency of the processor core.

[0111] In this embodiment, the disk affinity is set by presetting the interrupt affinity parameter to cancel the affinity of the disk layer and avoid the mutual influence of the affinity of multiple layers. Since the namespace is a logical storage unit rather than a physical storage unit, it can be rebuilt and repartitioned; in this case, the space affinity of each namespace can be dynamically changed by setting the parameters corresponding to the script. Even if the data node is rebuilt, the processor core bound to each namespace can be adaptively adjusted to meet different needs.

[0112] In an exemplary embodiment, the disk is a solid-state drive; the namespace includes an index namespace obtained by dividing the solid-state drive and a data namespace for storing logs and data, and the solid-state drive has a non-volatile memory host controller interface; according to the processor core bound to each namespace, the data nodes corresponding to each namespace are managed, including: obtaining a reconstruction drive letter; according to the processor core bound to the index namespace indicated by the reconstruction drive letter, formatting the index namespace indicated by the reconstruction drive letter and rebuilding the data node; and / or, when the number of data namespaces represented by the reconstruction drive letter and the number of mechanical hard disks represented by the reconstruction drive letter are inconsistent, skipping the physical disks that do not exist in the mechanical hard disk; when the numbers are consistent, detecting the virtual nodes of the mechanical hard disk, skipping the existing virtual nodes, and creating the virtual nodes according to the number of virtual nodes corresponding to the mechanical hard disk; binding the virtual nodes to the data namespace.

[0113] The reconstruction drive letter is a drive letter used to indicate the reconstruction of the data node. The reconstruction drive letter can directly point to at least one of the index namespace and the data namespace, and can also point to the mechanical hard disk bound to the data namespace. The reconstruction drive letter is obtained through instructions or events. Optionally, when the number of data namespaces represented by the reconstruction drive letter and the number of mechanical hard disk disks represented by the reconstruction drive letter are consistent, the reconstruction drive letter can be obtained again for verification, and when the consistency is confirmed again, the data node reconstruction is stopped.

[0114] When rebuilding the number of data namespaces represented by drive letters and rebuilding the number of mechanical hard disks represented by drive letters, non-existent HDD disks can be skipped and free namespaces can continue to exist, thereby updating the drive letter list.

[0115] Physical disks are disks at the hardware level. Each virtual node is responsible for storing a portion of data. The data can be evenly distributed to different virtual nodes through hash algorithms or other allocation strategies to form a distributed system. Distributed systems include but are not limited to HDFS, Ceph, etc. Physical disks are divided into multiple virtual nodes, such as LVM logical storage volumes; multiple physical disks can also be combined to form a virtual node, such as Ceph Pool.

[0116] In an optional embodiment, the index namespace indicated by the drive letter for reconstruction is formatted and the data node is reconstructed according to the processor core bound to the index namespace indicated by the drive letter for reconstruction, including: when the namespace indicated by the drive letter for reconstruction belongs to the index namespace, the index namespace indicated by the drive letter for reconstruction is formatted according to the processor core bound to the index namespace indicated by the drive letter for reconstruction, and the data node corresponding to the index namespace indicated by the drive letter for reconstruction is reconstructed according to the processor core bound to the index namespace indicated by the drive letter for reconstruction. Exemplarily, the formatting of the namespace can be directly executed in a multi-concurrency manner, and the ceph command can be called to create and activate the data node corresponding to the index namespace.

[0117] The data nodes corresponding to the data namespace can be bound to the data nodes in the mechanical hard disk through a mapping table, a linked list or a hash table.

[0118] For example, for data storage nodes used for log and data storage, the inspection tool will be used to determine whether the number of mechanical hard disks specified by the drive letter for reconstruction and the number of data namespaces specified by the drive letter for reconstruction are consistent. If the number is inconsistent, the number of mechanical hard disks specified by the drive letter for reconstruction and the number of data namespaces specified by the drive letter for reconstruction are reconstructed; if the number is consistent, the drive letter corresponding to the HDD disk is updated, and then the disk status corresponding to the HDD disk in the list is checked again to see if it is normal; if it is normal, the reconstruction is stopped, and if it is abnormal, the same steps as the inconsistent number are performed.

[0119] In the storage space of each data namespace, 4GB is allocated for logs, and the remaining space is used as the data storage partition of the mechanical hard disk data node. After all preparations are completed, the multi-concurrency method is still used to directly execute the formatting of the namespace and mechanical hard disk, and then call the command to create and activate the HDD data node, and bind the log / data storage partition.

[0120] In this embodiment, when the data node corresponding to the index namespace is rebuilt, there is no binding between the index namespace and the data node in the mechanical hard disk, so the formatting process can be performed directly, the data of the original data node can be cleared, and then the data node can be rebuilt. When the number of data namespaces represented by the drive letter used for reconstruction and the number of mechanical hard disks represented by the drive letter used for reconstruction are inconsistent, the physical disks that do not exist in the mechanical hard disk are skipped, and there is no need to rebuild the data nodes of the namespace for these non-existent physical disks to avoid waste of resources. When the number is consistent, the virtual nodes corresponding to the mechanical hard disk are detected, the existing virtual nodes are skipped, and virtual nodes are created according to the number of virtual nodes corresponding to the mechanical hard disk, so that the number of virtual nodes matches the data namespace; then, each virtual node is bound to the data namespace to complete the reconstruction in the distributed system. When the data node corresponding to the data namespace is rebuilt, the data namespace needs to be bound to the data node in the mechanical hard disk to store a large amount of data, so the data node corresponding to the data namespace is bound to the data node in the mechanical hard disk to ensure that the mechanical hard disks specified in the tool are all online and can be assigned to the corresponding namespace.

[0121] In an exemplary embodiment, the method further includes: when the number of data nodes changes, adjusting the parameters of the object storage cluster according to the preset parameters corresponding to the changed number of data nodes; wherein the object storage cluster includes a solid state drive and a mechanical hard drive.

[0122] Ceph Storage Cluster (Ceph) is a distributed object storage system that involves multiple data in the form of objects, each of which is composed of data blocks. The object storage cluster uses the CRUSH algorithm to distribute data objects to different data nodes (OSD) according to certain rules to ensure that data copies are distributed in different data namespaces, thereby improving data redundancy and availability.

[0123] The CRUSH algorithm refers to the use of a pseudo-random function to determine the storage location of a data object. This function can generate a pseudo-random value based on the object identifier (such as the file name or object ID) and the topology of the cluster, and then select the data node corresponding to the corresponding data namespace for storage based on this value. In the CRUSH algorithm, through a decentralized data mapping method, each client can independently calculate the storage location of the data without relying on a central server.

[0124] The object storage cluster manages objects through the Log-Structured Merge (LSM) tree structure. The Log-Structured Merge (LSM) tree structure divides objects into multiple levels. Each level of the Log-Structured Merge (LSM) tree contains multiple ordered key-value pair files (Sorted String Table, SSTable). Through the merging process of multiple ordered key-value pair files, storage redundancy is reduced and reading performance is optimized.

[0125] In an optional embodiment, when the number of data nodes changes, the parameters of the object storage cluster are adjusted according to the preset parameters corresponding to the changed number of data nodes, including: when the changed number of data nodes is determined according to an instruction or event, the parameters of the object storage cluster are adjusted according to the preset parameters corresponding to the changed number of data nodes.

[0126] In order to give full play to the high performance of NVME disks, combined with the workflow of RocksDB storage, that is, the workflow of log merging structure tree, the cluster's RocksDB storage parameters have been optimized and adjusted many times. After comparative testing, a set of stable parameter values ​​have been summarized, including but not limited to osd_op_num_shards, default_buffered_read and other parameter values. When the size of the data storage space corresponding to the data node changes with the number of data nodes, the corresponding max_bytes_for_level_base parameter will also change, but the overall performance of the cluster can still remain stable.

[0127] The above parameter value osd_op_num_shards is used to set the number of shards that the data node (OSD) operates. Increasing the number of shards can improve concurrent processing capabilities, but it will increase system resource consumption. The default_buffered_read is used to control whether to enable buffered reading. Enabling it can improve reading performance, but it will increase memory consumption. The above parameter value max_bytes_for_level_base is the maximum amount of data written to the first layer in RocksDB layered compression. The first layer in RocksDB layered compression contains the most recently written data. In addition, you can also select the setting

[0128] Parameters such as osd_op_num_threads_per_shard and osd_disk_threads change dynamically.

[0129] In this embodiment, the cluster performance of NVME solid-state drives is optimized; combining the high-speed characteristics of NVME disks and the merging characteristics of the object storage cluster through the log merging structure tree, the cluster parameters of the object storage cluster are adjusted in a targeted manner, the performance of RocksDB data storage is optimized, and the performance jitter caused by various namespaces when performing data storage data merging is reduced.

[0130] In an exemplary embodiment, the data nodes corresponding to each namespace are managed according to the processor core bound to each namespace, including: obtaining a repair drive letter; if the physical volume indicated by the repair drive letter does not exist, formatting the disk and returning to the step of determining the multiple namespaces of the disk; if the physical volume indicated by the repair drive letter exists, detecting whether the process of the data node indicated by the repair drive letter exists; if the process does not exist, clearing the physical volume data and stored data corresponding to the data node indicated by the repair drive letter, and returning to the step of determining the multiple namespaces of the disk.

[0131] The repair drive letter is used to indicate the drive letter for data node repair. The repair drive letter can directly point to the index namespace, and can also point to the mechanical hard disk bound to the data namespace. The repair drive letter is obtained through instructions or events.

[0132] A physical volume (PV) is a hardware device in a storage device; these devices are often used as the basis for a logical volume management (LVM) system. A hard disk or partition accessed by a drive letter (such as C:\, D:\, etc.). Optionally, a physical volume can be a structure in the mechanical hard disk mentioned above.

[0133] The data node process is a process that runs on the processor core to which the namespace is bound. The data node process is used for data reading and writing, replication and redundancy, fault recovery and self-repair, data distribution and load balancing, etc.

[0134] The data stored in the data node indicated by the repair drive letter is the data stored by the data node.

[0135] In this embodiment, the physical volume is detected first, and then the process is detected; when the physical volume does not exist, the disk formatting process is directly performed, and then the multiple namespaces of the disk are re-determined to ensure processing efficiency. When the process of the data node does not exist, the physical volume data and the data stored in the data node are cleared at the same time, and then the reconstruction process is performed.

[0136] In order to more clearly illustrate the advantages of the present application, the traditional distributed technology in the field of video storage is compared with the corresponding embodiments of the present application; in this embodiment, HDD is a mechanical hard disk, SSD is a solid state drive, SSD includes SATA (Serial Advanced Technology Attachment, serial advanced technology connection) and NVME (Non Volatile Memory Express, non-volatile memory host controller interface specification); in addition, an embedded key-value storage engine R can be used ocksDB For data storage.

[0137] In traditional technology, in order to control costs and obtain maximum storage capacity, a single storage server will be equipped with at least 30 HDD hard disks and up to 6 SATA SSD hard disks. Three SSD hard disks are used to create data nodes and add them to the metadata storage pool. The remaining three SSD disks are evenly divided into 10 log partitions and 10 Rocks partitions. DB The storage partition is bound to all HDD data nodes. As the cluster is used for a longer time, the data stored in each mechanical hard disk data node increases. DB The merging feature of the storage itself will cause a large jitter in the read and write performance during the execution process, which will not only occupy the read and write bandwidth of the SSD disk, but also increase the data read and write latency on the mechanical hard disk, causing upper-layer business to be stuck and affecting the user experience. Even if the SATA SSD is replaced with a better-performing NVME disk or by adjusting the Rocks DBThe storage parameters optimize the data storage data merging, the overall cluster performance will be slightly improved, but directly dividing the NVME disk into multiple partitions for log / data storage still has performance bottlenecks, and it also increases the actual cost of use for customers. Therefore, how to ensure the stability and optimization of cluster performance without increasing customer costs requires a practical solution.

[0138] In contrast, this embodiment is based on the existing new storage server architecture. In addition to helping customers effectively control server hardware costs, this method creates multiple namespaces for the NVME disks on the server, changes the traditional data node deployment process, and then affinity binds the interrupt request of each namespace to a specific CPU core, which can give full play to the performance of the storage cluster disk. In addition, this embodiment is used to rebuild the data node and optimize the corresponding Rocks DB Storage parameters, after testing, Rocks DB The cluster performance jitter is significantly reduced when the storage is merged, and the upper-layer business data reading and writing can continue to maintain stable operation.

[0139] In this case, the storage server of this embodiment is equipped with 4 NVME SSD disks and 36 HDD disks. Compared with the traditional server, 2 SSD disks are reduced and 6 HDD disks are added. Compared with ordinary SATA SSD disks, NVME SSD has better performance, longer service life and low failure rate, which reduces operation and maintenance costs. In addition, the increase in the number of HDD disks also increases the available capacity of the cluster and stores more data. After comprehensive evaluation, the cost performance is higher.

[0140] Therefore, this embodiment creates multiple logical partitions as namespaces on a single NVME physical device. Each namespace can be regarded as an independent logical storage unit with its own namespace identifier, capacity and access control attributes. These namespaces can be used independently to achieve better physical isolation. After NVME is initialized, the interrupt request of each namespace is bound to the affinity of the CPU, effectively reducing resource contention. Therefore, compared with simply dividing the NVME disk into multiple partitions for use, the performance of the NVME disk can be better exerted.

[0141] Based on this feature, the traditional data node creation and reconstruction management processes are optimized, and combined with the adjustment of data node parameters, Rocks DB The adjustment of storage parameters not only reduces the number of Rocks in the data nodes during the IO reading and writing process of the cluster, but also reduces the number of Rocks in the data nodes. DB The NVME disk performance jitter caused by storage merging also reduces the data read and write latency on the HDD data nodes, and the overall performance of the cluster is better.

[0142] It should be understood that from the perspective of node management, this embodiment is divided into a namespace binding module, a namespace reconstruction module and a namespace repair module. The above steps 202 and 204 belong to the namespace binding module, while step 206 belongs to the namespace reconstruction module and the namespace repair module.

[0143] like Figure 4 As shown, before rebuilding the data node, step 202 needs to be executed, and step 202 specifically includes: planning the number of namespaces of the NVME disks on each storage server according to demand. Specifically, for the object storage cluster, the server first obtains the number of NVME disks used for the object metadata index pool and the number of index namespaces corresponding to each NVME disk, which is used to create the data node of the index, and the data node of the index can be the index OSD. Then, remotely log in to each data storage node, obtain the number of NVME disks t and the drive letter, the number of HDD disks m and the drive letter on each data storage node, and then perform unbinding and deleting all namespace operations on each NVME disk to avoid reusing the existing namespace, that is, during initialization, the existing NVME1 is initialized according to the disk.

[0144] Step 202 specifically includes: Calculation of the number of log and data storage spaces: Since the number of NVMEs n of data nodes for metadata of each node has been obtained, the number of namespaces that need to be created for each remaining NVME disk =

[0145] m / (tn), each namespace corresponds to the log / data storage space usage of an HDD data node. If the number of HDD disks can be divided evenly, the space allocated to each namespace can be made the same.

[0146] Step 202 also includes: calculating again the space size and capacity corresponding to each namespace, calling the namespace creation command (create-namespace) to create a corresponding number of namespaces, and then binding the created namespaces to the NVME controller to enable controller recognition.

[0147] Correspondingly, step 204 includes: setting the preset interrupt neutrality parameter smp_affinity_enable to 0 so as not to use the interrupt affinity parameter provided by the system kernel; at the same time, after the NVME initialization is completed, executing the script to execute the NVME soft interrupt to bind the CPU, so that the CPU does not need to execute across namespaces when processing the interrupt, thereby reducing the latency of data request processing on the data node.

[0148] like Figure 5As shown, when the namespace reconstruction module executes step 206, it specifically includes: the tool reads the list of saved namespaces; the list contains the namespaces and HDD disk information that have been created, so the drive letter can be obtained. If it is used to index data nodes, the formatted namespace is directly executed in a multi-concurrency manner, and the ceph command is called to create and activate the data node; if it is used for log / data storage, it will first check whether the number of HDD disks and the number of namespaces specified in the tool are consistent. If the number is inconsistent, the list corresponding to the HDD disk is updated, that is, after re-obtaining the drive letter, the disk status in the list is checked again to see if it is normal, and the physical disk that does not exist in the mechanical hard disk is skipped. When the number is consistent, the virtual node of the mechanical hard disk is detected, the existing virtual node is skipped, and the virtual node is created according to the number of virtual nodes corresponding to the mechanical hard disk; the virtual node is bound to the data namespace. In addition, the namespace can be idle, that is, the data namespace that is not bound to the mechanical hard disk can exist.

[0149] This operation is mainly to ensure that the HDD disks specified in the tool are online and can be allocated to the corresponding namespace. There are two areas in the data node corresponding to each namespace. One area is divided into 4GB for logs, and the other area is the remaining space other than 4GB as the data storage partition of the HDD data node. After all preparations are completed, the multi-concurrency method is still used to directly execute the formatting of the namespace and HDD disk, call the command to create, activate the HDD data node, and bind the log / data storage partition.

[0150] When the namespace reconstruction module executes step 206, it specifically also includes: waiting for the tool to complete execution, using the ceph command to check whether the data node is added to the cluster and added to the corresponding data placement strategy (hostcrush rule) to be managed by the corresponding processor core.

[0151] In order to give full play to the high performance of NVME disks, combined with Rocks DB Storage workflow, Rocks for clusters DB The storage parameters have been optimized and adjusted many times, and a set of stable parameter values ​​have been summarized after comparative tests, including but not limited to osd_op_num_shards, default_buffered_read, etc. When the size of the data storage space corresponding to the data node changes with the number of data nodes, the corresponding max_bytes_for_level_base parameter will also change, but the overall performance of the cluster can still remain stable.

[0152] like Figure 6As shown, when the namespace repair module executes step 206, it specifically includes: when one or some data nodes are abnormal and need to be repaired, there is no need to re-execute the namespace uninstallation and deletion operation, and the drive letter of the disk corresponding to the data node can be directly specified. The tool will first detect whether the physical volume corresponding to the drive letter actually exists. If it does not exist, the disk will be formatted and the data node will be rebuilt. The HDD data node also needs to bind the corresponding log / data storage partition. The index data node does not need to perform the binding operation. The data node is rebuilt and added to the cluster. If the physical volume information corresponding to the drive letter exists, it means that the disk has been created with a data node before and may need to be repaired. The tool first detects whether the corresponding data node process exists. If it does not exist, the physical volume information corresponding to the data node and the information in the cluster crush are first cleaned up, and then formatted again and the data node is recreated.

[0153] In order to verify the improvement brought by the method proposed in this application, the performance of the cluster after creating data nodes using the traditional method and the method is compared and tested. The specific embodiments are described as follows:

[0154] Use three servers to deploy an EC2+1 object storage cluster, each server is equipped with 4 3T NVME disks, namely / dev / nvme0n1, / dev / nvme1n1, / dev / nvme2n1, / dev / nvme3n1, and 36 16T HDD disks, with drive letters from / dev / s data storage-sdz, / dev / sdaa / sdak. The actual available capacity of the cluster can reach 1PB. Since the data node corresponding to each HDD disk must be bound to the log / data storage partition and the metadata pool must be used, there are two types of use of NVME disks. Solution 1 adopts the traditional method; Solution 2 and Solution 3 both adopt the solution of this application:

[0155] Solution 1: 3 NVME SSDs are used for log / data partitions of mechanical hard disk data nodes. Each NVME SSD is partitioned and directly used for log and data storage of 12 mechanical hard disks. 4GB is allocated for logs of each mechanical hard disk, and the remaining space is divided equally into 12. The remaining 1 NVME SSD is also directly created as a data node. Therefore, the cluster has 3 data nodes corresponding to the index pool, and the data nodes corresponding to the 108 mechanical hard disks form the data pool.

[0156] Solution 2: Use the namespce technology of NVME solid-state drives. Two NVME solid-state drives are used for log / data partitions of the data nodes corresponding to the mechanical hard disk. Each NVME solid-state drive needs to be divided into 18 namespaces, and the remaining two NVME solid-state drives are divided into 6 namespaces each. Each namespace is used as an index data node. This solution is close to simulating extreme conditions. The cluster has no additional nodes to provide disks for creating index data nodes. It can also test the extreme load of NVME solid-state drives. The cluster has 36 index data nodes to form an index pool, and 108 mechanical hard disk data nodes to form a data pool.

[0157] Solution 3: Use the namespce technology of NVME SSD. Three NVME SSDs are used for log / data partitions of mechanical hard disk data nodes. Each NVME SSD needs to be divided into 12 namespaces, and the remaining 1 NVME SSD is divided into 6 namespaces, all of which are used as index data nodes. Compared with Solution 1, this solution will reduce the pressure on the NVME SSD used for log / data storage partitions, but will increase the pressure on the index NVME SSD. You can choose to divide the last NVME SSD into 9 namespaces to share part of the latency caused by metadata reading and writing. In the end, the cluster has 18 index data nodes forming the index pool, and 108 mechanical hard disk data nodes forming the data pool.

[0158] This comparative test selects scheme 1 and scheme 3 for execution. According to the plan of scheme 3, / dev / nvme0n1 creates 6 namespaces, each with 496GB, and the remaining 3 NVMEs create 12 namespaces, each with 248GB, for a total of 36 namespaces, which correspond exactly to the 36 HDD disks.

[0159] After creating and binding all namespaces, modify the kernel parameters, adjust the soft interrupt binding core, start rebuilding the data node, and then use the optimized data node parameters and Rocks DB Storage parameters update the cluster ceph.conf configuration.

[0160] Disk information after dividing the NVME disk into multiple namespaces. nvme0 has 6 namespaces, and nvme1, nvme2, and nvme3 have 12 namespaces.

[0161] Taking nvme0 and nvme1 as an example, nvme0 includes but is not limited to:

[0162]

[0163] And so on, six namespaces of nvme0 are formed.

[0164] nvme1 includes but is not limited to:

[0165]

[0166] And so on, 12 namespaces of nvme1 are formed.

[0167] After creating the namespace as an index data node and using it as a log / data storage partition, take nvme0n1 as an example, the code is as follows:

[0168]

[0169] Among them, the device name is nvme0n1, which is the first namespace of the first name disk; the major and minor device numbers are 259:1; the capacity is 496G; the type is disk (disk); the namespace is ceph--osd--212--vg-osd--212--data; the major and minor device numbers are 253:21; the capacity is 496G; the disk type is logical volume (lvm).

[0170] Among them, ceph--osd--212--vg-osd--212--data is the name of a namespace, and its specific meanings are as follows: ceph indicates that this namespace is used for the Ceph storage system. osd indicates that this is an object storage device (OSD). 212 indicates that this is the 212th OSD. vg indicates that this is a volume group (VolumeGroup). osd once again emphasizes that this volume group is created specifically for Ceph object storage devices. 212 once again confirms that this volume group is created for the 212th OSD. data indicates that this logical volume is used to store actual data.

[0171] That is, ceph--osd--212--vg-osd--212--data is a logical volume created specifically for the 212th object storage device (OSD) in the Ceph storage system to store actual data. The name structure and identifier help manage and identify the various components in the Ceph cluster.

[0172] In another embodiment, the nvme2n12 namespace is used as a log / data storage partition:

[0173] nvme2n12 259:37 0 248G 0disk

[0174] └─ceph--osd--95--95--vg-osd--95--dbs 253:73 0 243G 0lvm

[0175] └─ceph--osd--95--95--vg-osd--95--wal 253:82 0 4G 0lvm

[0176] Most of the explanations refer to nvme0n1, except that nvme2n12 has two LVM volumes.

[0177] ceph--osd--95--95--vg-osd--95--dbs and ceph--osd--95--95--vg-osd--95--wal, these volumes are used for the database and log of the Ceph storage system.

[0178] The node information of the Ceph cluster can be as follows:

[0179] mon:3daemons,quorumT01-P01-DN-001.sc.cn,T01-P01-DN-002.sc.cn,T01-P01-DN-003.sc.cn(age2d): / / There are 3 monitoring nodes running, they form a consensus group and have been running for 2 days.

[0180] mgr:T01-P01-DN-002.sc.cn(active,since4d),standbys:T01-P01-DN-001.sc.cn,T01-P01-DN-003.sc.cn: / / The current active management node is T01-P01-DN-002.sc.cn, which has been active for 4 days. The standby management nodes are T01-P01-DN-001.sc.cn and T01-P01-DN-003.sc.cn.

[0181] osd:126osds:126up(since2d),126in(since2d): / / There are 126 object storage devices. All devices are online and have been online for 2 days. All devices have joined the cluster for 2 days.

[0182] The test client uses the same number of concurrent connections to upload 1 million 4K small objects and 100,000 4M large objects. The cluster performs 4K and 4M object read and write performance comparisons between Solution 1 and Solution 3. The results are as follows: Figure 7 and Figure 8 The test conclusion is: after adjustment and optimization, under the same test conditions, the write IOPS of small objects can be increased by nearly 50%, the read IOPS can be increased by nearly double, and the read and write bandwidth of large objects can be slightly improved, all of which are close to the network card bottleneck.

[0183] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0184] Based on the same inventive concept, the embodiment of the present application also provides a data node management device for implementing the data node management method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiment of the management device for one or more data nodes provided below can refer to the limitations of the data node management method above, and will not be repeated here.

[0185] In an exemplary embodiment, Fig. 9 As shown, a data node management device is provided, including:

[0186] The determination module 902 is used to determine a plurality of namespaces of the disk; the plurality of namespaces are obtained by dividing the disk, and the plurality of namespaces are storage units for data isolation;

[0187] A binding module 904 is used to allocate a processor core according to each namespace, bind each namespace with the allocated corresponding processor core, and obtain the processor core bound to each namespace;

[0188] The management module 906 is used to manage the data nodes corresponding to each namespace according to the processor core bound to each namespace.

[0189] In one embodiment, the binding module 904 is used to:

[0190] Binding each of the namespaces to a corresponding allocated processor core through affinity parameters to obtain a processor core to which each of the namespaces is bound;

[0191] The affinity parameter indicates the binding relationship between the interrupt request of each namespace and the processor core.

[0192] In one of the embodiments, the affinity parameters include preset interrupt affinity parameters and parameters corresponding to the script;

[0193] The binding module 904 is used to:

[0194] Disabling the disk affinity between the disk and the processor core by using the preset interrupt affinity parameter; the disk affinity indicates that the interrupt request of the disk is processed by the processor core to which the disk is bound;

[0195] Through the parameters corresponding to the script, the space affinity of each namespace is bound to the corresponding allocated processor core to obtain the processor core bound to each namespace; the space affinity represents the interrupt request of each namespace, which is processed by the processor core bound to each namespace.

[0196] In one of the embodiments, the namespace includes an index namespace for storing metadata and a data namespace for storing logs and data;

[0197] The determining module 902 is used to:

[0198] Obtain the number of namespaces, the number of index namespaces, and the number of virtual nodes of the mechanical hard disk of the server; the number of namespaces of the server indicates the number of namespaces obtained by dividing each disk of the server;

[0199] Determine the index namespace according to the number of the index namespaces;

[0200] The difference between the number of namespaces of the server and the number of index namespaces is determined, and the data namespace bound to the virtual node is determined according to the namespaces with the difference and the number of virtual nodes.

[0201] In one embodiment, the disk is a solid state drive, the namespace includes an index namespace obtained by dividing the solid state drive, and a data namespace for storing logs and data, and the solid state drive has a non-volatile memory host controller interface;

[0202] The management module 906 is used to:

[0203] Get the drive letter for reconstruction;

[0204] According to the processor core bound to the index namespace indicated by the drive letter for reconstruction, formatting the index namespace indicated by the drive letter for reconstruction and reconstructing the data nodes; and / or,

[0205] When the number of the data namespace represented by the drive letter for reconstruction and the number of the mechanical hard disk disks represented by the drive letter for reconstruction are inconsistent, skip the physical disks that do not exist in the mechanical hard disk; when the numbers are consistent, detect the virtual nodes of the mechanical hard disk, skip the existing virtual nodes, and create the virtual nodes according to the number of virtual nodes corresponding to the mechanical hard disk; bind the virtual nodes to the data namespace.

[0206] In one embodiment, the management module 906 is used to:

[0207] When the number of data nodes changes, the parameters of the object storage cluster are adjusted according to the preset parameters corresponding to the changed number of data nodes;

[0208] The object storage cluster includes the solid state drive and the mechanical hard drive.

[0209] In one embodiment, the management module 906 is used to:

[0210] Get the drive letter for repair;

[0211] If the physical volume indicated by the repair drive letter does not exist, format the disk and return to the step of determining the multiple namespaces of the disk;

[0212] If the physical volume indicated by the repair drive letter exists, detecting whether the process of the data node indicated by the repair drive letter exists;

[0213] If the process does not exist, the physical volume data and the stored data corresponding to the data node indicated by the repair drive letter are cleared, and the process returns to the step of determining the multiple namespaces of the disk.

[0214] Each module in the management device of the above data node can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0215] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data node management method is implemented.

[0216] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0217] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0218] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0219] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0220] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0221] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0222] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0223] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data node management method, characterized in that: The method comprises: Determine multiple namespaces of a disk; the multiple namespaces are obtained by dividing the disk, and the multiple namespaces are storage units for data isolation; Allocating a processor core according to each of the namespaces, binding each of the namespaces with the corresponding allocated processor core, and obtaining the processor core bound to each of the namespaces; According to the processor core bound to each namespace, the data nodes corresponding to each namespace are managed.

2. The method according to claim 1, characterized in that The step of binding each of the namespaces to a corresponding allocated processor core to obtain the processor core to which each of the namespaces is bound includes: Binding each of the namespaces to a corresponding allocated processor core through affinity parameters to obtain a processor core to which each of the namespaces is bound; The affinity parameter indicates the binding relationship between the interrupt request of each namespace and the processor core.

3. The method according to claim 2, characterized in that The affinity parameters include preset interrupt affinity parameters and parameters corresponding to the script; The step of binding each of the namespaces to a corresponding allocated processor core by using affinity parameters to obtain a processor core to which each of the namespaces is bound includes: Disabling the disk affinity between the disk and the processor core by using the preset interrupt affinity parameter; the disk affinity indicates that the interrupt request of the disk is processed by the processor core to which the disk is bound; Through the parameters corresponding to the script, the space affinity of each namespace is bound to the corresponding allocated processor core to obtain the processor core bound to each namespace; the space affinity represents the interrupt request of each namespace, which is processed by the processor core bound to each namespace.

4. The method according to claim 1, characterized in that The namespace includes an index namespace for storing metadata and a data namespace for storing logs and data; The determining of multiple namespaces of the disk includes: Get the number of namespaces, index namespaces, and virtual nodes of the server; The number of namespaces of the server indicates the number of namespaces obtained by dividing each disk of the server; Determine the index namespace according to the number of the index namespaces; The difference between the number of namespaces of the server and the number of index namespaces is determined, and the data namespace bound to the virtual node is determined according to the namespaces with the difference and the number of virtual nodes.

5. The method according to claim 1, characterized in that The disk is a solid-state hard disk, the namespace includes an index namespace obtained by dividing the solid-state hard disk, and a data namespace for storing logs and data, and the solid-state hard disk has a non-volatile memory host controller interface; The managing the data nodes corresponding to each namespace according to the processor core bound to each namespace includes: Get the drive letter for reconstruction; According to the processor core bound to the index namespace indicated by the drive letter for reconstruction, formatting the index namespace indicated by the drive letter for reconstruction and reconstructing the data nodes; and / or, When the number of the data namespace represented by the drive letter for reconstruction and the number of the mechanical hard disk disks represented by the drive letter for reconstruction are inconsistent, skip the physical disks that do not exist in the mechanical hard disk; when the numbers are consistent, detect the virtual nodes of the mechanical hard disk, skip the existing virtual nodes, and create the virtual nodes according to the number of virtual nodes corresponding to the mechanical hard disk; bind the virtual nodes to the data namespace.

6. The method according to claim 5, characterized in that The method further comprises: When the number of data nodes changes, the parameters of the object storage cluster are adjusted according to the preset parameters corresponding to the changed number of data nodes; The object storage cluster includes the solid state drive and the mechanical hard drive.

7. The method according to claim 1, characterized in that The managing the data nodes corresponding to each namespace according to the processor core bound to each namespace includes: Get the drive letter for repair; If the physical volume indicated by the repair drive letter does not exist, format the disk and return to the step of determining the multiple namespaces of the disk; If the physical volume indicated by the repair drive letter exists, detecting whether the process of the data node indicated by the repair drive letter exists; If the process does not exist, the physical volume data and the stored data corresponding to the data node indicated by the repair drive letter are cleared, and the process returns to the step of determining the multiple namespaces of the disk.

8. A data node management device, characterized in that: The device comprises: A determination module, used to determine multiple namespaces of a disk; the multiple namespaces are obtained by dividing the disk, and the multiple namespaces are storage units for data isolation; A binding module, configured to allocate a processor core according to each of the namespaces, bind each of the namespaces with the allocated corresponding processor core, and obtain the processor core bound to each of the namespaces; The management module is used to manage the data nodes corresponding to each namespace according to the processor core bound to each namespace.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images