Thread core binding method and device, equipment and storage medium

Abstract

The invention discloses a thread core binding method, device and equipment and a storage medium, relates to the technical field of cloud computing distributed storage, and is applied to a preset distributed storage system. Identifying and classifying threads in a preset object storage component based on a preset classification rule, the system source code and the thread processing configuration information of the key business to obtain a classification result; based on the classification result and preset thread processing functions corresponding to the thread types in the classification result, whether the threads are bound with a local central processing unit or not is judged; determining a to-be-bound thread based on each thread judgment result, and determining core binding configuration item information corresponding to the to-be-bound thread by utilizing the classification result and a preset central processing unit resource allocation principle; and carrying out dynamic core binding on the thread to be subjected to core binding based on the preset core binding interface and the corresponding core binding configuration item information. And the performance optimization and the stability of the storage system are effectively realized.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing distributed storage technology, and in particular to a thread core binding method, device, equipment and storage medium. Background Art

[0002] The Ceph distributed storage system contains a variety of threads with different functions, such as business processing, heartbeat detection, network data transmission and reception, and command interaction threads. These threads have a significant impact on system performance and stability.

[0003] However, current technologies have many defects. In a multi-core architecture, the problem of unreasonable thread scheduling is prominent, and there is a phenomenon of CPU (Central Processing Unit) resource contention. Multiple threads compete for the same CPU resource, resulting in frequent context switching, which greatly increases system overhead. At the same time, there is a problem of memory access latency. When threads migrate between different CPU resources, memory access latency will increase, which will damage performance. Moreover, there is a load imbalance, and threads are unevenly distributed on various CPU resources. Some cores are overloaded, and other cores have low utilization. In addition, in special scenarios such as empty snapshot recycling, business processing threads are used in large quantities, and a single Ceph-osd (object storage daemon) process occupies a large number of CPUs. For storage nodes with a large number of data disks, the number of threads occupied is extremely large, and the requirements for the storage node CPU are too high, squeezing the CPU time slice of the heartbeat thread, causing the heartbeat thread to fail to operate normally, resulting in heartbeat timeouts, and easily causing problems such as system stability and performance degradation.

[0004] Therefore, how to improve and optimize the binding strategy between threads and CPU resources to achieve performance and stability tuning of the storage system is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a thread binding core method, device, equipment and storage medium, which can improve and optimize the binding strategy between threads and CPU resources, thereby realizing performance tuning of the storage system. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a thread-core binding method, which is applied to a preset distributed storage system, including:

[0007] During thread initialization or thread operation, threads in the preset object storage component are identified and classified based on preset classification rules, system source code, and thread processing configuration information of key businesses to obtain corresponding classification results;

[0008] Based on the classification results and the preset thread processing functions corresponding to the thread types in the classification results, respectively, determining whether each of the threads has been bound to the local central processing unit, so as to obtain thread determination results corresponding to each of the threads;

[0009] Determine the corresponding core thread to be bound based on the judgment result of each thread, and determine the core binding configuration item information corresponding to the core thread to be bound using the classification result and a preset central processing unit resource allocation principle;

[0010] The threads to be bound to cores are dynamically bound to cores based on the preset core binding interface and the corresponding core binding configuration item information to complete the corresponding thread core binding operation, and the obtained core-bound threads are used to trigger corresponding key business execution operations.

[0011] Optionally, determining the corresponding thread to be bound to the core based on the judgment result of each thread includes:

[0012] If the thread determination result indicates that the corresponding thread has been bound to the local central processing unit, triggering a corresponding key business execution operation based on the thread;

[0013] If the thread determination result indicates that the corresponding thread is not bound to the local central processing unit, the thread is determined to be a thread to be bound to a core.

[0014] Optionally, the determining the core binding configuration item information corresponding to the thread to be bound to the core by using the classification result and a preset central processing unit resource allocation principle includes:

[0015] When the thread is running, the core binding configuration item information corresponding to the thread to be bound is determined based on the total number of threads in the current central processor, the unbound resource collection items in the current local central processor, the classification results and the preset ratio thresholds corresponding to each thread type.

[0016] Optionally, the determining the core binding configuration item information corresponding to the thread to be bound to the core by using the classification result and a preset central processing unit resource allocation principle includes:

[0017] When the thread is initialized, the number of the to-be-bound core threads whose thread type is a business processing thread is determined based on the classification result, and the corresponding first core binding configuration item information is determined using the obtained first thread number, the total number of threads of the current central processing unit, and the corresponding first preset ratio threshold;

[0018] Determine the number of the to-be-bound core threads whose thread types are message sending and receiving threads and message distribution threads based on the classification result, and determine the corresponding second core binding configuration item information using the obtained number of second threads, the total number of threads of the current central processing unit, and the corresponding second preset ratio threshold;

[0019] Determine the number of the to-be-bound core threads whose thread type is a heartbeat thread based on the classification result, and determine corresponding third core binding configuration item information using the obtained number of third threads, the total number of threads of the current central processing unit, a preset thread isolation rule, and a corresponding third preset ratio threshold;

[0020] Based on the classification result, the number of threads to be bound to the core corresponding to the thread type of command interaction thread is determined, and the corresponding fourth core binding configuration item information is determined using the obtained fourth thread number, the total number of threads of the current central processing unit, the preset thread isolation rule and the corresponding fourth preset ratio threshold.

[0021] Optionally, the determining corresponding third core binding configuration item information by using the obtained third thread quantity, the total number of threads of the current central processing unit, a preset thread isolation rule, and a corresponding third preset ratio threshold includes:

[0022] Determine whether the obtained third number of threads is less than a first product value determined based on the total number of threads of the current central processing unit and a third preset ratio threshold value, so as to obtain a corresponding first numerical determination result;

[0023] If the first numerical judgment result indicates that it is not less than, a target resource set item isolated from the resource set item corresponding to the first core binding configuration item information is determined from the local central processor, and corresponding third core binding configuration item information is determined based on the target resource set item and the first product value.

[0024] Optionally, the determining corresponding fourth core binding configuration item information by using the obtained fourth thread quantity, the total number of threads of the current central processing unit, the preset thread isolation rule, and the corresponding fourth preset ratio threshold includes:

[0025] Determine whether the obtained fourth thread quantity is less than a second product value determined based on the total number of threads of the current central processing unit and a fourth preset ratio threshold value, so as to obtain a corresponding second numerical determination result;

[0026] If the second numerical judgment result indicates that it is not less than, a target resource set item isolated from the resource set item of the first core binding configuration item information is determined from the local central processor, and corresponding fourth core binding configuration item information is determined based on the target resource set item and the second product value.

[0027] Optionally, dynamically binding the thread to be bound to a core based on a preset core binding interface and the corresponding core binding configuration item information includes:

[0028] Based on the first preset thread processing function, a preset core binding interface is called, and the resource collection item corresponding to the first core binding configuration item information and the preset core binding interface are used to bind the core to be bound thread whose thread type is the business processing thread, so as to complete the first core binding operation;

[0029] Based on the second preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the second core binding configuration item information and the preset core binding interface are used to bind the to-be-bound core threads whose thread types are the message sending and receiving thread and the message distributing thread, so as to complete the second core binding operation;

[0030] Based on the third preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the third core binding configuration item information and the preset core binding interface are used to bind the core to be bound thread whose thread type is the heartbeat thread, so as to complete the third core binding operation;

[0031] The preset core binding interface is called based on the fourth preset thread processing function, and the resource collection item corresponding to the fourth core binding configuration item information and the preset core binding interface are used to bind the core to be bound whose thread type is the command interaction thread to complete the fourth core binding operation.

[0032] In a second aspect, the present application provides a thread core binding device, which is applied to a preset distributed storage system, including:

[0033] The thread classification module is used to identify and classify threads in the preset object storage component based on preset classification rules, system source code and thread processing configuration information of key businesses during thread initialization or thread running, so as to obtain corresponding classification results;

[0034] A thread determination module, used to determine whether each of the threads has been bound to a local central processing unit based on the classification results and preset thread processing functions corresponding to each thread type in the classification results, so as to obtain thread determination results corresponding to each of the threads;

[0035] An information determination module, configured to determine a corresponding core thread to be bound based on the judgment result of each of the threads, and to determine core binding configuration item information corresponding to the core thread to be bound using the classification result and a preset central processing unit resource allocation principle;

[0036] The thread core binding module is used to dynamically bind the to-be-bound core threads based on the preset core binding interface and the corresponding core binding configuration item information to complete the corresponding thread core binding operation, and use the obtained core-bound threads to trigger the corresponding key business execution operation.

[0037] In a third aspect, the present application provides an electronic device, including:

[0038] Memory, used to store computer programs;

[0039] The processor is used to execute the computer program to implement the aforementioned thread-core binding method.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned thread-core binding method is implemented.

[0041] In the present application, a distributed storage system is preset. When a thread is initialized or a thread is running, the threads in the preset object storage component are identified and classified based on preset classification rules, system source code and thread processing configuration information of key business to obtain corresponding classification results; based on the classification results and the preset thread processing functions corresponding to each thread type in the classification results, it is judged whether each of the threads has been bound to the local central processing unit to obtain thread judgment results corresponding to each of the threads; based on each of the thread judgment results, the corresponding core thread to be bound is determined, and the classification results and the preset central processing unit resource allocation principle are used to determine the core binding configuration item information corresponding to the core thread to be bound; based on the preset core binding interface and the corresponding core binding configuration item information, the core thread to be bound is dynamically bound to complete the corresponding thread core binding operation, and the corresponding key business execution operation is triggered by using each core-bound thread obtained. As can be seen from the above, when the thread is initialized or running, this application identifies and classifies the threads of the preset object storage component, determines whether the thread is bound to the local central processing unit based on the classification results and the thread processing function, and then determines the thread to be bound to the core based on the thread judgment result, and determines the core binding configuration item information in combination with the classification results and the central processing unit resource allocation principle, and dynamically binds the core to it using the preset core binding interface and core binding configuration item information, and finally triggers key business operations with the core-bound thread. In this way, this application can improve and optimize the binding strategy between threads and CPU resources, thereby improving the performance of the storage system and thus improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A thread core binding flow chart disclosed in this application;

[0044] Figure 2 A schematic diagram of a specific thread core binding process disclosed in this application;

[0045] Figure 3 A schematic diagram of a core binding process for a business processing thread disclosed in this application;

[0046] Figure 4 A schematic diagram of a core binding process for a message sending and receiving thread disclosed in this application;

[0047] Figure 5 This is a schematic diagram of the structure of a thread core binding device disclosed in this application;

[0048] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Under the multi-core architecture, the problem of unreasonable thread scheduling is prominent, there is a phenomenon of CPU resource contention, and multiple threads compete for the same CPU resource, resulting in frequent context switching, which greatly increases the system overhead. At the same time, there is a problem of memory access delay. When threads migrate between different CPU resources, the memory access delay will increase, which will damage performance. Moreover, there is a load imbalance, and threads are unevenly distributed on various CPU resources, some cores are overloaded, and other cores are underutilized. In addition, in special scenarios such as empty snapshot pruning, business processing threads are used in large quantities, and a single Ceph-osd process occupies more CPU. For storage nodes with a large number of data disks, the number of threads occupied is extremely large, and the requirements for the storage node CPU are too high, squeezing the CPU time slice of the heartbeat thread, causing the heartbeat thread to fail to operate normally, and heartbeat timeouts occur, which can easily cause problems such as system stability and performance degradation. To this end, the present application provides a thread binding core method to achieve performance and stability tuning of the storage system.

[0051] See also Figure 1 As shown, an embodiment of the present invention discloses a thread core binding method, which is applied to a preset distributed storage system, including:

[0052] Step S11: When a thread is initialized or running, the threads in the preset object storage component are identified and classified based on preset classification rules, system source code, and thread processing configuration information of key services to obtain corresponding classification results.

[0053] In this embodiment, it should be first explained that the preset distributed storage system can be a Ceph system, and the preset object storage component can be an osd component in the Ceph system that is responsible for the management and operation of storage object data. The preset object storage component relies on threads to perform various functions, such as the business processing thread (osd_op_tp thread) is responsible for all key business processing, including data reading and writing, data recovery and other operations; the network message sending and receiving thread (msgr-worker thread) is responsible for the system's network communication, receiving external network data, and sending network data to the other end; the message distribution thread (ms_dispatch thread) coordinates the message flow within the preset object storage component; the heartbeat thread (osd_srv_heartbt thread) maintains the connection status of the component with other related parts; the command interaction thread (tp_osd_cmd thread) processes external instructions. These threads work together to ensure that the preset object storage component completes related tasks stably and efficiently.

[0054] For threads in the preset object storage component, identification and classification are started based on preset classification rules, system source code, and thread processing configuration information of key businesses during the thread initialization phase or when the thread is running. For example, during the system startup process, the type of thread can be determined by analyzing the startup code and related configuration files. While the thread is running, the system will continue to monitor the thread's behavior and resource usage, which helps to classify the thread more accurately.

[0055] Step S12: based on the classification results and the preset thread processing functions corresponding to the thread types in the classification results, respectively determine whether each thread has been bound to the local CPU, so as to obtain thread determination results corresponding to each thread.

[0056] After obtaining the classification results, the classification results and the preset thread processing function are used to determine whether the thread is bound to the local CPU. For each thread category, there is a corresponding preset thread processing function. These preset thread processing functions can be built based on the behavior patterns and resource requirements of different types of threads. For example, for business processing threads, their preset thread processing functions will focus on the logical flow of data reading, writing and processing; for network message sending and receiving threads, their preset thread processing functions will focus on the processing of network communication protocols and related operations of data transmission.

[0057] Specifically, in the process of determining whether a thread is bound to a local CPU, it is necessary to go deep into the operating system level. The operating system provides a series of mechanisms and interfaces to obtain thread-CPU binding status information. By calling these system-level interfaces and combining them with preset thread processing functions, the binding status of each thread can be accurately determined. Taking the business processing thread as an example, its preset processing function will query the operating system for the CPU information to which the thread is currently bound during execution. If the query result shows that no CPU is bound or the binding status does not meet expectations, the corresponding thread judgment result will be generated.

[0058] Step S13: determining the corresponding core thread to be bound based on the judgment result of each thread, and determining the core binding configuration item information corresponding to the core thread to be bound using the classification result and the preset central processing unit resource allocation principle.

[0059] In this embodiment, the corresponding thread to be bound is determined based on the judgment result of each thread. Specifically, if the thread judgment result shows that the corresponding thread has been bound to the local central processing unit, the corresponding key business execution operation is triggered based on the thread; if the thread judgment result shows that the corresponding thread is not bound to the local central processing unit, the thread is determined to be a thread to be bound, and the operation of determining the thread with a bound core clarifies the target object for the subsequent binding core operation.

[0060] It should be emphasized that it is necessary to distinguish between the two states of thread initialization and running to determine the corresponding core binding configuration item information. When the thread is initialized, the core binding configuration item information can be determined based on relatively static parameters such as the thread type and number and the preset ratio threshold (which can be configured by the system default or dynamically adjusted based on actual needs) initially set by the system, so as to reasonably allocate CPU resources for each type of thread. When the thread is running, the system is in dynamic change, and the CPU load, thread execution status, and unbound resource collection items are changing in real time. The core binding configuration item information needs to be adjusted according to these dynamic factors to flexibly respond to situations such as business load fluctuations and changes in the network environment.

[0061] In a specific implementation, when the thread is initialized, the number of to-be-bound core threads whose corresponding thread types are business processing threads is determined based on the classification results, and the corresponding first core binding configuration item information is determined using the obtained number of first threads, the total number of current CPU threads, and the corresponding first preset ratio threshold (which can be configured by the system by default, or can be adjusted dynamically based on actual needs). For to-be-bound core threads whose thread types are message receiving and sending threads and message distribution threads, the number of to-be-bound core threads whose corresponding thread types are message receiving and sending threads and message distribution threads is determined based on the classification results, and the corresponding second core binding configuration item information is determined using the obtained number of second threads, the total number of current CPU threads, and the corresponding second preset ratio threshold (which can be configured by the system by default, or can be adjusted dynamically based on actual needs).

[0062] For the threads to be bound whose thread type is the heartbeat thread, the number of threads to be bound whose thread type is the heartbeat thread is determined based on the classification result, and the corresponding third core binding configuration item information is determined using the obtained third thread number, the total number of threads in the current central processing unit, the preset thread isolation rule, and the corresponding third preset ratio threshold (which can be configured by the system default or dynamically adjusted based on actual needs); wherein the preset thread isolation rule is constructed because the stable operation of the heartbeat thread is crucial to the reliability of the system, and the heartbeat thread needs to be isolated from other threads that may generate high loads on CPU resources to prevent the transmission of the heartbeat detection message from being interfered with. Therefore, it is determined whether the obtained third thread number is less than the first product value determined based on the total number of threads in the current central processing unit and the third preset ratio threshold to obtain the corresponding first numerical judgment result; if the first numerical judgment result indicates that it is not less than, the target resource collection item isolated from the resource collection item corresponding to the first core binding configuration item information is determined from the local central processing unit, and the corresponding third core binding configuration item information is determined based on the target resource collection item and the first product value.

[0063] Further, for the to-be-bound core threads whose thread type is the command interaction thread, the number of to-be-bound core threads whose thread type is the command interaction thread is determined based on the classification result, and the corresponding fourth core binding configuration item information is determined using the obtained fourth thread number, the total number of threads of the current central processing unit, the preset thread isolation rule, and the corresponding fourth preset ratio threshold (which can be configured by system default or dynamically adjusted based on actual needs). Similarly, the determination of the fourth core binding configuration item information also requires preset thread isolation rules. The command interaction thread needs to respond to external instructions in a timely manner, especially during key operations such as configuration changes, and its execution cannot be affected by other high-load threads. Therefore, it is determined whether the obtained fourth thread number is less than the second product value determined based on the total number of threads of the current central processing unit and the fourth preset ratio threshold to obtain the corresponding second numerical judgment result. If the second numerical judgment result indicates that it is not less than, the target resource set item isolated from the resource set item of the first core binding configuration item information is determined from the local central processing unit, and the corresponding fourth core binding configuration item information is determined based on the target resource set item and the second product value.

[0064] In another specific implementation, when the thread is running, the core binding configuration item information corresponding to the core thread to be bound is determined based on the total number of threads in the current central processing unit, the resource collection items that are not bound in the current local central processing unit, the classification results, and the preset ratio thresholds corresponding to each thread type. It should be explained that during the operation of the system, the load of the CPU and the running state of the thread may change. For example, as the data read and write requests increase or decrease, the load of the business processing thread will change accordingly; fluctuations in the network environment may affect the work intensity of the network message sending and receiving and distribution threads. The resource collection items that are not bound in the current local central processing unit are resources that can be reallocated to the core thread to be bound in this dynamic environment. By considering these factors in real time, the core binding configuration item information can be dynamically adjusted to make the binding of threads and CPU resources more flexible and optimized to adapt to different operating scenarios.

[0065] Step S14: dynamically bind the threads to be bound to cores based on the preset core binding interface and the corresponding core binding configuration item information to complete the corresponding thread core binding operation, and use the obtained core-bound threads to trigger the corresponding key business execution operation.

[0066] Specifically, in this embodiment, based on the first preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the first core binding configuration item information and the preset core binding interface are used to bind the core thread to be bound, whose thread type is a business processing thread, to complete the first core binding operation; based on the second preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the second core binding configuration item information and the preset core binding interface are used to bind the core thread to be bound, whose thread types are a message sending and receiving thread and a message distribution thread, to complete the second core binding operation; based on the third preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the third core binding configuration item information and the preset core binding interface are used to bind the core thread to be bound, whose thread type is a heartbeat thread, to complete the third core binding operation; based on the fourth preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the fourth core binding configuration item information and the preset core binding interface are used to bind the core thread to be bound, whose thread type is a command interaction thread, to complete the fourth core binding operation.

[0067] In this embodiment, after completing the above-mentioned various core binding operations, each core binding thread starts to trigger the corresponding key business execution operation. Since the entire distributed storage system is running under the optimized thread core binding strategy at this time, the business processing thread can efficiently handle data reading and writing and recovery business, the network message sending and receiving and distribution thread ensures the smoothness of information interaction, the heartbeat thread stably monitors the system connection status, and the command interaction thread responds to external instructions in a timely manner.

[0068] As can be seen from the above, when the thread is initialized or running, this application identifies and classifies the threads of the preset object storage component, determines whether the thread is bound to the local central processing unit based on the classification results and the thread processing function, and then determines the thread to be bound to the core based on the thread judgment result, and determines the core binding configuration item information in combination with the classification results and the central processing unit resource allocation principle, and dynamically binds the core to it using the preset core binding interface and core binding configuration item information, and finally triggers key business operations with the core-bound thread. In this way, this application can improve and optimize the binding strategy between threads and CPU resources, thereby improving the performance of the storage system and thus improving the user experience.

[0069] Combine the following Figure 2 , Figure 3 as well as Figure 4 The schematic diagram shown specifically illustrates the technical solution of the embodiment of the present application.

[0070] Specifically, in this embodiment, in a preset distributed storage system, by analyzing source code, such as Ceph-osd system source code, key business thread model information (i.e., thread processing configuration information of key business) and monitoring information during thread operation, threads in Ceph-osd are divided into osd_op_tp threads, msgr-worker threads, ms_dispatch threads, osd_srv_heartbt threads, tp_osd_cmd threads and other threads. Other threads may include Finisher threads and Safe_timer threads.

[0071] It should be noted that the osd_op_tp thread is the core thread of Ceph-osd, responsible for all key business processing, including data reading and writing, data recovery, etc. It is the busiest thread of Ceph-osd and also the thread with the largest concurrency. For data disks with HDD (Hard Disk Drive, Winchester hard disk) media, Ceph-osd has 5 osd_op_tp threads executing concurrently by default; and for data disks with SSD (Solid State Drive, solid state drive) media, Ceph-osd has 8 osd_op_tp threads executing concurrently by default. The msgr-worker thread is responsible for the network communication of Ceph-osd, receiving external network data and sending network data to the other end. Ceph-osd has 3 msgr-worker threads by default. The ms_dispatch thread is responsible for converting the network data received by the msgr-worker thread into Ceph internal messages, such as read and write requests, heartbeat messages, etc., and forwarding them to threads such as osd_op_tp for processing. Ceph-osd has 5 ms_dispatch threads. The osd_srv_heartbt thread is responsible for sending heartbeat detection messages to peer_osd (an object storage component that is equivalent to the preset object storage component). Ceph-osd has 1 osd_srv_heartbt thread. The tp_osd_cmd thread is responsible for processing external instructions issued to Ceph-osd, mainly for configuration changes. Ceph-osd has 1 tp_osd_cmd thread.

[0072] After classifying the threads, in order to avoid CPU resource preemption between threads, it is necessary to bind each type of thread to a different CPU, that is, to determine the thread binding core configuration item information (i.e. Figure 2The CPU collection configuration item in the binding core configuration item information includes the osd_op_tp_cpuset configuration item, the msgr-worker_cpuset configuration item, the ms_dispatch_cpuset configuration item, the osd_srv_heartbt_cpuset configuration item, and the tp_osd_cmd_cpuset configuration item.

[0073] Furthermore, in a specific implementation, for a scenario in which the current CPU is configured as 48 threads and 10 Ceph-osds are deployed on a single node, a process of determining the thread binding core configuration item information is specifically described.

[0074] The osd_op_tp thread is the main business processing thread of Ceph-osd, and a single osd has 5 osd_op_tp threads by default, and a single storage node has 10 Ceph-osds, so there are a total of 50 osd_op_tp threads. Therefore, 2 / 3 (that is, the first preset ratio threshold) of the CPU can be allocated to this thread, and the osd_op_tp_cpuset configuration item is osd_op_tp_cpuset="0-15,24-39". When the empty snapshot is recovered, 50 osd_op_tp threads on a single node are executed in high concurrency, occupying only 2 / 3 of the CPU, without affecting other threads from preempting the remaining 1 / 3 of the CPU.

[0075] The msgr-worker thread and ms_dispatch thread are responsible for receiving, sending and internal distribution of network messages. They are highly coupled and have busy business. Therefore, 1 / 8 of the CPU can be allocated to these two types of threads. The msgr-worker configuration item is msgr-worker_cpuset="16-18,40-42", and the ms_dispatch configuration item is ms_dispatch_cpuset="16-18,40-42".

[0076] In order to avoid the scenario where the osd_op_tp thread preempts CPU resources with high concurrency when recycling empty snapshots, it is necessary to isolate the osd_srv_heartbt thread from the CPU of the osd_op_tp thread. To do this, set the osd_srv_heartbt configuration item to osd_srv_heartbt_cpuset="19-20,43-44".

[0077] Similarly, to avoid the situation where the tp_osd_cmd thread cannot compete for CPU resources due to high concurrency of osd_op_tp preempting the CPU when recycling empty snapshots, it is necessary to isolate the tp_osd_cmd command issuing thread from the CPU of the osd_op_tp thread, osd_srv_heartbt_cpuset="21-22,45-46".

[0078] For other Ceph-osd threads, operating systems, and other services that are not bound to the CPU, they can use the remaining 1 / 24 of the CPU resources and can also preempt the above-allocated CPU resources.

[0079] After determining the core binding configuration item information, bind the thread to the CPU resource set specified by the core binding configuration item information, such as Figure 3 As shown in the figure, the processing function of the osd_op_tp thread is OSD::ShardedOpWQ::_process, and the specific code segment for binding the core can be shown as follows:

[0080] void OSD::ShardedOpWQ::_process (uint32_t thread_index, heartbeat_handle_d *hb){

[0081] cpu_set_t cpuset;

[0082] CPU_ZERO(&cpuset); / / Initialize CPU bit set

[0083] / / Get configuration items

[0084] size_t osd_op_tp_cpuset=g_conf().get_val< >("osd_op_tp_cpuset");

[0085] CPU_SET(osd_op_tp_cpuset, &cpuset); / / Set CPU bit set

[0086] / / Set the CPU affinity of the current thread

[0087] if (sched_setaffinity(0, sizeof(cpuset), &cpuset) == -1) {

[0088] perror("sched_setaffinity");

[0089] exit(EXIT_FAILURE);

[0090] }

[0091] / / Thread specific business processing

[0092] }

[0093] like Figure 4 As shown, the processing function of the msgr-worker thread is EventCenter::process_events, and the specific code segment for binding the core can be as follows:

[0094] int EventCenter::process_events{

[0095] cpu_set_t cpuset;

[0096] CPU_ZERO(&cpuset); / / Initialize CPU bit set

[0097] / / Get configuration items

[0098] size_t msgr-worker_cpuset=g_conf().get_val< >("msgr-worker_cpuset");

[0099] CPU_SET(osd_op_tp_cpuset, &cpuset); / / Set CPU bit set

[0100] / / Set the CPU affinity of the current thread

[0101] if (sched_setaffinity(0, sizeof(cpuset), &cpuset) == -1) {

[0102] perror("sched_setaffinity");

[0103] exit(EXIT_FAILURE);

[0104] }

[0105] / / Specific message sending and receiving processing

[0106] }

[0107] Similarly, modify the preset thread processing functions of the ms_dispatch thread, osd_srv_heartbt thread, and tp_osd_cmd thread respectively, read the thread configuration items in the core binding configuration item information, and finally bind each thread to the corresponding CPU resource set by calling the Linux core binding API (Application Programming Interface) and the sched_setaffinity function.

[0108] Accordingly, see Figure 5 As shown, the embodiment of the present application also provides a thread core binding device, which is applied to a preset distributed storage system, including:

[0109] The thread classification module 11 is used to identify and classify threads in the preset object storage component based on preset classification rules, system source code and thread processing configuration information of key services when the threads are initialized or running, so as to obtain corresponding classification results;

[0110] The thread determination module 12 is used to determine whether each of the threads has been bound to the local central processing unit based on the classification results and the preset thread processing functions corresponding to each thread type in the classification results, so as to obtain thread determination results corresponding to each of the threads;

[0111] An information determination module 13 is used to determine the corresponding core thread to be bound based on the judgment result of each thread, and determine the core binding configuration item information corresponding to the core thread to be bound by using the classification result and a preset central processing unit resource allocation principle;

[0112] The thread core binding module 14 is used to dynamically bind the threads to be bound to the cores based on the preset core binding interface and the corresponding core binding configuration item information to complete the corresponding thread core binding operation, and use the obtained core-bound threads to trigger the corresponding key business execution operation.

[0113] As can be seen from the above, when the thread is initialized or running, this application identifies and classifies the threads of the preset object storage component, determines whether the thread is bound to the local central processing unit based on the classification results and the thread processing function, and then determines the thread to be bound to the core based on the thread judgment result, and determines the core binding configuration item information in combination with the classification results and the central processing unit resource allocation principle, and dynamically binds the core to it using the preset core binding interface and core binding configuration item information, and finally triggers key business operations with the core-bound thread. In this way, this application can improve and optimize the binding strategy between threads and CPU resources, thereby improving the performance of the storage system and thus improving the user experience.

[0114] In a specific implementation, the information determination module 13 may specifically include:

[0115] A service execution unit, configured to trigger a corresponding key service execution operation based on the thread if the thread judgment result indicates that the corresponding thread has been bound to the local central processing unit;

[0116] The thread determination unit is used to determine that the thread is a thread to be bound to a core if the thread determination result indicates that the corresponding thread is not bound to the local central processing unit.

[0117] In a specific implementation, the information determination module 13 may specifically include:

[0118] The configuration information determination unit is used to determine the core binding configuration item information corresponding to the thread to be bound based on the total number of threads in the current central processing unit, the resource collection items that are not bound in the current local central processing unit, the classification results, and the preset ratio thresholds corresponding to each type of thread when the thread is running.

[0119] In a specific implementation, the information determination module 13 may specifically include:

[0120] A first information determination unit is used to determine, when the thread is initialized, the number of the to-be-bound core threads whose thread type is a business processing thread based on the classification result, and determine corresponding first core binding configuration item information using the obtained first thread number, the total number of threads of the current central processing unit, and the corresponding first preset ratio threshold;

[0121] A second information determination unit is used to determine the number of the to-be-bound core threads whose thread types are message sending and receiving threads and message distribution threads based on the classification result, and determine corresponding second core binding configuration item information using the obtained number of second threads, the total number of threads of the current central processing unit, and the corresponding second preset ratio threshold;

[0122] A third information determination unit is used to determine the number of the to-be-bound core threads whose corresponding thread type is a heartbeat thread based on the classification result, and determine corresponding third core binding configuration item information using the obtained number of third threads, the total number of threads of the current central processing unit, a preset thread isolation rule, and a corresponding third preset ratio threshold;

[0123] The fourth information determination unit is used to determine the number of the to-be-bound core threads whose corresponding thread type is the command interaction thread based on the classification result, and use the obtained fourth thread number, the total number of current central processing unit threads, the preset thread isolation rules and the corresponding fourth preset ratio threshold to determine the corresponding fourth core binding configuration item information.

[0124] In a specific implementation, the information determination module 13 may specifically include:

[0125] A first quantity determination unit, configured to determine whether the obtained third thread quantity is less than a first product value determined based on the total number of threads of the current central processing unit and a third preset ratio threshold value, so as to obtain a corresponding first numerical determination result;

[0126] The fifth information determination unit is used to determine the target resource set item isolated from the resource set item corresponding to the first core binding configuration item information from the local central processor if the first numerical judgment result indicates that it is not less than, and determine the corresponding third core binding configuration item information based on the target resource set item and the first product value.

[0127] In a specific implementation, the information determination module 13 may specifically include:

[0128] A second number determination unit, configured to determine whether the obtained fourth thread number is less than a second product value determined based on the total number of threads of the current central processing unit and a fourth preset ratio threshold, so as to obtain a corresponding second numerical value determination result;

[0129] The sixth information determination unit is used to determine a target resource set item isolated from the resource set item of the first core binding configuration item information from the local central processor if the second numerical judgment result indicates that it is not less than, and determine the corresponding fourth core binding configuration item information based on the target resource set item and the second product value.

[0130] In a specific implementation, the thread core binding module 14 may specifically include:

[0131] A first core binding unit, configured to call a preset core binding interface based on a first preset thread processing function, and perform core binding on the to-be-bound core thread whose thread type is the business processing thread by using a resource set item corresponding to the first core binding configuration item information and the preset core binding interface, so as to complete a first core binding operation;

[0132] A second core binding unit is used to call the preset core binding interface based on a second preset thread processing function, and use the resource collection item corresponding to the second core binding configuration item information and the preset core binding interface to bind the to-be-bound core threads whose thread types are the message sending and receiving thread and the message distributing thread to complete the second core binding operation;

[0133] A third core binding unit is used to call the preset core binding interface based on a third preset thread processing function, and use the resource collection item corresponding to the third core binding configuration item information and the preset core binding interface to bind the to-be-bound core thread whose thread type is the heartbeat thread to complete the third core binding operation;

[0134] The fourth core binding unit is used to call the preset core binding interface based on the fourth preset thread processing function, and use the resource collection item corresponding to the fourth core binding configuration item information and the preset core binding interface to bind the thread to be bound whose thread type is the command interaction thread to complete the fourth core binding operation.

[0135] Furthermore, the present application also discloses an electronic device. Figure 6 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the thread binding method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0136] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0137] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0138] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs that can be used to complete the thread binding method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0139] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the thread binding method disclosed above is implemented. For the specific steps of the method, reference can be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0140] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0141] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0142] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0143] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0144] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A thread-core binding method, characterized in that: Applied to the preset distributed storage system, including: During thread initialization or thread operation, threads in the preset object storage component are identified and classified based on preset classification rules, system source code, and thread processing configuration information of key businesses to obtain corresponding classification results; Based on the classification results and the preset thread processing functions corresponding to the thread types in the classification results, respectively, determining whether each of the threads has been bound to the local central processing unit, so as to obtain thread determination results corresponding to each of the threads; Determine the corresponding core thread to be bound based on the judgment result of each thread, and determine the core binding configuration item information corresponding to the core thread to be bound using the classification result and a preset central processing unit resource allocation principle; The threads to be bound to cores are dynamically bound to cores based on the preset core binding interface and the corresponding core binding configuration item information to complete the corresponding thread core binding operation, and the obtained core-bound threads are used to trigger corresponding key business execution operations.

2. The thread binding method according to claim 1, characterized in that: The determining the corresponding thread to be bound to the core based on the judgment result of each thread includes: If the thread determination result indicates that the corresponding thread has been bound to the local central processing unit, triggering a corresponding key business execution operation based on the thread; If the thread determination result indicates that the corresponding thread is not bound to the local central processing unit, the thread is determined to be a thread to be bound to a core.

3. The thread-core binding method according to claim 1, characterized in that: The determining the core binding configuration item information corresponding to the thread to be bound to a core by using the classification result and a preset central processing unit resource allocation principle includes: When the thread is running, the core binding configuration item information corresponding to the thread to be bound is determined based on the total number of threads in the current central processor, the unbound resource collection items in the current local central processor, the classification results and the preset ratio thresholds corresponding to each thread type.

4. The thread core binding method according to any one of claims 1 to 3, characterized in that: The determining the core binding configuration item information corresponding to the thread to be bound to a core by using the classification result and a preset central processing unit resource allocation principle includes: When the thread is initialized, the number of the to-be-bound core threads whose thread type is a business processing thread is determined based on the classification result, and the corresponding first core binding configuration item information is determined using the obtained first thread number, the total number of threads of the current central processing unit, and the corresponding first preset ratio threshold; Determine the number of the to-be-bound core threads whose thread types are message sending and receiving threads and message distribution threads based on the classification result, and determine the corresponding second core binding configuration item information using the obtained number of second threads, the total number of threads of the current central processing unit, and the corresponding second preset ratio threshold; Determine the number of the to-be-bound core threads whose thread type is a heartbeat thread based on the classification result, and determine corresponding third core binding configuration item information using the obtained number of third threads, the total number of threads of the current central processing unit, a preset thread isolation rule, and a corresponding third preset ratio threshold; Based on the classification result, the number of threads to be bound to the core corresponding to the thread type of command interaction thread is determined, and the corresponding fourth core binding configuration item information is determined using the obtained fourth thread number, the total number of threads of the current central processing unit, the preset thread isolation rule and the corresponding fourth preset ratio threshold.

5. The thread-core binding method according to claim 4, characterized in that: The method of determining corresponding third core binding configuration item information by using the obtained third thread quantity, the total number of threads of the current central processing unit, a preset thread isolation rule, and a corresponding third preset ratio threshold comprises: Determine whether the obtained third number of threads is less than a first product value determined based on the total number of threads of the current central processing unit and a third preset ratio threshold value, so as to obtain a corresponding first numerical determination result; If the first numerical judgment result indicates that it is not less than, a target resource set item isolated from the resource set item corresponding to the first core binding configuration item information is determined from the local central processor, and corresponding third core binding configuration item information is determined based on the target resource set item and the first product value.

6. The thread-core binding method according to claim 4, characterized in that: The method of determining corresponding fourth core binding configuration item information by using the obtained fourth thread quantity, the total number of threads of the current central processing unit, the preset thread isolation rule, and the corresponding fourth preset ratio threshold comprises: Determine whether the obtained fourth thread quantity is less than a second product value determined based on the total number of threads of the current central processing unit and a fourth preset ratio threshold value, so as to obtain a corresponding second numerical determination result; If the second numerical judgment result indicates that it is not less than, a target resource set item isolated from the resource set item of the first core binding configuration item information is determined from the local central processor, and corresponding fourth core binding configuration item information is determined based on the target resource set item and the second product value.

7. The thread-core binding method according to claim 4, characterized in that: The dynamically binding the to-be-bound core thread based on the preset core binding interface and the corresponding core binding configuration item information includes: Based on the first preset thread processing function, a preset core binding interface is called, and the resource collection item corresponding to the first core binding configuration item information and the preset core binding interface are used to bind the core to be bound thread whose thread type is the business processing thread, so as to complete the first core binding operation; Based on the second preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the second core binding configuration item information and the preset core binding interface are used to bind the to-be-bound core threads whose thread types are the message sending and receiving thread and the message distributing thread, so as to complete the second core binding operation; Based on the third preset thread processing function, the preset core binding interface is called, and the resource collection item corresponding to the third core binding configuration item information and the preset core binding interface are used to bind the core to be bound thread whose thread type is the heartbeat thread, so as to complete the third core binding operation; The preset core binding interface is called based on the fourth preset thread processing function, and the resource collection item corresponding to the fourth core binding configuration item information and the preset core binding interface are used to bind the core to be bound whose thread type is the command interaction thread to complete the fourth core binding operation.

8. A thread core binding device, characterized in that: Applied to the preset distributed storage system, including: The thread classification module is used to identify and classify threads in the preset object storage component based on preset classification rules, system source code and thread processing configuration information of key businesses during thread initialization or thread running, so as to obtain corresponding classification results; A thread determination module, used to determine whether each of the threads has been bound to a local central processing unit based on the classification results and preset thread processing functions corresponding to each thread type in the classification results, so as to obtain thread determination results corresponding to each of the threads; An information determination module, configured to determine a corresponding core thread to be bound based on the judgment result of each of the threads, and to determine core binding configuration item information corresponding to the core thread to be bound using the classification result and a preset central processing unit resource allocation principle; The thread core binding module is used to dynamically bind the to-be-bound core threads based on the preset core binding interface and the corresponding core binding configuration item information to complete the corresponding thread core binding operation, and use the obtained core-bound threads to trigger the corresponding key business execution operation.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the thread core binding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the thread-core binding method as described in any one of claims 1 to 7 is implemented.

Images