Redis optimization method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a Redis optimization method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: when Redis is started, acquiring CPU (Central Processing Unit) architecture information of a server; based on the CPU architecture information of the server, effective logic cores are determined, and the effective logic cores comprise a first logic core and at least one second logic core; determining a main thread, a sub-process and a background thread of the Redis; and binding the main thread with the first logic core, and binding the sub-process and the background thread with the at least one second logic core. By adopting the method, nuclear competition can be avoided, and the Redis processing performance is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a Redis optimization method, apparatus, computer device, readable storage medium, and program product. Background Technology

[0002] With the development of internet technology, Redis, as a high-performance key-value store, is widely used in scenarios such as caching, message queues, and leaderboards. However, multi-CPU architecture and multi-core CPU architecture also have a significant impact on the performance of Redis in processing requests.

[0003] Typically, a CPU processor has multiple physical cores, each with its own private L1 and L2 caches. Different physical cores within the same CPU also share an L3 cache. This cache stores frequently used data or instructions; higher-level caches have larger storage spaces but slower data processing speeds. In mainstream CPU processors, each physical core also runs multiple hyper-threads, also known as logical cores. Logical cores within the same physical core share the L1 and L2 caches, and different CPUs are connected via a bus.

[0004] When a Redis application runs on a single CPU core, it records runtime context information such as stack pointers and register values. It also caches the most frequently accessed instructions and data in L1 and L2 caches to improve execution speed. However, for multi-core CPU servers, when the application switches to a new physical CPU core, the runtime context information needs to be reloaded onto the new CPU core. Furthermore, the L1 and L2 caches of the new CPU core also need to be retrieved from the L3 cache or even memory to reload the frequently accessed instructions and data. The switching of CPU cores and cache loading will increase the processing time of the program and affect performance. Summary of the Invention

[0005] Therefore, it is necessary to provide a Redis optimization method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can avoid core contention and improve Redis processing performance in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a Redis optimization method, the method comprising:

[0007] When Redis starts, obtain the server's CPU architecture information;

[0008] Based on the CPU architecture information of the server, valid logical cores are determined, including a first logical core and at least one second logical core;

[0009] Determine the main thread, child processes, and background threads of Redis;

[0010] The main thread is bound to the first logical core, and the child process and background thread are bound to the at least one second logical core.

[0011] In one embodiment, binding the child process and background thread to the at least one second logical core includes:

[0012] When there is only one second logical core, both the child process and the background thread are bound to the second logical core;

[0013] When the second logical core includes at least two, and the number of the second logical cores is greater than or equal to the sum of the number of the child processes and the number of the background threads, the child processes and the background threads are bound to different second logical cores;

[0014] When the second logical core includes at least two, and the number of the second logical cores is less than the sum of the number of the child processes and the number of the background threads, the child processes and the background threads are bound to the second logical cores, and at least one second logical core is bound to multiple child processes and / or background threads.

[0015] In one embodiment, binding the main thread to the first logical core and binding the child process and background thread to the at least one second logical core includes:

[0016] Identify the functions in each file corresponding to the main thread, the child process, and the background thread. Each function in each file is a function used to create the corresponding thread or child process.

[0017] The function is programmed to bind each thread or subprocess to its corresponding logical core.

[0018] In one embodiment, determining the valid logical cores based on the server's CPU architecture information includes:

[0019] A CPU core bitmap is created based on the CPU architecture information of the server, wherein each bit in the CPU core bitmap is used to represent a logical core on the server;

[0020] Set each bit in the CPU core bitmap to its default value;

[0021] Based on the logical core started in the server, the corresponding bit in the CPU core bitmap is modified to a target value, which is used to identify the logical core as a valid logical core.

[0022] In one embodiment, the method further includes:

[0023] The CPU utilization of the logical core corresponding to each thread or subprocess can be obtained through the preset API interface;

[0024] If the CPU utilization of the logic core is greater than a first threshold, a target logic core with a CPU utilization less than a second threshold is obtained, wherein the first threshold is greater than the second threshold.

[0025] The threads or subprocesses corresponding to the logical cores whose CPU utilization is greater than the first threshold are adjusted to the target logical core.

[0026] In one embodiment, before acquiring the target logic core with CPU utilization less than the second threshold, the method further includes:

[0027] Determine the CPU utilization rate of each thread or subprocess in the logical core where the CPU utilization rate is greater than the first threshold;

[0028] If the ratio is greater than or equal to the third threshold, the logical cores of each thread or subprocess corresponding to the Redis remain unchanged. The ratio is the ratio of the CPU utilization rate of each thread or subprocess corresponding to the Redis to the CPU utilization rate of the logical cores whose CPU utilization rate is greater than the first threshold.

[0029] If the ratio is less than the third threshold, continue to execute the step of obtaining the target logic core with a CPU utilization rate less than the second threshold.

[0030] Secondly, this application also provides a Redis optimization device, the device comprising:

[0031] The architecture information acquisition module is used to obtain the server's CPU architecture information when Redis starts up;

[0032] The first determining module is used to determine the valid logical cores based on the CPU architecture information of the server, wherein the valid logical cores include a first logical core and at least one second logical core;

[0033] The second determination module is used to determine the main thread, child processes, and background threads of Redis;

[0034] A binding module is used to bind the main thread to the first logical core, and to bind the child process and background thread to at least one second logical core.

[0035] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods in any of the above embodiments.

[0037] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0038] The aforementioned Redis optimization method, apparatus, computer equipment, computer-readable storage medium, and computer program product, upon Redis startup, acquire the server's CPU architecture information; based on the server's CPU architecture information, determine the effective logical cores, including a first logical core and at least one second logical core; determine the Redis main thread, child processes, and background threads; bind the main thread to the first logical core, and bind the child processes and background threads to the at least one second logical core. This pre-binds each thread or child process corresponding to Redis to different logical cores, primarily binding the main thread to the first logical core and the child processes and background threads to the at least one second logical core. This avoids frequent switching of the core corresponding to the main thread. At the source code level, CPU affinity binding is directly applied to the main thread and each child process and background thread, optimizing Redis processing performance. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating a Redis optimization method in one embodiment;

[0041] Figure 2 This is a structural diagram of the server's CPU in one embodiment;

[0042] Figure 3 This is a schematic diagram of the binding relationship in one embodiment;

[0043] Figure 4 This is a flowchart of the dynamic adjustment steps in one embodiment;

[0044] Figure 5 This is a block diagram of the Redis optimization device in one embodiment;

[0045] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0048] In one embodiment, such as Figure 1 As shown, a Redis optimization method is provided. This embodiment illustrates the method's application to a terminal, but it is understood that the method can also be applied to a server, and to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0049] S102: Obtain the server's CPU architecture information when Redis starts.

[0050] The server's CPU architecture information includes multiple layers, which sequentially include processors, physical cores, and logical cores. Each layer specifies the number of cores and their relative numbering. A server may include multiple CPU processors, each CPU processor may include multiple physical cores, and each physical core may include multiple logical cores. The relative numbering information can be derived by uniquely encoding each processor, physical core, and logical core. For example, processors, physical cores, and logical cores can be encoded using different symbols; processors might use uppercase letters, physical cores use lowercase letters, and logical cores use Arabic numerals. For instance, the first logical core under the first physical core of the first processor could be encoded as Aa1, uniquely indicating the first logical core under the first physical core of the first processor. It should be noted that this is merely an example and does not impose any limit on the number of processors, physical cores, and logical cores.

[0051] Combination Figure 2 As shown, Figure 2This is a structural diagram of a server's CPU in one embodiment. The server may include multiple CPU processors connected via a bus. Each CPU processor includes multiple physical cores, each with its own private L1 and L2 caches. Different physical cores within the same CPU processor also share an L3 cache. The cache stores frequently used data or instructions; higher-level caches have larger spaces but slower data processing speeds. Each physical core in the CPU processor also runs multiple hyper-threads, also known as logical cores. A physical core includes at least one logical core, and logical cores within the same physical core share the L1 and L2 caches.

[0052] In this embodiment, when Redis starts, the CPU processor can automatically identify the CPU structure information.

[0053] S104: Based on the server's CPU architecture information, determine the valid logical cores, which include a first logical core and at least one second logical core.

[0054] A valid logic core is a running logic core. In some optional embodiments, valid logic cores can also be obtained by filtering running logic cores based on screening criteria, where the screening criteria may include a logic core's load rate being less than a load rate threshold. In other embodiments, the screening criteria may also include other conditions.

[0055] In this application, valid logical cores can be determined from the server's CPU architecture information based on filtering criteria and whether the server is running. For ease of description, the valid logical cores include a first logical core and at least one second logical core. Here, the terms "first logical core" and "at least one second logical core" are used for convenience only, and the first logical core and at least one second logical core are the same.

[0056] S106: Determine the main thread, child processes, and background threads of Redis.

[0057] Redis consists of a main thread, child processes, and background threads. The main thread is used by Redis to perform the main tasks. The background threads of Redis are created in the bioProcessBackgroundJobs function in the bio.c file. The child processes mainly include RDB child processes and AOF child processes. The RDB child process is implemented in the rdbSaveBackground function in the rdb.c file, and the AOF child process is implemented in the rewriteAppendOnlyFileBackground function in the aof.c file.

[0058] For ease of understanding, the relationship between the main thread, child processes, and background threads is given below:

[0059]

[0060] The main thread is responsible for the core business logic, while the child process is responsible for independent, heavy tasks that cannot affect the main thread, such as persistence. The background thread handles highly repetitive (network I / O read / write) or time-consuming (asynchronous deletion of large keys) transactions.

[0061] S108: Bind the main thread to the first logical core, and bind child processes and background threads to at least one second logical core.

[0062] In this embodiment, since the main thread has the greatest impact on Redis, it is necessary to minimize the impact on the main thread. Therefore, the main thread is bound to a single first logical core. However, the real-time requirements of the child processes and background threads are not high. Therefore, when logical core resources are limited, the child processes and background threads can be bound to the same second logical core. When logical core resources are sufficient, each child process and background thread can be bound to a second logical core.

[0063] In some alternative embodiments, if there are sufficient resources for the logical cores, the logical cores can be screened, for example, the logical core with the lowest load rate can be selected as the first logical core, so as to provide as many resources as possible to the main thread and reduce the impact on the main thread.

[0064] In some optional embodiments, binding the main thread to a first logical core and binding the child process and background thread to at least one second logical core includes: determining each function in each file corresponding to the main thread, child process, and background thread, wherein each function in each file is a function used to create the corresponding thread or child process; and programming in the function to implement the binding of each thread or child process to the corresponding logical core.

[0065] The main thread, child processes, and background threads are all created in functions within their respective files. Specifically, Redis's background thread is created in the `bioProcessBackgroundJobs` function of the `bio.c` file, where the core binding operation for the background thread is programmed. The child process for generating Redis RDBs is implemented in the `rdbSaveBackground` function of the `rdb.c` file, where the core binding operation for the RDB child process is also programmed. Similarly, the child process for rewriting Redis's AOF logs is implemented in the `rewriteAppendOnlyFileBackground` function of the `aof.c` file, where the core binding operation for the AOF log rewriting child process is also programmed. In this way, by modifying the Redis kernel engine code, the affinity binding between the main thread, child processes, and background threads and the CPU cores is achieved.

[0066] The aforementioned Redis optimization method involves obtaining the server's CPU architecture information upon Redis startup; based on this information, determining the effective logical cores, which include a first logical core and at least one second logical core; identifying the Redis main thread, child processes, and background threads; binding the main thread to the first logical core, and binding the child processes and background threads to at least one second logical core. This pre-binding of each Redis thread to a different logical core, primarily the main thread to the first logical core and the child processes and background threads to at least one second logical core, avoids frequent core switching for the main thread. At the source code level, CPU affinity binding is directly applied to the main thread, child processes, and background threads, optimizing Redis processing performance.

[0067] In some optional embodiments, binding child processes and background threads to at least one second logical core includes: binding both child processes and background threads to the second logical core when there is only one second logical core; binding child processes and background threads to different second logical cores when there are at least two second logical cores and the number of second logical cores is greater than or equal to the sum of the number of child processes and the number of background threads; and binding child processes and background threads to the second logical cores when there are at least two second logical cores and the number of second logical cores is less than the sum of the number of child processes and the number of background threads, wherein at least one second logical core is bound to multiple child processes and / or background threads.

[0068] For ease of understanding, the following is combined Figure 3As shown, in the case where there are only two logical cores—a first logical core and a second logical core—the main thread is bound to the first logical core, while child processes and background threads are bound to the second logical core. Optionally, the CPU load rate of the first logical core before it is bound to the main thread is lower than that of the second logical core, thus minimizing the impact on the main thread.

[0069] In the case of three logical cores—one first logical core and two second logical cores—the main thread is bound to the first logical core, the child process is bound to one of the second logical cores, and the background thread is bound to one of the second logical cores. Optionally, the CPU load rate of the first logical core before it is bound to the main thread is lower than that of the second logical cores, thus minimizing the impact on the main thread.

[0070] When there are four or more logical cores (one first logical core and at least three second logical cores), the main thread is bound to the first logical core, one child process is bound to one second logical core, another child process is bound to one second logical core, and a background thread is bound to one second logical core. Optionally, the CPU load rate of the first logical core before it is bound to the main thread is lower than that of the second logical cores, thus minimizing the impact on the main thread.

[0071] In some optional embodiments, the allocation order can be defined to improve efficiency. Specifically, allocation can be done first to the main thread, then to the background thread for asynchronously closing large file descriptors (BIO_CLOSE_FILE), then to the background thread for asynchronously executing AOF file flushing (BIO_AOF_FSYNC), then to the background thread for asynchronously delaying memory release (BIO_LAZY_FREE), and finally to child processes. Furthermore, if logical core resources are sufficient, multiple logical cores can be allocated to child processes to quickly complete their corresponding business logic. For ease of understanding, assuming eight logical cores exist, the first logical core is bound to the main thread to ensure its exclusivity and minimum latency. The second, third, and fourth logical cores are bound to the three background threads, and the remaining logical cores are bound to child processes. These child processes are CPU-intensive and require multiple cores to complete quickly, avoiding prolonged occupation of the main thread's resources.

[0072] In this embodiment, only two child processes and one background thread are used as examples for illustration. In other embodiments, other threads may also be included, and no specific limitation is made here.

[0073] In some optional embodiments, determining a valid logical core based on the server's CPU architecture information includes: creating a CPU core bitmap based on the server's CPU architecture information, wherein each bit in the CPU core bitmap represents a logical core on the server; setting each bit in the CPU core bitmap to a default value; and modifying the corresponding bit in the CPU core bitmap to a target value based on the logical core that is started on the server, wherein the target value is used to identify the logical core as a valid logical core.

[0074] The CPU core bitmap is generated based on CPU architecture information. It consists of multiple bits, each representing a logical core on the server. For example, each logical core in each physical core of each processor corresponds to one bit in the CPU core bitmap. To avoid confusion, each logical core in each physical core of each processor is sorted according to the order of processor, physical core, and logical core. Then, the logical core corresponding to each bit in the CPU core bitmap is determined based on this sorting.

[0075] The CPU architecture information includes the logical cores of each physical core of each processor in the server. Therefore, each bit in the CPU core bitmap is set to a default value. Subsequently, based on the logical cores started in the server, the corresponding bits in the CPU core bitmap are modified to target values. Here, the started logical core is considered a valid logical core. This target value is used to identify the logical core as valid, meaning it can be bound to a thread or child process in Redis. In other embodiments, each bit in the CPU core bitmap also includes CPU load information. This allows for selection of both valid and target logical cores based on the target value during binding, minimizing subsequent core migration operations.

[0076] For ease of understanding, this application modifies the Redis source code to bind the main thread of the Redis instance to a single logical core; background threads and child processes are bound to different logical cores to avoid competing for CPU resources with the main thread. The Redis kernel threads primarily utilize the core-binding related data structures and functions provided by the operating system as follows:

[0077] The cpu_set_t data structure creates a CPU core bitmap, where each bit represents a logical CPU core on the server.

[0078] The CPU_ZERO(&mask) function takes a bitmap of a cpu_set_t structure as input and sets all bits in the bitmap to 0.

[0079] The CPU_SET(self_id % num, &mask) function takes the CPU logical core number and the cpu_set_t bitmap as parameters and sets the bit in the bitmap corresponding to the input logical core number to 1.

[0080] The functions pthread_setaffinity_np(pid, sizeof(cpu_set_t), &mask) and sched_setaffinity(self_id, sizeof(mask), &mask) take a thread ID and cpu_set_t as parameters, check which bit in cpu_set_t is 1, and then bind the thread or child process represented by the input ID to the corresponding logical core.

[0081] For ease of understanding, the specific implementation logic of core binding is as follows:

[0082] 1. First, create a bitmap variable of the cpu_set_t structure.

[0083] 2. Call the CPU_ZERO function to set all bits of the cpu_set_t structure bitmap variable to 0.

[0084] 3. Call the CPU_SET function to set the corresponding position of the logical core number to be bound to the cpu_set_t structure bitmap variable to 1.

[0085] 4. Call the pthread_setaffinity_np or sched_setaffinity function to bind the program to the logical core that is set to 1 in the cpu_set_t structure bitmap.

[0086] The following is a code example of how to implement background thread core binding:

[0087] void *bioProcessBackgroundJobs(void *arg){

[0088] ...

[0089] / / Get CPU count

[0090] int cpu_num = sysconf(_SC_NPROCESSORS_CONF);

[0091] pid_t self_id = getpid();

[0092] / / Create bitmap variable

[0093] cpu_set_t mask;

[0094] / / Set all bits of the bitmap variable to 0

[0095] CPU_ZERO(&mask);

[0096] / / Bind to a specific CPU core and set the corresponding bit in the bitmap to 1.

[0097] CPU_SET(self_id % cpu_num, &mask);

[0098] / / Set affinity to bind the program to the logical core that is set to 1 in the cpu_set_t structure bitmap.

[0099] pthread_setaffinity_np(self_id, sizeof(mask), &mask);

[0100] / / Actual thread function operation

[0101] ...

[0102] }

[0103] The following is a code example of how to implement core binding for child processes:

[0104] int rdbSaveBackground(char *filename, rdbSaveInfo *rsi){

[0105] ...

[0106] / / Create a child process using fork

[0107] pid_t childpid = redisFork(CHILD_TYPE_RDB);

[0108] if(childpid < 0){

[0109] printf("fork error\n");

[0110] }else if(!childpid ){

[0111] / / Child process code section

[0112] cpu_set_t cpuset; / / Create a bitmap variable

[0113] CPU_ZERO(&cpu_set); / / Set all bits of the bitmap variable to 0

[0114] CPU_SET(3, &cpuset); / / Sets the 3rd bit of the bitmap to 1

[0115] sched_setaffinity(0, sizeof(cpuset), &cpuset); / / Binds the program to logical core 3.

[0116] / / Actual subprocess work

[0117] ...

[0118] }

[0119] ...

[0120] }

[0121] In the above embodiments, the Redis optimization method based on CPU affinity binding can significantly improve the performance of Redis under multi-core CPU architecture by reducing CPU resource contention, reducing context switching, and increasing cache hit rate.

[0122] In some alternative embodiments, the above method further includes a dynamic adjustment step, combined with Figure 4 As shown, Figure 4 Here is a flowchart of a dynamic adjustment step in one embodiment, which includes:

[0123] S402: Obtain the CPU utilization of the logical core corresponding to each thread or subprocess through the preset API interface.

[0124] During server operation, the CPU utilization rate of each logical core can be obtained, that is, the CPU load rate of each logical core. For example, information such as the CPU utilization rate of the physical core corresponding to each thread or subprocess can be obtained through APIs such as getrusage and pthread_getschedparam, so as to monitor the load of each physical core in the CPU in real time.

[0125] S404: If the CPU utilization of a logic core is greater than the first threshold, obtain a target logic core with a CPU utilization less than the second threshold, where the first threshold is greater than the second threshold.

[0126] S406: Adjust the thread or subprocess corresponding to the logical core whose CPU utilization is greater than the first threshold to the target logical core.

[0127] If the CPU utilization of a logical core is greater than the first threshold, that is, if the load rate of the logical core is high, such as when the number of machine cores is limited and the load is high, then threads or subprocesses can be migrated to logical cores with lower load rates to reduce context switching and thus improve overall performance.

[0128] Therefore, if the CPU utilization of the logical core to which a thread or subprocess is bound is determined to be greater than the first threshold, then the target logical core with a CPU utilization less than the second threshold, i.e., a logical core with a lower load rate, is selected. This allows the thread or subprocess corresponding to the logical core with a CPU utilization greater than the first threshold to be moved to the target logical core.

[0129] During the adjustment process, the number of target logical cores with CPU utilization below a second threshold can be determined first. If this number is greater than one, the logical core with the lowest CPU utilization is selected as the target logical core. If this number is equal to one, the unique logical core is selected as the target logical core.

[0130] The threads or subprocesses corresponding to logical cores whose CPU utilization exceeds the first threshold can include main threads, background threads, and subprocesses, without specific restrictions.

[0131] To facilitate understanding, let's illustrate with an example. Suppose the CPU utilization of the first logical core to which the main thread is bound is 80%, the CPU utilization of the second logical core to which the child process is bound is 50%, and the CPU utilization of the third logical core to which the background thread is bound is 20%. The first threshold is 60%. Therefore, it is determined that the CPU utilization of the first logical core to which the main thread is bound is greater than the first threshold, so a core shifting operation is required, that is, moving the main thread to another logical core. To reduce the risk of shifting cores again after the initial shift, a target logical core with a CPU utilization lower than the second threshold is selected. In this case, the second threshold is 30%, so the third logical core to which the background thread is bound can be selected as the target logical core, and the main thread is moved to this target logical core. In other embodiments, if there are other target logical cores with CPU utilization lower than the second threshold, the main thread can be moved to one of these target logical cores first, thus ensuring that the main thread and the background thread are still bound to different logical cores.

[0132] The values ​​of the first threshold and the second threshold can be dynamically set by the user according to the actual situation. The first threshold is used to filter out logic cores with high CPU utilization, and the second threshold is used to filter out logic cores with low CPU utilization as target logic cores for core migration. No specific limitation is made here.

[0133] In some optional embodiments, before obtaining the target logical core with a CPU utilization rate less than the second threshold, the method further includes: determining the CPU utilization rate of each thread or subprocess in the logical core with a CPU utilization rate greater than the first threshold; if the ratio is greater than or equal to the third threshold, keeping the logical cores of each thread or subprocess corresponding to Redis unchanged, and the ratio being the ratio of the CPU utilization rate of each thread or subprocess corresponding to Redis to the CPU utilization rate of the logical core with a CPU utilization rate greater than the first threshold; if the ratio is less than the third threshold, continuing to execute the step of obtaining the target logical core with a CPU utilization rate less than the second threshold.

[0134] In this application, after determining the logical core with CPU utilization exceeding the first threshold, the CPU utilization of each thread or subprocess within the logical core with CPU utilization exceeding the first threshold is also obtained. Here, the threads or subprocesses include the threads or subprocesses corresponding to Redis, as well as the threads or subprocesses corresponding to other services. Then, based on the ratio of the CPU utilization of each thread or subprocess corresponding to Redis to the CPU utilization of the logical core with CPU utilization exceeding the first threshold, if the ratio is greater than or equal to the third threshold, the ratio is used to measure the threads in the logical core with CPU utilization exceeding the first threshold that cause the CPU utilization to exceed the first threshold, that is, to explain the contribution of each thread or subprocess in the logical core to the CPU utilization. If the contribution of each thread or subprocess corresponding to Redis is greater than the third threshold, it means that the high CPU utilization of the CPU's logical core is caused by the Redis thread or subprocess. Therefore, there is no need to determine the logical core migration action. If the ratio is less than the third threshold, it means that the high CPU utilization of the CPU's logical core is not caused by the Redis thread or subprocess. Therefore, a core migration operation is required, that is, to find the target logical core with a CPU utilization less than the second threshold and then perform the core migration operation.

[0135] For ease of understanding, continuing the previous example, the CPU utilization of the first logical core bound to the main thread is 80%. This first logical core runs the Redis main thread and other business threads. At this time, the CPU utilization of the Redis main thread is 60%, and the CPU utilization of the other business threads is 20%. The ratio of the Redis main thread's CPU utilization of 60% to the CPU utilization of the first logical core bound to the main thread of 80% is 75%. The third threshold is used to measure the contribution of each thread or subprocess in the logical core to the CPU utilization. Therefore, this third threshold can be greater than or equal to 50%, which users can adjust as needed. Thus, 75% is greater than 50% (taking a third threshold of 50% as an example), indicating that the high CPU utilization of the first logical core is caused by the Redis main thread, and therefore, core migration is unnecessary.

[0136] In the above embodiments, during operation, information such as CPU utilization of each thread or subprocess is obtained through APIs such as getrusage and pthread_getschedparam, and the CPU load is monitored in real time. The binding relationship between threads and CPU cores is dynamically adjusted according to a certain threshold strategy. Especially when the number of machine cores is limited and the load is high, migrating threads to cores with lower load can reduce context switching and thus improve overall performance.

[0137] In the above embodiments, CPU affinity binding can improve Redis's performance. It avoids the tedious manual core binding operation, simplifying the deployment and management of Redis instances. By optimizing the binding of threads and CPU cores, it improves CPU resource utilization. It provides performance monitoring and dynamic adjustment mechanisms, enabling Redis instances to adapt to different workloads and maintain optimal performance. Through these methods, CPU affinity binding of Redis instances can significantly reduce the frequency of context switching, improve cache hit rate, thereby reducing latency, increasing throughput, and enhancing Redis performance.

[0138] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0139] Based on the same inventive concept, this application also provides a Redis optimization device for implementing the Redis optimization method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more Redis optimization device embodiments provided below can be found in the limitations of the Redis optimization method described above, and will not be repeated here.

[0140] In one exemplary embodiment, such as Figure 5 As shown, a Redis optimization device is provided, including: an architecture information acquisition module 501, a first determination module 502, a second determination module 503, and a binding module 504, wherein:

[0141] The architecture information acquisition module 501 is used to acquire the server's CPU architecture information when Redis starts up;

[0142] The first determining module 502 is used to determine the valid logical cores based on the CPU architecture information of the server, wherein the valid logical cores include a first logical core and at least one second logical core;

[0143] The second determination module 503 is used to determine the main thread, child process, and background thread of Redis;

[0144] The binding module 504 is used to bind the main thread to the first logical core, and to bind the child process and background thread to the at least one second logical core.

[0145] In some optional embodiments, the binding module 504 is specifically used to bind both the child process and the background thread to the second logical core when there is only one second logical core; to bind the child process and the background thread to different second logical cores when there are at least two second logical cores and the number of second logical cores is greater than or equal to the sum of the number of child processes and the number of background threads; and to bind the child process and the background thread to the second logical core when there are at least two second logical cores and the number of second logical cores is less than the sum of the number of child processes and the number of background threads, wherein at least one second logical core is bound to multiple child processes and / or background threads.

[0146] In some optional embodiments, the binding module 504 is specifically used to determine each function in each file corresponding to the main thread, the child process, and the background thread, wherein each function in each file is a function used to create the corresponding thread or child process; and to programmatically implement the binding of each thread or child process to the corresponding logical core in the function.

[0147] In some optional embodiments, the first determining module 502 is specifically used to create a CPU core bitmap based on the CPU architecture information of the server, wherein each bit in the CPU core bitmap is used to represent a logical core on the server; set each bit in the CPU core bitmap to a default value; and modify the corresponding bit in the CPU core bitmap to a target value based on the logical core started in the server, wherein the target value is used to identify the logical core as a valid logical core.

[0148] In some optional embodiments, the above-mentioned device further includes: a dynamic adjustment module, configured to obtain the CPU utilization rate of the logical core corresponding to each thread or sub-process through a preset API interface; if the CPU utilization rate of the logical core is greater than a first threshold, obtain a target logical core with a CPU utilization rate less than a second threshold, wherein the first threshold is greater than the second threshold; and adjust the thread or sub-process corresponding to the logical core with the CPU utilization rate greater than the first threshold to the target logical core.

[0149] In some optional embodiments, the dynamic adjustment module is further configured to determine the CPU utilization rate of each thread or subprocess in the logical cores whose CPU utilization rate is greater than a first threshold; if the ratio is greater than or equal to a third threshold, keep the logical cores of each thread or subprocess corresponding to Redis unchanged, the ratio being the ratio of the CPU utilization rate of each thread or subprocess corresponding to Redis to the CPU utilization rate of the logical cores whose CPU utilization rate is greater than the first threshold; if the ratio is less than the third threshold, continue to execute the step of obtaining the target logical cores whose CPU utilization rate is less than a second threshold.

[0150] The modules in the aforementioned Redis optimization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0151] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores the data involved in the aforementioned methods. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a Redis optimization method.

[0152] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0158] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0159] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A Redis optimization method, characterized in that, The method includes: When Redis starts, obtain the server's CPU architecture information; Based on the CPU architecture information of the server, valid logical cores are determined, including a first logical core and at least one second logical core; Determine the main thread, child processes, and background threads of Redis; The main thread is bound to the first logical core, and the child process and background thread are bound to the at least one second logical core.

2. The method according to claim 1, characterized in that, The binding of the child process and background thread to the at least one second logical core includes: When there is only one second logical core, both the child process and the background thread are bound to the second logical core; When the second logical core includes at least two, and the number of the second logical cores is greater than or equal to the sum of the number of the child processes and the number of the background threads, the child processes and the background threads are bound to different second logical cores; When the second logical core includes at least two, and the number of the second logical cores is less than the sum of the number of the child processes and the number of the background threads, the child processes and the background threads are bound to the second logical cores, and at least one second logical core is bound to multiple child processes and / or background threads.

3. The method according to claim 1, characterized in that, The step of binding the main thread to the first logical core and binding the child process and background thread to the at least one second logical core includes: Identify the functions in each file corresponding to the main thread, the child process, and the background thread. Each function in each file is a function used to create the corresponding thread or child process. The function is programmed to bind each thread or subprocess to its corresponding logical core.

4. The method according to claim 1, characterized in that, The determination of valid logical cores based on the server's CPU architecture information includes: A CPU core bitmap is created based on the CPU architecture information of the server, wherein each bit in the CPU core bitmap is used to represent a logical core on the server; Set each bit in the CPU core bitmap to its default value; Based on the logical core started in the server, the corresponding bit in the CPU core bitmap is modified to a target value, which is used to identify the logical core as a valid logical core.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The CPU utilization of the logical core corresponding to each thread or subprocess can be obtained through the preset API interface; If the CPU utilization of the logic core is greater than a first threshold, a target logic core with a CPU utilization less than a second threshold is obtained, wherein the first threshold is greater than the second threshold. The threads or subprocesses corresponding to the logical cores whose CPU utilization is greater than the first threshold are adjusted to the target logical core.

6. The method according to claim 5, characterized in that, Before acquiring the target logic core with a CPU utilization rate less than the second threshold, the process also includes: Determine the CPU utilization rate of each thread or subprocess in the logical core where the CPU utilization rate is greater than the first threshold; If the ratio is greater than or equal to the third threshold, the logical cores of each thread or subprocess corresponding to the Redis remain unchanged. The ratio is the ratio of the CPU utilization rate of each thread or subprocess corresponding to the Redis to the CPU utilization rate of the logical cores whose CPU utilization rate is greater than the first threshold. If the ratio is less than the third threshold, continue to execute the step of obtaining the target logic core with a CPU utilization rate less than the second threshold.

7. A Redis optimization device, characterized in that, The device includes: The architecture information acquisition module is used to obtain the server's CPU architecture information when Redis starts up; The first determining module is used to determine the valid logical cores based on the CPU architecture information of the server, wherein the valid logical cores include a first logical core and at least one second logical core; The second determination module is used to determine the main thread, child processes, and background threads of Redis; A binding module is used to bind the main thread to the first logical core, and to bind the child process and background thread to at least one second logical core.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images