Application process monitoring method and device, medium and computer program product

Abstract

The invention discloses an application process monitoring method and device, a medium and a computer program product, and relates to the technical field of computers. Determining a weight score of the application process according to the business logic weight of the application process and the priority weight of the application process, determining a memory depletion score of the application process according to the process memory occupation parameter of the application process and the weight score of the application process so as to determine a target process, and executing a memory recovery operation on the target process, the process selection mode of the memory depletion terminator mechanism is optimized to meet the business logic requirement, so that the situation that the key process of the business application is shut down when the system memory is recovered is reduced, the normal operation of the system is guaranteed, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and in particular, to an application process monitoring method, device, medium, and computer program product. Background Art

[0002] The memory recycling mechanism is used for management when the system memory is insufficient to prevent the system from crashing. When the system memory is insufficient, the device selectively shuts down some of the multiple application processes running on the operating system to release memory so that the system can continue to run. To avoid important processes from being shut down, in related technologies, priorities are used to determine the importance of application processes. For example, the user can customize the priority score for each application process, and based on the priority score, determine which processes to select for memory recycling when the system memory is insufficient. However, this mechanism of only determining which processes to recycle based on the priority score easily causes key processes in business applications to be mistakenly shut down, seriously affecting the normal operation of the system.

[0003] How to reduce the situation where key processes in business applications are shut down during system memory recycling is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The present invention provides an application process monitoring method, device, medium, and computer program product to at least solve the problem in related technologies that the memory recycling mechanism easily causes key processes to be shut down.

[0005] The present invention provides an application process monitoring method, including: Monitoring the memory occupancy status of the target device; When the memory occupancy status of the target device meets the memory exhaustion terminator trigger condition, obtaining the business logic weight of the application processes running on the target device; Determining the weight score of the application process according to the business logic weight and the priority weight of the application process; Determining the memory exhaustion score of the application process according to the process memory occupancy parameter and the weight score of the application process; Determining a target process according to the memory exhaustion score of the application process, and performing a memory recycling operation on the target process.

[0006] The present invention also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above application process monitoring methods when executing the computer program.

[0007] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the above application process monitoring methods.

[0008] The present invention also provides a computer program product including a computer program, which, when executed by a processor, implements the steps of any one of the above application process monitoring methods.

[0009] Through the present invention, when it is detected that the memory occupancy status of the target device satisfies the memory exhaustion terminator trigger condition, the business logic weight of the application process running on the target device is obtained, the weight score of the application process is determined according to the business logic weight and the priority weight of the application process, and the memory exhaustion score of the application process is determined according to the process memory occupancy parameter and the weight score of the application process. Then, the target process is determined according to the memory exhaustion score of the application process, and the memory recycling operation of the target process is executed. Compared with the traditional memory exhaustion terminator mechanism that only considers the priority weight and ignores the business logic, resulting in the shutdown of critical processes, the present invention optimizes the process selection method of the memory exhaustion terminator mechanism by introducing the business logic weight into the calculation of the memory exhaustion score of the application process, making it meet the business logic requirements, thereby reducing the situation where critical processes of business applications are shut down during system memory recycling, ensuring the normal operation of the system, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] To more clearly illustrate the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 It is a flowchart of an application process monitoring method provided by an embodiment of the present invention; Figure 2 It is a flowchart of another application process monitoring method provided by an embodiment of the present invention; Figure 3 It is a flowchart of the training steps of a prediction model provided by an embodiment of the present invention; Figure 4 It is a flowchart of a memory pressure prediction step provided by an embodiment of the present invention; Figure 5 It is a flowchart of a memory pressure active intervention step provided by an embodiment of the present invention; Figure 6 It is a flowchart of the construction steps of a fault context provided by an embodiment of the present invention; Figure 7A flowchart of an event notification step provided by an embodiment of the present invention. Detailed implementation manners

[0012] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0013] It should be noted that in the description of the present invention, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] In order to enable those skilled in the art in the technical field to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0015] Some key terms used in the embodiments of the present invention will be explained here first.

[0016] The operating systems involved in the present invention include the Linux system. The memory management mechanism of the Linux system mainly includes:

[0017] The buddy system is one of the core components for the Linux kernel to manage physical page frames. It divides continuous physical memory into blocks of different sizes to meet memory allocation requests of different sizes. Its basic principle is based on a binary tree structure, and the memory space is divided level by level. When reclaiming memory, the system checks adjacent free blocks (buddy blocks). If they are of the same size and belong to the same parent block, they will be merged into a larger block and merged upward level by level.

[0018] The Slab allocator is mainly used to optimize the allocation efficiency of small chunks of memory. For the frequent allocation and release of small chunks of memory, the traditional memory allocation method will generate a large amount of memory fragmentation. The Slab allocator pre-creates a series of caches (Slab caches) containing multiple objects of the same size. When there is a request for allocating a small chunk of memory, it directly obtains an object from the corresponding Slab cache instead of allocating from the underlying buddy system. When there is a request for allocating an object, it finds an empty slot in the cache and returns it to the requester. When releasing, the object is put back into the original cache, reducing the generation of memory fragmentation and improving the allocation speed of small chunks of memory.

[0019] The page reclaim mechanism reclaims free pages through the Least Recently Used (LRU) algorithm. The system maintains an LRU list, which sorts memory pages in the order of access time. When memory is under pressure, it starts scanning from the tail of the LRU list and reclaims those pages that have not been accessed for a long time. The core idea is that the least recently used pages are considered to have a lower probability of being accessed again in the future. In the specific implementation, the page access flag bits (such as the Accessed Bit and the Dirty Bit) are used to determine whether a page can be reclaimed. If the page is clean (not modified), it can be directly reclaimed; if it is a dirty page, its data needs to be written back to disk first and then reclaimed.

[0020] The Out Of Memory Killer (OOM Killer) mechanism is triggered when the system detects a global memory shortage and cannot be alleviated by the memory reclaim mechanism. It records the out-of-memory score (oom_score) of each process, which is used to evaluate the "score" for each process to be terminated. The higher the score, the greater the likelihood of the process being terminated.

[0021] With the widespread popularity of cloud computing and containerization technologies, the memory pressure faced by the Linux system is increasing day by day. In high-load scenarios, the competition for memory resources is fierce, which easily causes the system to frequently trigger the OOM Killer mechanism. The current OOM Killer mechanism has many defects. For example, the process selection mechanism is rough, relying only on the out-of-memory score to determine which process to terminate, which easily leads to key processes being mis-killed and seriously affects the system operation. In addition, the current OOM Killer mechanism has a lagging response. It is triggered only after the memory is exhausted, which has caused service avalanches and had a great impact on the system stability and availability. Moreover, the current OOM Killer mechanism lacks effective communication and coordination with user-space applications, cannot actively avoid the risk of memory shortage, and is difficult to meet the requirements of complex and changing business scenarios.

[0022] To at least solve the problem that the current memory exhaustion terminator mechanism is prone to killing critical processes by mistake, the application process monitoring solution provided by the embodiments of the present invention, when it is detected that the memory occupancy status of the target device meets the memory exhaustion terminator trigger condition, obtains the business logic weight of the application processes running on the target device, determines the weight score of the application process according to the business logic weight and the priority weight of the application process, and determines the memory exhaustion score of the application process according to the process memory occupancy parameter and the weight score of the application process, so as to determine the target process according to the memory exhaustion score of the application process and perform the memory recovery operation on the target process. Compared with the traditional memory exhaustion terminator mechanism that only considers the priority weight and ignores the business logic, resulting in critical processes being shut down, the present invention optimizes the process selection method of the memory exhaustion terminator mechanism by introducing the business logic weight into the calculation of the memory exhaustion score of the application process, making it meet the business logic requirements, thereby reducing the situation where critical processes of business applications are shut down during system memory recovery, ensuring the normal operation of the system and improving the user experience.

[0023] Embodiments of the present invention provide an application process monitoring method. Combining with the execution process of the application process monitoring method, the method will be described in detail below.

[0024] Figure 1 It is a flowchart of an application process monitoring method provided by an embodiment of the present invention.

[0025] As Figure 1 shown, the application process monitoring method provided by the embodiments of the present invention may include:

[0026] S101: Monitor the memory occupancy status of the target device.

[0027] S102: When the memory occupancy status of the target device meets the memory exhaustion terminator trigger condition, obtain the business logic weight of the application processes running on the target device.

[0028] S103: Determine the weight score of the application process according to the business logic weight and the priority weight of the application process.

[0029] S104: Determine the memory exhaustion score of the application process according to the process memory occupancy parameter and the weight score of the application process.

[0030] S105: Determine the target process according to the memory exhaustion score of the application process and perform the memory recovery operation on the target process.

[0031] In specific implementation, the target device in the embodiments of the present invention may be a computing device running the Linux system, or a cluster composed of multiple computing devices running the Linux system.

[0032] The target device runs a Linux system, and there are application processes of multiple business applications running in the Linux system, which can be oriented to one or more users (user terminals).

[0033] The application process monitoring method provided by the embodiments of the present invention can be executed by deploying a script on the target device.

[0034] In a specific implementation, for S101 and S102, the trigger condition of the memory exhaustion terminator can be that the memory occupancy ratio of the target operating system of the target device reaches the corresponding threshold, or the free memory size of the target operating system reaches the corresponding threshold.

[0035] In the embodiments of the present invention, all application processes of the target device can be used as candidate processes for the memory exhaustion terminator mechanism, or only some application processes can be used as candidate processes for the memory exhaustion terminator mechanism.

[0036] For S102, the embodiments of the present invention calculate the memory exhaustion score of the application process by using the business logic weight and the priority weight of the application process, so it is necessary to obtain the business logic weight of the application process running on the target device.

[0037] The business logic of a process refers to the specific tasks, operations, and functions executed by the process during operation, as well as the relationships and sequences among these tasks, operations, and functions. It is the core part of the process and determines the behavior and purpose of the process. When calculating the memory exhaustion score, the memory exhaustion terminator mechanism in the related art only considers the priority weight of the application process, and most of these priority weights are customized by users. Different users only have the permission to customize their own application processes, and the business logic of the application process is ignored during the definition process, resulting in key processes being easily mis-killed by the memory exhaustion terminator mechanism.

[0038] In the embodiments of the present invention, the priority weight of the application process is introduced into the calculation of the memory exhaustion score to uniformly consider the business logic of different application processes from the perspective of the entire target device to calculate the memory exhaustion score, reducing the probability that key processes are easily mis-killed by the memory exhaustion terminator mechanism.

[0039] In the embodiments of the present invention, the business logic weight of the application process includes at least one of the process type weight of the application process, the historical memory mode weight of the application process, and the service level agreement (SLA) weight of the application process.

[0040] Specifically, for the process type weight, the process type weight corresponding to setting the application process as a real-time process can be set to be less than the process type weights of other process types. Because real-time processes have high requirements for system real-time performance and cannot be easily terminated.

[0041] For the historical memory stability score, it can be set that the higher the historical memory stability score of the application process, the lower the historical memory mode weight of the application process. For the historical memory mode, by analyzing the memory usage of the process over a period of time in the past, its memory usage stability is judged. If a process has been relatively stable in memory usage and meets its business requirements, then its weight can be appropriately reduced when calculating the memory exhaustion score.

[0042] For the service level agreement weight, it can be set that the service level agreement weight is set according to the service quality requirements of different business systems corresponding to the application process. For example, for an online trading system, the service level agreement weight of the database query process can be set relatively high because it is crucial for the normal operation of the system and the user experience.

[0043] It can be understood that among the above business logic weights, there are types of business logic weights that vary with time and due to different types of business applications. In the embodiments of the present invention, by pre-configuring a unified business logic weight calculation formula for the target device, and regularly collecting the status information of each application process to update the business logic weights of each application process, and adding the current business logic weight to the calculation of the memory exhaustion score when triggering the memory exhaustion terminator mechanism.

[0044] For S103, the embodiments of the present invention calculate the memory exhaustion score of the application process by using the business logic weight and the priority weight of the application process. First, the weight score of the application process is determined according to the business logic weight and the priority weight of the application process. In the embodiments of the present invention, the priority weight of the application process may include the system process priority weight of the application process and the user-defined weight of the application process. The system process priority weight of the application process is the priority weight configured by the target device when creating the application process. The user-defined weight is the priority weight customized by the user of the application process for the application process.

[0045] Among them, according to the design of the Linux system, it usually sets different system process priority weights for kernel-mode processes and user-mode processes when creating application processes, and this system process priority weight can usually also be changed by the user. The user-defined weight is the priority weight set by the user according to the importance of the application process.

[0046] Specifically, in S103, determining the weight score of the application process according to the business logic weight and the priority weight of the application process may include: using the sum of the business logic weight and the priority weight of the application process as the weight score of the application process.

[0047] Furthermore, the process memory occupancy parameter of the application process can be the process memory occupancy ratio of the application process. Determining the memory exhaustion score of the application process according to the process memory occupancy parameter and the weight score of the application process in S104 can include: taking the weight score of the application process as the power, calculating the value of multiplying the power of 2 by the process memory occupancy ratio and dividing by 100, to obtain the memory exhaustion score of the application process.

[0048] Then, the memory exhaustion score provided by the embodiment of the present invention can be calculated by the following formula: ; Wherein, oom_score represents the memory exhaustion score, Memory Occupancy Ratio represents the process memory occupancy ratio, nice value represents the system process priority weight, User-defined Weight represents the user-defined weight, Process Type Weight represents the process type weight, History Memory Pattern Weight represents the historical memory pattern weight, and SLA Weight represents the service level agreement weight.

[0049] In the embodiment of the present invention, it can be set that the system process priority weight, the user-defined weight, the process type weight, the historical memory pattern weight, and the service level agreement weight all correspond to the same value range. For example, all the weight ranges can be -20 to 19, or one or more of the weights can be adjusted according to actual needs to have a larger or smaller value range.

[0050] For the process type weight, in addition to setting the process type weight corresponding to the application process when it is a real-time process to be less than the process type weights of other process types, the processes involved in the user production environment can be classified. The process type weights of key business type processes and real-time transaction type processes can be set lower to reduce the probability of these application processes being terminated. For processes that do not affect key services, such as some monitoring processes, non-essential application type processes, or temporary test type processes, lower application type processes can be set to increase the probability of these application processes being terminated. Thus, when the system memory is insufficient, it can be ensured that important key processes are not affected first.

[0051] For the historical memory stability score, after calculating the historical memory stability score of the application process according to the historical memory usage of the application process, and then calculating the historical memory pattern weight according to the historical memory stability score, the higher the historical memory stability score of the application process, the lower the historical memory pattern weight of the application process, so as to normalize the historical memory stability score to the weight range of the historical memory stability score.

[0052] The service level agreement weight is used to measure the impact of application processes on the stability of the device system. For processes with less impact on system resources, a lower-priority weight value is set to reduce the probability of these application processes being terminated. For processes that affect the system service quality, a higher-priority weight value is set to increase the probability of these application processes being terminated. When the memory is insufficient, it ensures the normal operation experience of the system. Specifically, one or more service level agreement (SLA) parameter indicators can be set, including the resource consumption rate of the application process, the proportion of running time, data throughput, the load balance degree of the distribution of each CPU core occupied, data processing rate, the number of abnormal exits, the number of retries / recoveries, etc.

[0053] The parameter values calculated based on multiple service level agreement parameter indicators can be normalized to the weight range of the service level agreement weight. When calculating the service level agreement weight, it can be set that the smaller the resource consumption rate of the application process, the less the proportion of running time, the larger the data throughput, the more balanced the load balance degree of the distribution of each CPU core occupied, the faster the data processing rate, the fewer the number of abnormal exits, and the fewer the number of retries / recoveries, then the smaller the weight; the less the overall resource consumption, the higher the efficiency, and the fewer the number of exceptions, then the smaller the service level agreement weight. That is to say, if the overall resource consumption of the application process is less, the efficiency is higher, and the number of exceptions is fewer, then a smaller service level agreement weight is set.

[0054] In the embodiment of the present invention, for the process type weight, the historical memory mode weight, and the service level agreement weight, a service weight library can be preset. During the operation of the system, when the memory exhaustion terminator trigger condition is met, the real-time state parameters of each application process are obtained, and the corresponding weight values are queried according to the service weight library to calculate the memory exhaustion score. Further, during the operation state of the system, the weights in the service weight library can be updated according to the memory usage of each application process, so that the execution of the memory exhaustion terminator avoids terminating the application processes of key services.

[0055] For S105, it can be set that the larger the memory exhaustion score, the greater the probability of the application process being terminated. That is to say, the system will select one or more application processes with the top memory exhaustion scores for termination to release memory.

[0056] In the embodiment of the present invention, some target processes can be selected to be shut down, and some other target processes can be selected for partial memory recycling.

[0057] The application process monitoring method provided by the embodiments of the present invention, when it is detected that the memory occupancy status of the target device meets the trigger condition of the memory exhaustion terminator, obtains the business logic weight of the application processes running on the target device, determines the weight score of the application process according to the business logic weight and the priority weight of the application process, and determines the memory exhaustion score of the application process according to the process memory occupancy parameter and the weight score of the application process. Then, according to the memory exhaustion score of the application process, the target process is determined, and the memory recovery operation on the target process is executed. Compared with the traditional memory exhaustion terminator mechanism that only considers the priority weight and ignores the business logic, resulting in the shutdown of critical processes, the present invention optimizes the process selection method of the memory exhaustion terminator mechanism by introducing the business logic weight into the calculation of the memory exhaustion score of the application process, making it meet the business logic requirements, thereby reducing the situation where critical processes of business applications are shut down during system memory recovery, ensuring the normal operation of the system, and improving the user experience.

[0058] Figure 2 It is a flowchart of another application process monitoring method provided by the embodiments of the present invention.

[0059] The above embodiments introduce the introduction of the business logic weight into the calculation of the memory exhaustion score, that is, the optimized process selection algorithm part as Figure 2 shown.

[0060] As Figure 2 shown, in a high-load environment, short-term burst loads are common. If a large number of users access the server simultaneously, a large number of requests are generated instantaneously, resulting in a sharp increase in the memory requirements of the server.

[0061] In the Linux system, especially in a containerized environment, multiple containers share the host memory. When there is a memory leak or abnormal memory occupancy behavior inside a certain container, it will cause a local memory exhaustion event. Since the isolation mechanism between containers is not completely equivalent to the process isolation of a physical machine, this local memory exhaustion event will miskill the processes of non-faulty containers. That is, when the memory exhaustion terminator mechanism selects the process to be terminated, it will miskill the processes in other normally running containers.

[0062] In order to trigger the memory exhaustion terminator mechanism more reasonably, in the embodiments of the present invention, the memory occupancy status of the target device in S101 satisfies the memory exhaustion terminator trigger condition, which may include: monitoring in units of control groups in the target operating system kernel of the target device, and if there is a control group whose memory occupancy status satisfies the corresponding threshold of the control group, it is determined that the memory occupancy status of the target device satisfies the memory exhaustion terminator trigger condition. In S105, determining the target process according to the memory exhaustion score of the application process and performing the memory recovery operation on the target process may include: for the control group that satisfies the corresponding threshold, determining the target process according to the memory exhaustion score of the application process in the control group and performing the memory recovery operation on the target process.

[0063] As Figure 2 shown, the embodiments of the present invention provide a memory exhaustion terminator trigger method that sets local thresholds in units of control groups (cgroups). Local thresholds are set at the control group level, and processes within the same control group are preferentially terminated. For a containerized environment, each container belongs to a specific control group. Independent thresholds can be set for each control group. When the memory usage of a certain control group reaches its set threshold, processes are preferentially selected for termination within that control group, rather than directly within the entire system. For example, in a Kubernetes cluster containing multiple containers, different memory exhaustion terminator thresholds are set for each control group where the containers are located (such as set to 50%, 60%, etc. according to the importance of the container and resource allocation). When the memory usage of a certain container reaches its corresponding threshold, some non-critical processes are first attempted to be terminated within the container to release memory and avoid affecting the normal operation of other containers.

[0064] In some optional embodiments of the embodiments of the present invention, the memory occupancy status of the control group satisfying the corresponding threshold of the control group may include: the memory occupancy status parameter of the control group reaches the first threshold corresponding to the control group.

[0065] In some other optional embodiments of the embodiments of the present invention, the memory occupancy status of the control group satisfying the corresponding threshold of the control group may further include: multiple load status parameters of the control group reach the corresponding thresholds; one of the load status parameters is the memory occupancy status parameter.

[0066] In order to trigger the memory exhaustion terminator mechanism more reasonably, the embodiments of the present invention may adopt a dynamic trigger threshold scheme based on system load. In addition to the memory occupancy status parameter, other load status parameters can be calculated in the form of combined values for judgment.

[0067] In the embodiments of the present invention, for multiple load status parameters of the control group to reach the corresponding thresholds, it may include: the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the control group reach the corresponding second threshold; or, the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the application process with the highest memory occupancy in the control group reach the corresponding second threshold. For example, when the free memory of the system is lower than a certain percentage, and the overall load of the system reaches a certain value, or the load of the process with the highest memory occupancy exceeds a certain value, the memory exhaustion terminator mechanism will be triggered.

[0068] In actual use, it may be necessary to dynamically adapt and adjust the memory exhaustion terminator threshold according to the actual business application usage. For example, it is set to be triggered when the free memory is lower than 4.7% and the load exceeds 83%, or when the free memory is lower than 4.7% and the load of the process with the highest memory occupancy exceeds 50%.

[0069] By real-time monitoring of the system free memory and load conditions, the system can more accurately determine whether it is necessary to start the memory exhaustion terminator mechanism.

[0070] In actual implementation, the free memory and load information can be obtained in real time through relevant functions in the kernel (such as monitor_memory_and_load()), the memory usage of the process can be obtained through the process descriptor task_struct structure, and the judgment can be made according to the set thresholds.

[0071] In some other alternative embodiments of the embodiments of the present invention, multiple load status parameters can also be monitored in units of the entire target device, and the memory exhaustion terminator mechanism is triggered only when multiple load status parameters of the target system reach the corresponding thresholds. One of the load status parameters is the memory occupancy status parameter, and the implementation can refer to the above implementation in units of the control group.

[0072] Figure 3 It is a flowchart of the training steps of a prediction model provided by the embodiments of the present invention; Figure 4 It is a flowchart of a memory pressure prediction step provided by the embodiments of the present invention; Figure 5 It is a flowchart of a memory pressure active intervention step provided by the embodiments of the present invention.

[0073] Regarding the problem of the current memory exhaustion terminator mechanism being laggy, such as Figure 2As shown in the figure, the application process monitoring method provided by the embodiment of the present invention may further include: training a prediction model using the historical memory usage data of the target device; obtaining the memory usage data of the target device at the current moment; inputting the memory usage data and the time information at the current moment into the prediction model to obtain a predicted value of the memory usage parameter within the first future time; and performing a lightweight memory recycling operation on the application process according to the predicted value of the memory usage parameter.

[0074] Among them, performing a lightweight memory recycling operation on the application process according to the predicted value of the memory usage parameter may include: if the predicted value of the memory usage parameter meets the third threshold, performing a lightweight memory recycling operation on the application process; the memory recycling ratio of the lightweight memory recycling operation is lower than the ratio of the memory recycling operation performed when the memory exhaustion terminator trigger condition is met.

[0075] That is to say, a prediction model is trained based on historical memory usage data to trigger lightweight recycling or flow limiting in advance. In a file server system, when the memory is exhausted, file read and write operations will become extremely slow or even stagnant because there is not enough memory to cache data. Process scheduling will also be affected, resulting in other normally running processes being unable to obtain central processing unit resources in a timely manner. In this scenario, a time series analysis algorithm (such as the ARIMA model) can be used to analyze and predict historical memory usage data. By collecting the memory usage data of the system over a past period of time (such as one week), including information such as the memory occupancy rate, memory allocation and release frequencies at different time periods, and inputting and training the ARIMA model.

[0076] Such as Figure 3 As shown in the figure, the types of historical memory usage data adopted may include at least one of the memory occupancy rate, memory allocation information, and memory release frequency. In some optional implementation manners of the embodiment of the present invention, three types of historical memory usage data, namely the memory occupancy rate, memory allocation information, and memory release frequency, may be used to train the prediction model.

[0077] Such as Figure 4As shown, after the model training is completed, according to the real-time input memory usage data, the memory usage situation in a future period (such as the next 10 minutes) can be predicted in real time. Obtain the memory usage data at the current moment, which can specifically include the memory occupancy rate, memory allocation information, and memory release frequency at the current moment. Input these memory usage data into the trained prediction model to output the predicted value of the memory usage parameter. Determine whether the predicted value of the memory usage parameter meets the third threshold. If so, trigger a lightweight memory recycling operation, and then continuously monitor the memory usage data, that is, obtain the memory usage data at the current moment. If not, continuously monitor the memory usage data, that is, obtain the memory usage data at the current moment. That is to say, by continuously monitoring the memory usage data, when it is predicted that there may be an upcoming increase in memory pressure (that is, the predicted value of the memory usage parameter meets the third threshold), a lightweight recycling mechanism is triggered in advance, such as recycling some cache pages, or throttling some non-critical services, to avoid affecting the file reading and writing operations and process scheduling of the system when the memory is exhausted, resulting in other normally running processes being unable to obtain the central processing unit resources in time.

[0078] As Figure 2 As shown, the application process monitoring method provided by the embodiment of the present invention may further include: if the predicted value of the memory usage parameter meets the third threshold, send a memory pressure notification to the user-mode application of the application process, so that the user-mode application performs a memory pressure mitigation operation.

[0079] Specifically, sending a memory pressure notification to the user-mode application of the application process so that the user-mode application performs a memory pressure mitigation operation may include: pre-adding a memory pressure notification interface to the virtual memory management parameter configuration directory of the target operating system to enable the user-mode application to register a callback function for responding to memory pressure events; sending a memory pressure notification to the user-mode application interface of the user-mode application through the memory pressure notification interface, so that the user-mode application calls the callback function to perform a memory pressure mitigation operation.

[0080] / proc / sys / vm / under the system is the virtual memory management parameter configuration directory provided by the Linux kernel, which is used to dynamically adjust the memory allocation, recycling policy, and performance optimization. Add a / proc / sys / vm / mem_pressure interface (Application Programming Interface, API) therein, allowing applications to register callback functions to respond to memory pressure events.

[0081] As Figure 5As shown, in the embodiment of the present invention, under the strategy of real-time monitoring of memory by the above system and prediction by the prediction model, it is possible to perform threshold judgment on the memory usage data at the current moment, and at the same time input the memory usage data at the current moment into the prediction model to obtain the predicted value of the memory usage parameter, that is, to determine whether there is an increase in memory pressure by using the memory usage situation at the current moment and the prediction of the future memory usage situation. When there is a lack of memory pressure or a possible increase in the future, the lightweight recycling mechanism can be triggered in advance, and a memory pressure notification is sent to the user-space application interface through this interface. After receiving the notification, the user-space application calls the registered callback function to perform corresponding memory pressure mitigation operations, such as releasing some temporary resources, adjusting business logic to reduce memory usage, etc.

[0082] Figure 6 It is a flowchart of the steps for constructing a fault context provided by an embodiment of the present invention; Figure 7 It is a flowchart of the steps for event notification provided by an embodiment of the present invention.

[0083] After the memory exhaustion terminator mechanism is triggered in the related technology, the terminated application process lacks context records, resulting in the user-space application not knowing the specific situation of the terminated process, which is not conducive to subsequent fault analysis.

[0084] In response to this, as Figure 2 shown, the application process monitoring method provided by the embodiment of the present invention may further include: after starting the memory recycling operation, recording the information of the memory recycling operation in the system log of the target device, and the information of the memory recycling operation includes at least one of the process identifier of the terminated process, the name of the terminated process, and the user privilege of the terminated process; collecting the metadata information of the terminated process from the kernel audit subsystem of the target device to generate the memory mapping data of the terminated process; obtaining the resource time series data of the terminated process within the second time before failure; constructing a fault context according to the information of the memory recycling operation, the memory mapping data, and the resource time series data.

[0085] Specifically, as Figure 6 shown, after the memory exhaustion terminator is triggered and a process is terminated, the system can collect and persistently store more metadata information of the terminated process (such as the process identifier PID of the parent process, process priority, process life cycle, etc.) through the kernel audit subsystem, and synchronously generate a memory snapshot including complete memory mapping details (user-space heap, kernel-space allocator (SLAB), etc.) and key stack / page table structures.

[0086] Meanwhile, it is possible to trace the resource time series data of the terminated process within the second-to-last time period (e.g., 60 seconds) before failure (such as the interruption usage of the process, the CPU utilization rate, the throughput data volume of block device I / O performed by the process, the network bandwidth occupancy rate, etc.) to obtain the load status parameters of the terminated process for recording.

[0087] In addition, based on process tracing (ptrace) or process topology analysis of control groups (cgroups), it is possible to record the connection status between processes, the blood relationship of the process tree, and the shared memory dependency graph, and construct a complete fault context to provide the data required for analyzing memory faults and perform detailed root cause diagnosis.

[0088] The application process monitoring method provided by the embodiments of the present invention may further include: sending a structured event of the terminated process to the user-space application to enable the user-space application to perform proactive operation and maintenance intervention.

[0089] As Figure 7 shown, after the memory exhaustion terminator is triggered and the process is terminated, the structured information of the terminated process can be sent to the user space through the Netlink mechanism to support proactive operation and maintenance intervention.

[0090] Netlink is a mechanism for communication between the user space and the kernel space in the Linux kernel, mainly used for efficiently transmitting structured data and control information. When the memory exhaustion terminator mechanism is triggered or the system memory pressure reaches a certain threshold, the kernel can send the above-mentioned structured information of the terminated process to the user-space monitoring application through Netlink, which can include detailed information such as the memory snapshot of the terminated process, the resource occupancy history, and the associated process information.

[0091] After receiving these events, the user-space monitoring application (user-space operation and maintenance tool) can analyze the event content, perform proactive intervention based on the event content, and execute intervention optimization operations, such as remotely logging in to the system for further troubleshooting and processing, or automatically performing some pre-set recovery operations to reduce the possible further impacts on the system after the memory exhaustion terminator mechanism is triggered.

[0092] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0093] An embodiment of the present invention further provides an application process monitoring device, including: a monitoring unit for monitoring the memory occupancy status of a target device; a first acquisition unit for acquiring the business logic weight of an application process running on the target device when the monitoring unit monitors that the memory occupancy status of the target device meets the memory exhaustion terminator trigger condition; a first determination unit for determining the weight score of the application process according to the business logic weight of the application process and the priority weight of the application process; a second determination unit for determining the memory exhaustion score of the application process according to the process memory occupancy parameter of the application process and the weight score of the application process; a first control unit for determining a target process according to the memory exhaustion score of the application process and performing a memory recovery operation on the target process.

[0094] In an embodiment of the present invention, the business logic weight of an application process may include at least one of the process type weight of the application process, the historical memory mode weight of the application process, and the service level agreement weight of the application process; the process type weight corresponding to the application process being a real-time process is less than the process type weights of other process types; the higher the historical memory stability score of the application process, the lower the historical memory mode weight of the application process; the service level agreement weight is set according to the requirements of the business system corresponding to the application process for different service qualities.

[0095] In an embodiment of the present invention, the priority weight of an application process may include the system process priority weight of the application process and the user-defined weight of the application process; the system process priority weight of the application process is the priority weight configured by the target device when creating the application process; the user-defined weight is the priority weight customized by the user of the application process for the application process.

[0096] In an embodiment of the present invention, the first determination unit determines the weight score of the application process according to the business logic weight of the application process and the priority weight of the application process, which may include: taking the sum of the business logic weight of the application process and the priority weight of the application process as the weight score of the application process.

[0097] In an embodiment of the present invention, the process memory occupancy parameter of the application process may be the process memory occupancy ratio of the application process; the second determination unit determines the memory exhaustion score of the application process according to the process memory occupancy parameter of the application process and the weight score of the application process, which may include: calculating the value of 2 to the power of the weight score of the application process multiplied by the process memory occupancy ratio and divided by 100 to obtain the memory exhaustion score of the application process.

[0098] In an embodiment of the present invention, when the monitoring unit monitors that the memory occupancy status of the target device meets the trigger condition of the memory exhaustion terminator, it may include: monitoring in units of control groups in the target operating system kernel of the target device, and if there is a control group whose memory occupancy status meets the threshold corresponding to the control group, it is determined that the memory occupancy status of the target device meets the trigger condition of the memory exhaustion terminator. The first control unit determines the target process according to the memory exhaustion score of the application process and performs the memory recycling operation on the target process, which may include: for the control group that meets the corresponding threshold, determining the target process according to the memory exhaustion score of the application process in the control group and performing the memory recycling operation on the target process.

[0099] In an embodiment of the present invention, that the memory occupancy status of the control group meets the threshold corresponding to the control group may include: the memory occupancy status parameter of the control group reaches the first threshold corresponding to the control group.

[0100] In an embodiment of the present invention, that the memory occupancy status of the control group meets the threshold corresponding to the control group may include: multiple load status parameters of the control group reach the corresponding thresholds; one of the load status parameters is the memory occupancy status parameter.

[0101] In an embodiment of the present invention, that multiple load status parameters of the control group reach the corresponding thresholds may include: the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the control group reach the corresponding second threshold; or, the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the application process with the highest memory occupancy in the control group reach the corresponding second threshold.

[0102] The application process monitoring device provided by the embodiment of the present invention may further include: a training unit for training a prediction model using the historical memory usage data of the target device; a second acquisition unit for acquiring the memory usage data of the target device at the current moment; a calculation unit for inputting the memory usage data and the time information at the current moment into the prediction model to obtain a predicted value of the memory usage parameter within the first future time; a second control unit for performing a lightweight memory recycling operation on the application process if the predicted value of the memory usage parameter meets the third threshold; the memory recycling ratio of the lightweight memory recycling operation is lower than the ratio of the memory recycling operation performed when the trigger condition of the memory exhaustion terminator is met.

[0103] The application process monitoring device provided by the embodiment of the present invention may further include: a first sending unit for sending a memory pressure notification to the user-mode application of the application process if the predicted value of the memory usage parameter meets the third threshold, so that the user-mode application performs a memory pressure mitigation operation.

[0104] The application process monitoring device provided by an embodiment of the present invention may further include: a recording unit, configured to, after starting a memory recycling operation, record information of the memory recycling operation in the system log of the target device, where the information of the memory recycling operation includes at least one of a process identifier of the terminated process, a name of the terminated process, and a user privilege of the terminated process; collect metadata information of the terminated process from the kernel audit subsystem of the target device, and generate memory mapping data of the terminated process; obtain resource time series data of the terminated process within a second period before failure; and construct a fault context according to the information of the memory recycling operation, the memory mapping data, and the resource time series data.

[0105] For the description of the features in the corresponding embodiment of the application process monitoring device, reference may be made to the relevant description in the corresponding embodiment of the application process monitoring method, which will not be elaborated here.

[0106] An embodiment of the present invention further provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above embodiments of the application process monitoring method.

[0107] An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the steps in any one of the above embodiments of the application process monitoring method when running.

[0108] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media that can store computer programs such as a USB flash drive, a read-only memory (ROM for short), a random access memory (RAM for short), a mobile hard disk, a magnetic disk, or an optical disc.

[0109] An embodiment of the present invention further provides a computer program product, where the computer program product includes a computer program, and the computer program implements the steps in any one of the above embodiments of the application process monitoring method when executed by a processor.

[0110] An embodiment of the present invention further provides another computer program product, including a non-volatile computer-readable storage medium, where the non-volatile computer-readable storage medium stores a computer program, and the computer program implements the steps in any one of the above embodiments of the application process monitoring method when executed by a processor.

[0111] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered as exceeding the scope of the present invention.

[0112] The above has introduced in detail an application process monitoring method, device, medium, and computer program product provided by the present invention. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A method for monitoring an application process, characterized in that: include: Monitor the memory usage status of the target device; When the memory occupancy state of the target device meets the memory exhaustion terminator triggering condition, obtaining the business logic weight of the application process running on the target device; Determining a weight score of the application process according to the business logic weight of the application process and the priority weight of the application process; Determining a memory exhaustion score of the application process according to a process memory occupation parameter of the application process and a weight score of the application process; A target process is determined according to the memory exhaustion score of the application process, and a memory recovery operation is performed on the target process.

2. The application process monitoring method according to claim 1, characterized in that: The business logic weight of the application process includes at least one of a process type weight of the application process, a historical memory mode weight of the application process, and a service level agreement weight of the application process; When the application process is a real-time process, the corresponding process type weight is less than the process type weights of other process types; The higher the historical memory stability score of the application process, the lower the historical memory mode weight of the application process; The service level agreement weight is set according to different service quality requirements of the business system corresponding to the application process.

3. The application process monitoring method according to claim 1, characterized in that: The priority weight of the application process includes the system process priority weight of the application process and the user-defined weight of the application process; The system process priority weight of the application process is the priority weight configured by the target device when creating the application process; The user-defined weight is a priority weight that is customized by a user of the application process for the application process.

4. The application process monitoring method according to claim 1, characterized in that: Determining a weight score of the application process according to the business logic weight of the application process and the priority weight of the application process includes: The sum of the business logic weight of the application process and the priority weight of the application process is taken as the weight score of the application process.

5. The application process monitoring method according to claim 4, characterized in that: The process memory occupancy parameter of the application process is the process memory occupancy ratio of the application process; Determining a memory exhaustion score of the application process according to a process memory occupation parameter of the application process and a weight score of the application process includes: Taking the weight score of the application process as a power, calculate the value of multiplying the power of 2 by the memory occupancy ratio of the process and dividing by 100 to obtain the memory exhaustion score of the application process.

6. The application process monitoring method according to claim 1, characterized in that: The memory occupancy state of the target device satisfies the triggering condition of the memory exhaustion terminator, including: Monitoring is performed in units of control groups in the target operating system kernel of the target device, and if there is a control group whose memory occupancy state satisfies a threshold value corresponding to the control group, determining that the memory occupancy state of the target device satisfies the memory exhaustion terminator triggering condition; Determining a target process according to the memory exhaustion score of the application process, and performing a memory recovery operation on the target process, including: For the control group that meets the corresponding threshold, the target process is determined according to the memory exhaustion score of the application process in the control group, and a memory recovery operation is performed on the target process.

7. The application process monitoring method according to claim 6, characterized in that: The memory occupancy state of the control group satisfies a threshold corresponding to the control group, including: The memory occupancy state parameter of the control group reaches a first threshold corresponding to the control group.

8. The application process monitoring method according to claim 6, characterized in that: The memory occupancy state of the control group satisfies a threshold corresponding to the control group, including: The multiple load status parameters of the control group reach corresponding thresholds; One of the load status parameters is a memory occupancy status parameter.

9. The application process monitoring method according to claim 8, characterized in that: The multiple load status parameters of the control group reach corresponding thresholds, including: The memory occupancy state parameter of the control group reaches a corresponding first threshold and the other load state parameters of the control group reach a corresponding second threshold; Or, the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the application process with the highest memory occupancy in the control group reach the corresponding second threshold.

10. The application process monitoring method according to claim 1, characterized in that: Also includes: Using the historical memory usage data of the target device to train a prediction model; Obtain memory usage data of the target device at the current moment; Input the memory usage data and the current time information into the prediction model to obtain a memory usage parameter prediction value within the first time in the future; If the predicted value of the memory usage parameter meets a third threshold, performing a lightweight memory recovery operation on the application process; The memory reclamation ratio of the lightweight memory reclamation operation is lower than the ratio of the memory reclamation operation performed when the memory exhaustion terminator triggering condition is met.

11. The application process monitoring method according to claim 10, characterized in that: Also includes: If the predicted value of the memory usage parameter meets the third threshold, a memory pressure notification is sent to the user state application of the application process, so that the user state application performs a memory pressure relief operation.

12. The application process monitoring method according to claim 1, characterized in that: Also includes: After starting the memory reclaiming operation, recording information of the memory reclaiming operation in a system log of the target device, the information of the memory reclaiming operation including at least one of a process identifier of a terminated process, a name of the terminated process, and a user permission of the terminated process; Collecting metadata information of the terminated process from the kernel audit subsystem of the target device, and generating memory mapping data of the terminated process; Obtaining resource time series data of the terminated process within a second time before failure; A fault context is constructed according to the information of the memory reclaim operation, the memory mapping data and the resource time series data.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the application process monitoring method as described in any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the application process monitoring method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the application process monitoring method according to any one of claims 1 to 12 are implemented.

Images