Application process monitoring method, device, medium and computer program product

Abstract

The present invention discloses an application process monitoring method, device, medium and computer program product, which relate to the field of computer technology. When monitoring that the memory occupancy status of a target device meets the triggering condition of a memory exhaustion terminator, the weight score of the application process is determined according to the business logic weight of the application process and the priority weight of the application process, and the memory exhaustion score of the application process is determined according to the process memory occupancy parameter of the application process and the weight score of the application process to determine the target process, and a memory recovery operation is performed on the target process. The process selection method of the memory exhaustion terminator mechanism is optimized to make it meet the business logic requirements, thereby reducing the situation where key processes of business applications are shut down when the system memory is recovered, ensuring the normal operation of the system, and improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an application process monitoring method, device, medium and computer program product. Background Art

[0002] The memory recycling mechanism is used to manage when the system is low on memory to prevent the system from crashing. When the system is low on memory, the device selectively shuts down some of the multiple application processes running by the operating system to free up memory so that the system can continue to run. To prevent important processes from being shut down, related technologies use priority to determine the importance of application processes. For example, users can customize the priority score of each application process, and determine which processes to select for memory recycling when the system is low on memory based on the priority score. However, this mechanism, which only determines which processes to select for recycling based on priority scores, can easily lead to critical processes in business applications being mistakenly shut down, seriously affecting the normal operation of the system.

[0003] How to reduce the situation where key processes of business applications are shut down when system memory is recycled is a technical problem that those skilled in the art need to solve. Summary of the Invention

[0004] The present invention provides an application process monitoring method, device, medium and computer program product to at least solve the problem in the related art that the memory recycling mechanism easily causes the key process to be shut down.

[0005] The present invention provides an application process monitoring method, comprising:

[0006] Monitor the memory usage of the target device;

[0007] When the memory occupancy state of the target device meets the memory exhaustion terminator triggering condition, obtaining the business logic weight of the application process running on the target device;

[0008] Determining a weight score of the application process according to the business logic weight of the application process and the priority weight of the application process;

[0009] determining a memory exhaustion score of the application process according to a process memory occupancy parameter of the application process and a weight score of the application process;

[0010] A target process is determined according to the memory exhaustion score of the application process, and a memory reclaiming operation is performed on the target process.

[0011] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned application process monitoring methods when executing the computer program.

[0012] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned application process monitoring methods are implemented.

[0013] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned application process monitoring methods when executed by a processor.

[0014] Through the present invention, when it is monitored that the memory occupancy status of the target device meets the memory exhaustion terminator triggering condition, the business logic weight of the application process running on the target device is obtained, the weight score of the application process is determined according to the business logic weight of the application process and the priority weight of the application process, and the memory exhaustion score of the application process is determined according to the process memory occupancy parameter of the application process and the weight score of the application process, so as to determine the target process according to the memory exhaustion score of the application process, and perform the memory recovery operation on the target process. Compared with the traditional memory exhaustion terminator mechanism that only considers the priority weight but ignores the business logic, resulting in the shutdown of key processes, the present invention optimizes the process selection method of the memory exhaustion terminator mechanism by introducing the business logic weight into the calculation of the memory exhaustion score of the application process, so that it meets the business logic requirements, thereby reducing the situation where key processes of business applications are shut down when the system memory is recovered, ensuring the normal operation of the system, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 A flowchart of an application process monitoring method provided by an embodiment of the present invention;

[0017] Figure 2 A flowchart of another application process monitoring method provided by an embodiment of the present invention;

[0018] Figure 3 A flowchart of the training steps of a prediction model provided by an embodiment of the present invention;

[0019] Figure 4 A flowchart of memory pressure prediction steps provided by an embodiment of the present invention;

[0020] Figure 5 A flowchart of a memory pressure active intervention step provided by an embodiment of the present invention;

[0021] Figure 6 A flowchart of the steps for constructing a fault context provided by an embodiment of the present invention;

[0022] Figure 7 A flowchart of an event notification step provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0024] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0025] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0026] Here, some key terms used in the embodiments of the present invention are explained.

[0027] The operating system involved in the present invention includes the Linux system. The memory management mechanism of the Linux system mainly includes:

[0028] The buddy system is a core component of the Linux kernel's physical page frame management. It divides contiguous physical memory into blocks of varying sizes to accommodate memory allocation requests of varying sizes. Its fundamental principle is based on a binary tree structure, which partitions the memory space hierarchically. When reclaiming memory, the system checks for adjacent free blocks (buddy blocks). If they are of the same size and belong to the same parent block, they are merged into a larger block, and the merging continues upwards.

[0029] The slab allocator is primarily used to optimize the efficiency of small-block memory allocation. Traditional memory allocation methods generate significant memory fragmentation due to the frequent allocation and release of small-block memory. The slab allocator pre-creates a series of caches (slab caches) containing multiple objects of the same size. When a small-block memory allocation request arrives, the object is retrieved directly from the corresponding slab cache, rather than being allocated from the underlying partner system. When an object allocation request arrives, an idle slot is found in the cache and returned to the requester. Upon release, the object is returned to the original cache, reducing memory fragmentation and improving the speed of small-block memory allocation.

[0030] The page recycling mechanism reclaims free pages using the Least Recently Used (LRU) algorithm. The system maintains an LRU linked list that sorts memory pages in order of access time. When memory is tight, it starts scanning from the end of the LRU linked list and reclaims pages that have not been accessed for a long time. The core idea is that the least recently used page is less likely to be accessed again in the future. In the specific implementation, the page access flag bits (such as the Accessed Bit and the Dirty Bit) are used to determine whether the page can be recycled. If the page is clean (unmodified), it can be recycled directly; if it is a dirty page, its data needs to be written back to disk before it can be recycled.

[0031] The Out Of Memory Killer (OOM Killer) mechanism is triggered when the system detects insufficient global memory and cannot be alleviated by the memory reclamation mechanism. The OOM Killer mechanism records the memory exhaustion score (oom_score) of each process. The memory exhaustion score is used to evaluate the "score" of each process for termination. The higher the score, the greater the possibility that the process will be terminated.

[0032] With the widespread adoption of cloud computing and containerization technologies, Linux systems are facing increasing memory pressure. In high-load scenarios, competition for memory resources is fierce, which can easily cause the system to frequently trigger the out-of-memory killer mechanism. The current out-of-memory killer mechanism has many flaws. For example, its process selection mechanism is crude, relying solely on the out-of-memory score to determine process termination, which can easily lead to the accidental killing of critical processes, seriously impacting system operations. Furthermore, the current out-of-memory killer mechanism has a delayed response, triggering only after memory exhaustion, causing service avalanches and significantly impacting system stability and availability. Furthermore, the current out-of-memory killer mechanism lacks effective communication and coordination with user-mode applications, making it unable to proactively avoid the risk of insufficient memory and unable to meet the needs of complex and ever-changing business scenarios.

[0033] In order to at least solve the problem that the current memory exhaustion terminator mechanism is prone to accidentally killing critical processes, an embodiment of the present invention provides an application process monitoring solution. When it is monitored that the memory occupancy status of the target device meets the memory exhaustion terminator triggering condition, the business logic weight of the application process running on the target device is obtained, the weight score of the application process is determined according to the business logic weight of the application process and the priority weight of the application process, and the memory exhaustion score of the application process is determined according to the process memory occupancy parameter of the application process and the weight score of the application process. The target process is determined according to the memory exhaustion score of the application process, and a memory recovery operation is performed on the target process. Compared with the traditional memory exhaustion terminator mechanism that only considers the priority weight but ignores the business logic, resulting in the shutdown of critical processes, the present invention optimizes the process selection method of the memory exhaustion terminator mechanism by introducing the business logic weight into the calculation of the memory exhaustion score of the application process, so that it meets the business logic requirements, thereby reducing the situation where critical processes of business applications are shut down when the system memory is recovered, ensuring the normal operation of the system, and improving the user experience.

[0034] An embodiment of the present invention provides an application process monitoring method. The method is described in detail below in conjunction with the execution flow of the application process monitoring method.

[0035] Figure 1 The present invention provides a flowchart of an application process monitoring method.

[0036] like Figure 1 As shown, the application process monitoring method provided by the embodiment of the present invention may include:

[0037] S101: Monitor the memory usage status of the target device.

[0038] S102: When the memory usage status of the target device meets the memory exhaustion terminator triggering condition, the business logic weight of the application process running on the target device is obtained.

[0039] S103: Determine a weight score of the application process according to the business logic weight of the application process and the priority weight of the application process.

[0040] S104: Determine a memory exhaustion score of the application process according to the process memory usage parameter of the application process and the weight score of the application process.

[0041] S105: Determine a target process according to the memory exhaustion score of the application process, and perform a memory reclaiming operation on the target process.

[0042] In a specific implementation, the target device in the embodiment of the present invention may be a computing device running a Linux system, or may be a cluster consisting of multiple computing devices running a Linux system.

[0043] The target device runs the Linux system, which runs multiple business application processes and can be used for one or more users (user terminals).

[0044] The application process monitoring method provided by the embodiment of the present invention can be executed by deploying a script on a target device.

[0045] In a specific implementation, for S101 and S102 , the memory exhaustion terminator triggering condition may be that the memory usage ratio of the target operating system of the target device reaches a corresponding threshold, or the free memory size of the target operating system reaches a corresponding threshold.

[0046] In the embodiment of the present invention, all application processes of the target device may be used as candidate processes for the memory exhaustion terminator mechanism, or only some application processes may be used as candidate processes for the memory exhaustion terminator mechanism.

[0047] For S102 , the embodiment of the present invention uses the business logic weight of the application process and the priority weight of the application process to calculate the memory exhaustion score of the application process, so it is necessary to obtain the business logic weight of the application process running on the target device.

[0048] The business logic of a process refers to the specific tasks, operations, and functions performed by the process during its operation, as well as the relationships and sequence between these tasks, operations, and functions. It is the core of the process and determines its behavior and purpose. The memory exhaustion killer mechanism in related technologies only considers the priority weight of the application process when calculating the memory exhaustion score. These priority weights are mostly user-defined, and different users only have the right to customize their own application processes. The business logic of the application process is ignored during the definition process, making it easy for critical processes to be mistakenly killed by the memory exhaustion killer mechanism.

[0049] In an embodiment of the present invention, the priority weight of the application process is introduced into the calculation of the memory exhaustion score, so that the business logic of different application processes is uniformly considered from the perspective of the entire target device to calculate the memory exhaustion score, thereby reducing the probability of critical processes being mistakenly killed by the memory exhaustion terminator mechanism.

[0050] In an embodiment of the present invention, the business logic weight of the application process includes at least one of a process type weight of the application process, a historical memory mode weight of the application process, and a service level agreement (SLA) weight of the application process.

[0051] Specifically, for process type weights, the process type weight corresponding to the real-time process can be set to be smaller than the process type weights of other process types. This is because real-time processes have high requirements for system real-time performance and cannot be easily terminated.

[0052] For the historical memory stability score, you can set a higher historical memory stability score for an application process, giving it a lower weight for its historical memory pattern. For historical memory patterns, we analyze the process's memory usage over time to determine its memory usage stability. If a process's memory usage has been relatively stable and meets business requirements, its weight can be appropriately reduced when calculating the memory exhaustion score.

[0053] SLA weights can be set based on the service quality requirements of the business system corresponding to the application process. For example, in an online trading system, the SLA weight of the database query process can be set higher because it is critical to the normal operation of the system and the user experience.

[0054] It is understandable that some of the aforementioned business logic weights may vary over time and depending on the type of business application. In an embodiment of the present invention, a unified business logic weight calculation formula is preconfigured for the target device, and the status information of each application process is periodically collected to update the business logic weight of each application process. When the memory exhaustion terminator mechanism is triggered, the current business logic weight is incorporated into the calculation of the memory exhaustion score.

[0055] For S103, an embodiment of the present invention uses the business logic weight of the application process and the priority weight of the application process to calculate the memory exhaustion score of the application process. First, the weight score of the application process is determined based on the business logic weight of the application process and the priority weight of the application process. In an embodiment of the present invention, the priority weight of the application process may include the system process priority weight of the application process and the user-defined weight of the application process. The system process priority weight of the application process is the priority weight configured by the target device when creating the application process. The user-defined weight is the priority weight customized by the user of the application process for the application process.

[0056] The Linux system is designed to set different system process priority weights for kernel-mode and user-mode processes when creating an application process. These system process priority weights can also be modified by the user. User-defined weights are priority weights set by the user based on the importance of the application process.

[0057] Specifically, determining the weight score of the application process according to the business logic weight and the priority weight of the application process in S103 may include taking the sum of the business logic weight and the priority weight of the application process as the weight score of the application process.

[0058] Specifically, the process memory usage parameter of the application process may be a process memory usage ratio of the application process. Determining the memory exhaustion score of the application process based on the process memory usage parameter of the application process and the weight score of the application process in S104 may include: using the weight score of the application process as a power, calculating a value of 2 multiplied by the process memory usage ratio, and dividing the result by 100 to obtain the memory exhaustion score of the application process.

[0059] The memory exhaustion score provided by the embodiment of the present invention can be calculated using the following formula:

[0060] ;

[0061] Among them, oom_score indicates the memory exhaustion score, Memory Occupancy Ratio indicates the process memory occupancy ratio, nice value indicates the system process priority weight, User-defined Weight indicates the user-defined weight, Process Type Weight indicates the process type weight, History Memory Pattern Weight indicates the historical memory pattern weight, and SLA Weight indicates the service level agreement weight.

[0062] In an embodiment of the present invention, the system process priority weight, user-defined weight, process type weight, historical memory mode weight and service level agreement weight can be set to correspond to the same value range. For example, all weight ranges can be -20~19, or one or more weights can be adjusted to have a larger or smaller value range according to actual needs.

[0063] Regarding process type weights, in addition to setting the corresponding process type weight for real-time application processes lower than that for other process types, processes involved in the user's production environment can be categorized. Key business processes and real-time transaction processes can be assigned lower process type weights to reduce the probability of these application processes being terminated. For processes that do not impact critical business operations, such as some monitoring processes, non-essential application processes, or temporary testing processes, even lower application type weights can be assigned to increase their probability of termination. This allows for priority protection of critical processes when system memory is insufficient.

[0064] For the historical memory stability score, the historical memory stability score of the application process is calculated based on the historical memory usage of the application process, and then the historical memory mode weight is calculated based on the historical memory stability score. The higher the historical memory stability score of the application process, the lower the historical memory mode weight of the application process, so that the historical memory stability score is normalized to within the weight range of the historical memory stability score.

[0065] Service Level Agreement (SLA) weights measure the impact of application processes on device system stability. Processes with minimal impact on system resources are assigned lower priority weights to reduce the probability of their termination. Processes that impact system service quality are assigned higher priority weights to increase their probability of termination, ensuring the system's normal operation when memory is insufficient. Specifically, one or more SLA parameters can be set, including application process resource consumption rate, run time percentage, data throughput, load balance across CPU cores, data processing rate, number of abnormal exits, and number of retries / recoveries.

[0066] The parameter values calculated based on multiple SLA parameter indicators can be normalized to the SLA weight range. When calculating the SLA weight, the smaller the application process resource consumption rate, the smaller the proportion of running time, the greater the data throughput, the more balanced the load distribution of CPU cores, the faster the data processing rate, the fewer abnormal exits, the fewer retries / recoveries, etc., the smaller the weight; the less overall resource consumption, the higher the efficiency, and the fewer abnormalities occur, the smaller the SLA weight. In other words, if the application process consumes less resources overall, has higher efficiency, and fewer abnormalities occur, a smaller SLA weight can be set.

[0067] In an embodiment of the present invention, a business weight library can be pre-set for process type weights, historical memory mode weights, and service level agreement weights. This allows, during system operation, when the out-of-memory terminator trigger conditions are met, to obtain the real-time status parameters of each application process and, based on a query from the business weight library, obtain the corresponding weight value to calculate the out-of-memory score. Furthermore, during system operation, the weights in the business weight library can be updated based on the memory usage of each application process, so that the out-of-memory terminator can be executed without terminating critical business application processes.

[0068] For S105 , it can be set that the greater the memory exhaustion score, the greater the probability of the application process being terminated. In other words, the system will select one or more application processes with high memory exhaustion scores to terminate in order to free up memory.

[0069] In the embodiment of the present invention, some target processes may be selected to be shut down, and other target processes may be selected to have some memory reclaimed.

[0070] The application process monitoring method provided by the embodiment of the present invention obtains the business logic weight of the application process running on the target device when it is monitored that the memory occupancy status of the target device meets the memory exhaustion terminator triggering condition, determines the weight score of the application process according to the business logic weight of the application process and the priority weight of the application process, and determines the memory exhaustion score of the application process according to the process memory occupancy parameter of the application process and the weight score of the application process, thereby determining the target process according to the memory exhaustion score of the application process and performing a memory recovery operation on the target process. Compared with the traditional memory exhaustion terminator mechanism that only considers the priority weight but ignores the business logic, resulting in the shutdown of key processes, the present invention optimizes the process selection method of the memory exhaustion terminator mechanism by introducing the business logic weight into the calculation of the memory exhaustion score of the application process, so that it meets the business logic requirements, thereby reducing the situation where key processes of business applications are shut down when the system memory is recovered, ensuring the normal operation of the system, and improving the user experience.

[0071] Figure 2 A flowchart of another application process monitoring method provided by an embodiment of the present invention.

[0072] The above embodiment introduces the business logic weight into the calculation of the memory exhaustion score, that is, Figure 2 The optimization process selection algorithm section is shown.

[0073] like Figure 2 As shown in Figure 1, short bursts of load are common in high-load environments. If a large number of users access the server simultaneously, a large number of requests are generated instantly, causing a sharp increase in server memory requirements.

[0074] In Linux systems, especially in containerized environments, multiple containers share host memory. A memory leak or abnormal memory usage within a container can lead to localized memory exhaustion. Because the isolation between containers is not completely equivalent to process isolation on physical machines, this localized memory exhaustion event can inadvertently kill non-faulty container processes. Specifically, the out-of-memory killer mechanism can accidentally kill processes in other, normally running containers when selecting a process to terminate.

[0075] To more reasonably trigger the memory exhaustion terminator mechanism, in an embodiment of the present invention, the memory occupancy status of the target device in S101 satisfies the memory exhaustion terminator triggering condition, which may include: monitoring the target device's target operating system kernel in units of control groups, and if there is a control group whose memory occupancy status satisfies the corresponding threshold of the control group, then determining that the target device's memory occupancy status satisfies the memory exhaustion terminator triggering condition. Determining the target process based on the memory exhaustion score of the application process and executing a memory recovery operation on the target process in S105 may include: for the control group that satisfies the corresponding threshold, determining the target process based on the memory exhaustion score of the application process within the control group, and executing a memory recovery operation on the target process.

[0076] like Figure 2 As shown, an embodiment of the present invention provides a memory exhaustion terminator triggering method that sets local thresholds in units of control groups (cgroups). Local thresholds are set at the control group level, and processes within the same control group are terminated first. For a containerized environment, each container belongs to a specific control group. An independent threshold can be set for each control group. When the memory usage of a control group reaches its set threshold, processes within the control group are selected for termination first, rather than directly selecting across the entire system. For example, in a Kubernetes cluster containing multiple containers, different memory exhaustion terminator thresholds are set for the control group where each container is located (such as 50%, 60%, etc., based on the importance of the container and resource allocation). When the memory usage of a container reaches its corresponding threshold, some non-critical processes are first attempted to be terminated within the container to free up memory and avoid affecting the normal operation of other containers.

[0077] In some optional implementations of the embodiments of the present invention, the memory occupancy status of the control group satisfies a threshold corresponding to the control group, which may include: a memory occupancy status parameter of the control group reaches a first threshold corresponding to the control group.

[0078] In other optional implementations of the embodiments of the present invention, the memory occupancy status of the control group satisfies the threshold corresponding to the control group, and may also include: multiple load status parameters of the control group reach the corresponding threshold; one of the load status parameters is a memory occupancy status parameter.

[0079] In order to more reasonably trigger the memory exhaustion terminator mechanism, the embodiment of the present invention can adopt a threshold triggering scheme dynamically according to the system load. In addition to the memory occupancy status parameter, other load status parameters can be calculated as a combined value for judgment.

[0080] In an embodiment of the present invention, multiple load status parameters of a control group reaching corresponding thresholds may include: the memory occupancy status parameter of the control group reaching a corresponding first threshold and other load status parameters of the control group reaching a corresponding second threshold; or the memory occupancy status parameter of the control group reaching a corresponding first threshold and other load status parameters of the application process with the highest memory usage in the control group reaching a corresponding second threshold. For example, the memory exhaustion terminator mechanism will be triggered only when the system free memory is below a certain percentage and the overall system load reaches a certain value, or when the load of the process with the highest memory usage exceeds a certain value.

[0081] In actual use, it may be necessary to dynamically adapt and adjust the memory exhaustion terminator threshold based on actual business application usage. For example, it can be set to trigger when the free memory is less than 4.7% and the load exceeds 83%, or when the free memory is less than 4.7% and the load of the process with the highest memory usage exceeds 50%.

[0082] By monitoring the system's free memory and load in real time, the system can more accurately determine whether it is necessary to activate the memory exhaustion terminator mechanism.

[0083] In actual implementation, free memory and load information can be obtained in real time through relevant functions in the kernel (such as monitor_memory_and_load()), and the memory usage of the process can be obtained through the process descriptor task_struct structure, and judgment can be made based on the set threshold.

[0084] In other optional implementations of the embodiments of the present invention, multiple load status parameters can also be monitored as a whole unit of the target device, and the memory exhaustion terminator mechanism can be triggered only when multiple load status parameters of the target system reach corresponding thresholds. One of the load status parameters is a memory occupancy status parameter. The implementation method can refer to the above-mentioned implementation method based on the control group.

[0085] Figure 3 A flowchart of the training steps of a prediction model provided by an embodiment of the present invention; Figure 4 A flowchart of memory pressure prediction steps provided by an embodiment of the present invention; Figure 5 A flowchart of steps for actively intervening in memory pressure provided by an embodiment of the present invention.

[0086] In response to the problem of the current memory exhaustion terminator mechanism's delayed response, such as Figure 2As shown, the application process monitoring method provided by the embodiment of the present invention may also include: using the historical memory usage data of the target device to train a prediction model; obtaining the memory usage data of the target device at the current moment; inputting the memory usage data and the time information of the current moment into the prediction model to obtain a predicted value of the memory usage parameter in the first time in the future; and performing a lightweight memory recovery operation on the application process according to the predicted value of the memory usage parameter.

[0087] Among them, performing a lightweight memory recovery operation on the application process according to the memory usage parameter prediction value may include: if the memory usage parameter prediction value meets the third threshold, performing a lightweight memory recovery operation on the application process; the memory recovery ratio of the lightweight memory recovery operation is lower than the ratio of the memory recovery operation performed when the memory exhaustion terminator trigger condition is met.

[0088] This means training a prediction model based on historical memory usage data to proactively trigger lightweight recycling or throttling. In a file server system, when memory is exhausted, file read and write operations become extremely slow or even stall due to insufficient memory to cache data. This also impacts process scheduling, preventing other normally running processes from obtaining CPU resources in a timely manner. In this scenario, time series analysis algorithms (such as the ARIMA model) can be used to analyze and predict historical memory usage data. This involves collecting system memory usage data over a period of time (e.g., a week), including information such as memory utilization, memory allocation and release frequencies, and other information over different time periods, and then inputting this information into the ARIMA model for training.

[0089] like Figure 3 As shown, the type of historical memory usage data used may include at least one of memory occupancy, memory allocation information, and memory release frequency. In some optional implementations of the embodiments of the present invention, the three types of historical memory usage data, namely memory occupancy, memory allocation information, and memory release frequency, may be used to train a prediction model.

[0090] like Figure 4As shown, after model training is complete, memory usage over the next period of time (e.g., the next 10 minutes) can be predicted in real time based on real-time memory usage data input. Current memory usage data is obtained, specifically including the current memory occupancy rate, memory allocation information, and memory release frequency. This memory usage data is input into the trained prediction model, and a predicted memory usage parameter value is output. A determination is then made as to whether the predicted memory usage parameter value meets a third threshold. If so, a lightweight memory reclamation operation is triggered, and memory usage data is continuously monitored, i.e., the current memory usage data is obtained. If not, memory usage data is continuously monitored, i.e., the current memory usage data is obtained. In other words, by continuously monitoring memory usage data, when an imminent increase in memory pressure is predicted (i.e., the predicted memory usage parameter value meets the third threshold), a lightweight reclamation mechanism is triggered in advance, such as reclaiming some cached pages or throttling certain non-critical services. This prevents memory exhaustion from impacting the system's file read and write operations and process scheduling, preventing other normally running processes from obtaining CPU resources in a timely manner.

[0091] like Figure 2 As shown, the application process monitoring method provided by the embodiment of the present invention may further include: if the predicted value of the memory usage parameter meets the third threshold, sending a memory pressure notification to the user-state application of the application process, so that the user-state application performs a memory pressure relief operation.

[0092] Specifically, sending a memory pressure notification to a user-mode application of an application process so that the user-mode application performs a memory pressure relief operation may include: pre-adding a memory pressure notification interface to a virtual memory management parameter configuration directory of a target operating system so that the user-mode application registers a callback function that responds to memory pressure events; and sending a memory pressure notification to a user-mode application interface of the user-mode application through the memory pressure notification interface so that the user-mode application calls the callback function to perform a memory pressure relief operation.

[0093] / proc / sys / vm / is the Linux kernel's virtual memory management parameter configuration directory, used to dynamically adjust memory allocation and reclamation policies and optimize performance. A new API (Application Programming Interface) has been added to / proc / sys / vm / mem_pressure, allowing applications to register callback functions in response to memory pressure events.

[0094] like Figure 5As shown, in an embodiment of the present invention, under the above-mentioned system real-time monitoring of memory and prediction model prediction strategy, a threshold judgment can be made on the memory usage data at the current moment, and the memory usage data at the current moment is input into the prediction model to obtain the memory usage parameter prediction value, that is, the memory usage at the current moment and the prediction of future memory usage are used to determine whether the memory pressure increases. When the memory pressure is insufficient or is about to increase, the lightweight recovery mechanism can be triggered in advance, and a memory pressure notification can be sent to the user-state application interface through the interface. After receiving the notification, the user-state application calls the registered callback function to perform the corresponding memory pressure relief operation, such as releasing some temporary resources, adjusting business logic to reduce memory usage, etc.

[0095] Figure 6 A flowchart of the steps for constructing a fault context provided by an embodiment of the present invention; Figure 7 A flowchart of an event notification step provided by an embodiment of the present invention.

[0096] In related technologies, after the memory exhaustion terminator mechanism is triggered, the terminated application process lacks context records, resulting in the user-mode application being unaware of the specific circumstances of the terminated process, which is not conducive to subsequent fault analysis.

[0097] In this regard, Figure 2 As shown, the application process monitoring method provided by an embodiment of the present invention may also include: after starting the memory recovery operation, recording information about the memory recovery operation in the system log of the target device, the information about the memory recovery operation includes at least one of the process identifier of the terminated process, the name of the terminated process, and the user rights of the terminated process; collecting metadata information of the terminated process from the kernel audit subsystem of the target device, and generating memory mapping data of the terminated process; obtaining resource time series data of the terminated process within a second time before failure; and constructing a fault context based on the information about the memory recovery operation, the memory mapping data, and the resource time series data.

[0098] Specifically, if Figure 6 As shown in the figure, after the memory exhaustion killer is triggered and the process is terminated, the system can collect and persistently store more metadata information of the terminated process (such as the process identifier PID of the parent process, process priority, process life cycle, etc.) through the kernel audit subsystem, and simultaneously generate a memory snapshot containing complete memory mapping details (user-mode heap, kernel-mode allocator (SLAB)), etc. and key stack / page table structures.

[0099] At the same time, the resource time series data of the terminated process within the second time (such as 60 seconds) before failure (such as the process's interrupt usage, central processing unit utilization, block device I / O throughput data volume performed by the process, network bandwidth occupancy, etc.) can be traced to obtain the load status parameters of the terminated process for recording.

[0100] In addition, process topology analysis based on process tracing (ptrace) or control groups (cgroups) can record the connection status between processes, process tree lineage relationships, and shared memory dependency maps, building a complete fault context to provide the data needed to analyze memory faults and perform detailed root cause diagnosis.

[0101] The application process monitoring method provided by the embodiment of the present invention may further include: sending a structured event of the terminated process to the user-state application, so that the user-state application performs active operation and maintenance intervention.

[0102] like Figure 7 As shown in the figure, after the memory exhaustion terminator is triggered and the process is terminated, the structured information of the terminated process can be sent to the user state through the network link (Netlink) mechanism to support active operation and maintenance intervention.

[0103] Netlink is a communication mechanism between user space and kernel space in the Linux kernel, primarily used to efficiently transmit structured data and control information. When the out-of-memory killer mechanism is triggered or system memory pressure reaches a certain threshold, the kernel can use Netlink to send the structured information of the terminated process to the user-space monitoring application. This information can include a memory snapshot of the terminated process, resource usage history, and detailed information about associated processes.

[0104] After receiving these events, the user-mode monitoring application (user-mode operation and maintenance tool) can analyze the event content, actively intervene based on the event content, and perform intervention optimization operations, such as remotely logging into the system for further investigation and processing, or automatically executing some pre-set recovery operations to reduce the potential impact on the system after the memory exhaustion terminator mechanism is triggered.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0106] An embodiment of the present invention also provides an application process monitoring device, comprising: a monitoring unit for monitoring the memory occupancy status of a target device; a first acquisition unit for acquiring the business logic weight of the application process running on the target device when the monitoring unit detects that the memory occupancy status of the target device meets the memory exhaustion terminator triggering condition; a first determination unit for determining the weight score of the application process based on the business logic weight of the application process and the priority weight of the application process; a second determination unit for determining the memory exhaustion score of the application process based on the process memory occupancy parameter of the application process and the weight score of the application process; a first control unit for determining the target process based on the memory exhaustion score of the application process, and performing a memory recovery operation on the target process.

[0107] In an embodiment of the present invention, the business logic weight of an application process may include at least one of the process type weight of the application process, the historical memory mode weight of the application process, and the service level agreement weight of the application process; when the application process is a real-time process, the corresponding process type weight is less than the process type weight of other process types; the higher the historical memory stability score of the application process, the lower the historical memory mode weight of the application process; the service level agreement weight is set according to the different service quality requirements of the business system corresponding to the application process.

[0108] In an embodiment of the present invention, the priority weight of an application process may include the system process priority weight of the application process and the user-defined weight of the application process; the system process priority weight of the application process is the priority weight configured by the target device when creating the application process; the user-defined weight is the priority weight customized by the user of the application process for the application process.

[0109] In an embodiment of the present invention, the first determination unit determines the weight score of the application process based on the business logic weight of the application process and the priority weight of the application process, which may include: taking the sum of the business logic weight of the application process and the priority weight of the application process as the weight score of the application process.

[0110] In an embodiment of the present invention, the process memory occupancy parameter of the application process may be the process memory occupancy ratio of the application process; the second determination unit determines the memory exhaustion score of the application process based on the process memory occupancy parameter of the application process and the weight score of the application process, which may include: taking the weight score of the application process as a power, calculating the power of 2 multiplied by the process memory occupancy ratio and dividing the value by 100 to obtain the memory exhaustion score of the application process.

[0111] In an embodiment of the present invention, the monitoring unit detecting that the memory usage status of the target device meets the memory exhaustion terminator triggering condition may include: monitoring control groups in the target operating system kernel of the target device as units, and if the memory usage status of any control group meets the threshold corresponding to the control group, determining that the memory usage status of the target device meets the memory exhaustion terminator triggering condition. The first control unit determining the target process based on the memory exhaustion score of the application process and performing a memory reclaim operation on the target process may include: for the control group that meets the corresponding threshold, determining the target process based on the memory exhaustion score of the application process within the control group, and performing a memory reclaim operation on the target process.

[0112] In the embodiment of the present invention, the memory occupancy status of the control group satisfies a threshold corresponding to the control group, which may include: a memory occupancy status parameter of the control group reaches a first threshold corresponding to the control group.

[0113] In the embodiment of the present invention, the memory occupancy status of the control group satisfies a threshold corresponding to the control group, which may include: multiple load status parameters of the control group reach corresponding thresholds; and one of the load status parameters is a memory occupancy status parameter.

[0114] In an embodiment of the present invention, multiple load status parameters of the control group reach corresponding thresholds, which may include: the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the control group reach the corresponding second threshold; or, the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the application process with the highest memory occupancy in the control group reach the corresponding second threshold.

[0115] The application process monitoring device provided by an embodiment of the present invention may also include: a training unit, used to train a prediction model using the historical memory usage data of the target device; a second acquisition unit, used to obtain the memory usage data of the target device at the current moment; a calculation unit, used to input the memory usage data and the time information of the current moment into the prediction model to obtain a predicted value of the memory usage parameter within the first time in the future; a second control unit, used to perform a lightweight memory recovery operation on the application process if the predicted value of the memory usage parameter meets a third threshold; the memory recovery ratio of the lightweight memory recovery operation is lower than the ratio of the memory recovery operation performed when the memory exhaustion terminator trigger condition is met.

[0116] The application process monitoring device provided by an embodiment of the present invention may further include: a first sending unit, configured to send a memory pressure notification to a user-state application of the application process if the predicted value of the memory usage parameter meets a third threshold, so that the user-state application performs a memory pressure relief operation.

[0117] The application process monitoring device provided by an embodiment of the present invention may also include: a recording unit, which is used to record information about the memory recovery operation in the system log of the target device after the memory recovery operation is started, and the information about the memory recovery operation includes at least one of the process identifier of the terminated process, the name of the terminated process, and the user permissions of the terminated process; collect metadata information of the terminated process from the kernel audit subsystem of the target device, and generate memory mapping data of the terminated process; obtain resource time series data of the terminated process within the second time before failure; and construct a fault context based on the information about the memory recovery operation, the memory mapping data, and the resource time series data.

[0118] For the description of the features in the embodiment corresponding to the application process monitoring device, please refer to the relevant description of the embodiment corresponding to the application process monitoring method, which will not be repeated here.

[0119] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned application process monitoring method embodiments.

[0120] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned application process monitoring method embodiments when running.

[0121] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0122] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned application process monitoring method embodiments are implemented.

[0123] An embodiment of the present invention also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned application process monitoring method embodiments.

[0124] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0125] The above is a detailed introduction to the application process monitoring method, device, medium and computer program product provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for monitoring an application process, characterized in that: include: Monitor the memory usage of the target device; When the memory occupancy state of the target device meets the memory exhaustion terminator triggering condition, obtaining the business logic weight of the application process running on the target device; Determining a weight score of the application process according to the business logic weight of the application process and the priority weight of the application process; determining a memory exhaustion score of the application process according to a process memory occupancy parameter of the application process and a weight score of the application process; Determining a target process according to the memory exhaustion score of the application process, and performing a memory reclaiming operation on the target process; A prediction model is obtained by training the target device using historical memory usage data; the types of the historical memory usage data include memory occupancy rate, memory allocation information and memory release frequency; Obtain memory usage data of the target device at the current moment; Inputting the memory usage data and the current time information into the prediction model to obtain a memory usage parameter prediction value within the first time in the future; If the predicted value of the memory usage parameter meets a third threshold, performing a lightweight memory reclaim operation on the application process; The memory reclamation ratio of the lightweight memory reclamation operation is lower than the ratio of the memory reclamation operation executed when the memory exhaustion terminator triggering condition is met; The step of obtaining the business logic weight of the application process running on the target device includes: Obtaining a process type weight of the application process; the process type weight of the real-time process is less than the process type weights of other process types; Calculating a historical memory stability score of the application process based on the historical memory usage of the application process, and calculating a historical memory mode weight of the application process based on the historical memory stability score; the higher the historical memory stability score of the application process, the lower the historical memory mode weight of the application process; The service level agreement weight is set according to the service quality requirements of different processes of the business system corresponding to the application process; The business logic weight of the application process is determined according to the process type weight of the application process, the historical memory mode weight of the application process, and the service level agreement weight of the application process.

2. The application process monitoring method according to claim 1, characterized in that: The priority weight of the application process includes the system process priority weight of the application process and the user-defined weight of the application process; The system process priority weight of the application process is the priority weight configured by the target device when creating the application process; The user-defined weight is a priority weight that is customized by a user of the application process for the application process.

3. The application process monitoring method according to claim 1, characterized in that: Determining a weight score of the application process according to the business logic weight of the application process and the priority weight of the application process includes: The sum of the business logic weight of the application process and the priority weight of the application process is taken as the weight score of the application process.

4. The application process monitoring method according to claim 3, characterized in that: The process memory occupancy parameter of the application process is the process memory occupancy ratio of the application process; Determining a memory exhaustion score of the application process according to a process memory occupation parameter of the application process and a weight score of the application process includes: Taking the weight score of the application process as a power, calculate the value of multiplying the power of 2 by the memory usage ratio of the process and dividing the result by 100 to obtain the memory exhaustion score of the application process.

5. The application process monitoring method according to claim 1, characterized in that: The memory occupancy status of the target device satisfies the memory exhaustion terminator triggering condition, including: Monitoring is performed in units of control groups in a target operating system kernel of the target device, and if a memory occupancy status of any of the control groups satisfies a threshold corresponding to the control group, determining that the memory occupancy status of the target device satisfies a trigger condition of the memory exhaustion terminator; Determining a target process according to the memory exhaustion score of the application process and performing a memory reclaiming operation on the target process includes: For the control group that meets the corresponding threshold, the target process is determined according to the memory exhaustion score of the application process in the control group, and a memory reclaiming operation is performed on the target process.

6. The application process monitoring method according to claim 5, characterized in that: The memory usage status of the control group meets the threshold corresponding to the control group, including: The memory occupancy status parameter of the control group reaches a first threshold corresponding to the control group.

7. The application process monitoring method according to claim 5, characterized in that: The memory usage status of the control group meets the threshold corresponding to the control group, including: The multiple load status parameters of the control group reach corresponding thresholds; One of the load status parameters is a memory occupancy status parameter.

8. The application process monitoring method according to claim 7, characterized in that: Multiple load status parameters of the control group reach corresponding thresholds, including: The memory occupancy status parameter of the control group reaches a corresponding first threshold and the other load status parameters of the control group reach a corresponding second threshold; Alternatively, the memory occupancy status parameter of the control group reaches the corresponding first threshold and other load status parameters of the application process with the highest memory occupancy in the control group reach the corresponding second threshold.

9. The application process monitoring method according to claim 1, characterized in that: Also includes: If the predicted value of the memory usage parameter meets the third threshold, a memory pressure notification is sent to the user-mode application of the application process, so that the user-mode application performs a memory pressure relief operation.

10. The application process monitoring method according to claim 1, characterized in that: Also includes: After initiating the memory reclaim operation, recording information about the memory reclaim operation in a system log of the target device, the information about the memory reclaim operation including at least one of a process identifier of a terminated process, a name of the terminated process, and user rights of the terminated process; Collecting metadata information of the terminated process from the kernel audit subsystem of the target device and generating memory mapping data of the terminated process; Obtaining resource time series data of the terminated process within a second time period before failure; A fault context is constructed according to the information of the memory reclaim operation, the memory mapping data, and the resource time series data.

11. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the application process monitoring method according to any one of claims 1 to 10 when executing the computer program.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the application process monitoring method according to any one of claims 1 to 10 are implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the application process monitoring method according to any one of claims 1 to 10 are implemented.

Images