Memory management method and device, equipment and medium

Abstract

The invention relates to the technical field of memory management, and discloses a memory management method and device, equipment and a medium, and the method comprises the steps: obtaining memory use information, and generating a memory feature vector based on the memory use information; the memory feature vectors are input into various basic models in the stacking model, a plurality of first prediction results are obtained, and the first prediction results comprise possible reasons for limiting the system performance due to the use condition of the memory; inputting the plurality of first prediction results into a meta-model in the stack model, and performing secondary prediction on the plurality of first prediction results based on the meta-model to obtain a second prediction result; and performing memory management based on the second prediction result. Through the multiple basic models, possible reasons for limiting system performance by memory use conditions are predicted, prediction results of the multiple basic models are learned through the meta-model, the advantages of different basic models are effectively integrated, possible deviation of a single basic model is reduced, and the overall prediction capacity of the stacked model is improved.

Description

Technical Field

[0001] The present invention relates to the field of memory management technology, and in particular to a method, device, equipment and medium for memory management. Background Art

[0002] QNX Realtime Operating System is a real-time operating system widely used in embedded systems, with high performance, stability and reliability. With the rapid development of information technology, operating systems have increasingly higher requirements for memory management when processing complex tasks and large amounts of data. As a real-time operating system widely used in embedded systems, the efficiency and stability of memory management of the QNX system directly affect the overall performance of the system. However, the memory management method of the QNX system in the related art has some shortcomings, which limits the further improvement of system performance.

[0003] The memory management methods in related technologies rely on a single machine learning model when identifying memory bottlenecks. This may lead to insufficient generalization capabilities and difficulty in handling diverse memory usage patterns due to the complexity of data distribution or limitations of model structure. Summary of the invention

[0004] In view of this, the present invention provides a method, apparatus, device and medium for memory management to solve the problem of identifying memory bottlenecks through a single machine learning model, the model's generalization ability is insufficient, and it cannot cope with diverse memory usage patterns.

[0005] In a first aspect, the present invention provides a method for memory management, the method comprising: obtaining memory usage information, and generating a memory feature vector based on the memory usage information; inputting the memory feature vector into multiple basic models in a stacking model to obtain multiple first prediction results, wherein the first prediction results include possible reasons why system performance is limited due to memory usage; inputting multiple first prediction results into a meta-model in the stacking model, and performing secondary prediction on the multiple first prediction results based on the meta-model to obtain a second prediction result; and performing memory management based on the second prediction result.

[0006] In an optional embodiment, the memory feature vector is input into multiple basic models in a stacked model to obtain multiple first prediction results, including: creating a memory usage data set based on the memory usage information; performing random sampling based on the memory usage data set to obtain multiple memory usage data subsets; building a decision tree model based on the memory usage data subsets; and inputting the memory feature vector into each of the decision tree models to generate the first prediction result.

[0007] In an optional embodiment, the method of inputting the memory feature vector into multiple basic models in a stacked model to obtain multiple first prediction results also includes: initializing a predictor; in each iteration process, calculating the residual of each training sample based on the predictor; training a weak learner based on the residual; adding the weak learner to the predictor based on a learning rate; repeating the aforementioned residual calculation, weak learner training and weak learner addition steps until a predictor training stop condition is met to obtain a prediction model; and inputting the memory feature vector into the prediction model to generate the first prediction result.

[0008] In an optional embodiment, the step of inputting the memory feature vector into multiple basic models in a stacked model to obtain multiple first prediction results also includes: initializing a deep neural network structure; inputting training data into the deep neural network for forward calculation, transmitting information layer by layer through the deep neural network to obtain a network output; calculating the difference between a true value and the network output based on a loss function; calculating the gradient of the loss function with respect to network parameters, and reversely updating the network parameters based on the gradient to minimize the loss function; repeating the steps of forward calculation, calculation of the difference and minimization of the loss function until a deep neural network training stop condition is met to obtain a deep neural network model; and inputting the memory feature vector into the deep neural network model to generate the first prediction result.

[0009] In an optional embodiment, the step of inputting the multiple first prediction results into the meta-model in the stacking model, and performing secondary prediction on the multiple first prediction results based on the meta-model to obtain a second prediction result includes: inputting the multiple first prediction results into the meta-model in the stacking model, and performing weighted averaging on the multiple first prediction results based on the meta-model to generate the second prediction result.

[0010] In an optional embodiment, the memory management is performed based on the second prediction result, including: if the second prediction result characterizes memory allocation and release frequency limiting system performance; allocating a preset memory pool, and dividing the preset memory pool into multiple first memory blocks; when allocating memory to a target program, allocating the first memory block to the target program; when the target program releases memory, releasing the first memory block to the preset memory pool.

[0011] In an optional implementation, the memory management based on the second prediction result further includes: if the second prediction result indicates that the memory fragmentation rate limits system performance; moving the data in the memory to a continuous memory area.

[0012] In a second aspect, the present invention provides a device for memory management, the device comprising: a vector acquisition module, used to acquire memory usage information, and generate a memory feature vector based on the memory usage information; a first prediction module, used to input the memory feature vector into multiple basic models to obtain multiple first prediction results, wherein the first prediction results include possible reasons for limiting system performance due to memory usage; a second prediction module, used to input multiple first prediction results into a meta-model in the stacking model, and perform secondary prediction on the multiple first prediction results based on the meta-model to obtain a second prediction result; and a memory management module, used to perform memory management based on the second prediction result.

[0013] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for memory management of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for memory management of the above-mentioned first aspect or any corresponding embodiment thereof.

[0015] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for memory management of the first aspect or any corresponding embodiment thereof.

[0016] Through multiple basic models, we predict the possible reasons why memory usage limits system performance. By learning the prediction results of multiple basic models through meta-models, we effectively integrate the advantages of different basic models, reduce the possible deviations of a single basic model, improve the overall prediction ability of the stacked model, and further improve the accuracy of the prediction. Through the integrated stacking method, we improve the overall performance of the methods used for memory management, making the formulation of memory management strategies more scientific and effective. This not only helps to discover potential memory problems in advance, but also guides system administrators or developers to take targeted management measures, thereby improving the overall performance and stability of the QNX system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a method for memory management provided by an embodiment of the present invention is shown;

[0019] Figure 2 A schematic flow chart of another method for memory management provided by an embodiment of the present invention is shown;

[0020] Figure 3 A schematic diagram of the structure of a device for memory management provided by an embodiment of the present invention is shown;

[0021] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] The QNX system adopts a microkernel architecture, is highly modular and scalable, and is suitable for a variety of embedded devices. With the rapid development of technologies such as the Internet of Things and artificial intelligence, the application of the QNX system in the embedded market will become more extensive. QNX Software Systems continues to optimize its products, strengthen cooperation with upstream and downstream companies in the industry chain, and promote the implementation of the QNX system in more fields.

[0024] The memory management methods in related technologies rely on a single machine learning model when identifying memory bottlenecks. This may result in insufficient generalization capabilities and difficulty in handling diverse memory usage patterns due to the complexity of data distribution or limitations of model structure. At the same time, some complex models may perform well on training data but perform poorly when processing new data, i.e., there is an overfitting problem, which may affect the overall judgment accuracy.

[0025] According to an embodiment of the present invention, a method embodiment for memory management is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] In this embodiment, a method for memory management is provided, which can be used in embedded terminals, such as mobile phones, household appliances or cars, etc. Figure 1A schematic diagram of a method for memory management provided by an embodiment of the present invention is shown. Figure 1 As shown, the process includes the following steps:

[0027] Step S101, obtaining memory usage information, and generating a memory feature vector based on the memory usage information.

[0028] In this step, the current system memory usage is analyzed. You can use the memstat tool provided by the QNX system to obtain the current system memory usage information. The memory usage information includes one or more of the following: total memory, used memory, free memory, memory allocation times, memory release times, memory occupancy time, and cache memory.

[0029] Specifically, start the memstat tool, set the appropriate sampling interval and duration, so as to obtain enough data points to reflect the memory usage; memstat output includes total memory size, used memory, free memory, memory allocation times, memory release times, memory occupancy time, and detailed information of cache memory; export and store these data in log files for subsequent analysis. In addition, memstat is also executed regularly through scripts to monitor the trend of memory usage over time, which can provide real-time and historical data support for the identification of memory bottlenecks.

[0030] Based on the memory usage information, the generated memory features include at least one of the following: total memory, used memory, free memory, and cache memory. The aforementioned memory features are quantized, and the selected memory features are converted into a numerical value. If the range of the eigenvalues ​​varies greatly, the eigenvalues ​​with large range differences can be standardized or normalized to ensure that the features have similar weights in the eigenvector. The quantized eigenvalues ​​are combined to obtain a memory feature vector.

[0031] The memory feature vector includes at least one of the following: memory usage, memory allocation times, memory release times, memory occupation time, etc.

[0032] Step S102, inputting the memory feature vector into multiple basic models in the stacking model to obtain multiple first prediction results, wherein the first prediction results include possible reasons for limiting system performance due to memory usage.

[0033] In this step, the base model is a plurality of different machine learning models that are selected and trained separately for prediction in the stacking ensemble learning method. These models can be any type of machine learning algorithm, such as linear regression, decision tree, random forest, support vector machine, etc. Specifically, the base model can be a random forest model, a gradient boosting machine model, or a deep neural network model.

[0034] The first prediction result further includes: the frequency of memory allocation and release, and the memory fragmentation rate to show the system performance.

[0035] Step S103: input the plurality of first prediction results into a meta-model in the stacking model, and perform secondary prediction on the plurality of first prediction results based on the meta-model to obtain a second prediction result.

[0036] In this step, the goal of the meta-model is to learn how to further extract information from the first prediction result generated by the basic model to make a more accurate prediction. The meta-model can be a machine learning algorithm, such as logistic regression or a neural network.

[0037] The first prediction result obtained based on the basic model is input into the meta-model as a new feature. The meta-model can significantly improve the prediction performance and generalization results by combining the prediction results of multiple basic models.

[0038] Step S104: performing memory management based on the second prediction result.

[0039] In this step, the second prediction result combines the first prediction results of multiple basic models and has higher accuracy and reliability. Based on the second prediction result, the system can predict the trend of future memory demand changes and allocate or release memory resources in advance. For example, if the prediction result indicates that there may be insufficient memory to limit system performance, more memory resources can be allocated to avoid performance degradation caused by insufficient memory.

[0040] If the second prediction result indicates low memory utilization, high memory fragmentation, memory leaks, slow memory call response, or reduced throughput, the memory allocation strategy can be adjusted based on the second prediction result, such as using a more efficient memory allocation algorithm or moving infrequently used data to disk to free up memory.

[0041] The method for memory management provided in this embodiment predicts the possible reasons why memory usage limits system performance through multiple basic models, and effectively integrates the advantages of different basic models by learning the prediction results of multiple basic models through meta-model, reduces the possible deviation of a single basic model, improves the overall prediction ability of the stacked model, and further improves the accuracy of the prediction. Through the integrated stacking method, the overall performance of the method for memory management is improved, making the formulation of memory management strategies more scientific and effective; this not only helps to discover potential memory problems in advance, but also can guide system administrators or developers to take targeted management measures, thereby improving the overall performance and stability of the QNX system.

[0042] In some optional embodiments, a memory feature vector is input into multiple basic models in a stacked model to obtain multiple first prediction results, including: creating a memory usage data set based on memory usage information; performing random sampling based on the memory usage data set to obtain multiple memory usage data subsets; building a decision tree model based on the memory usage data subsets; and inputting the memory feature vector into each decision tree model for integration to generate a first prediction result.

[0043] In this embodiment, the memory bottleneck can be identified by the random forest basic model. The memory bottleneck refers to the use efficiency or capacity limitation of memory resources that affects the overall performance of the system. By identifying these bottlenecks, measures can be taken to optimize memory usage, thereby improving system performance.

[0044] The memory feature vector can be classified or regressed through each decision tree model. The prediction results of all decision trees can be integrated through voting classification or average regression. If the possible reasons for the limitation of system performance due to memory usage are predicted by classification, the first prediction result obtained based on the random forest model is the mode of the first prediction results of all decision tree models. If the possible reasons for the limitation of system performance due to memory usage are predicted by regression prediction, the first prediction result obtained based on the random forest model is the average of the first prediction results of all decision tree models.

[0045] Specifically, the first prediction result generated based on the random forest basic model can be obtained by the following formula:

[0046]

[0047] Where T is the number of decision tree models, is the prediction result of the decision tree model of lesson i, is the first prediction result generated based on the random forest base model, and i is the index variable used to iterate each decision tree model in the summation process.

[0048] The first prediction result may be a classification of the memory bottleneck or a prediction probability. In this embodiment, it may be represented by the voting results of multiple decision tree models. The random forest reduces overfitting by constructing multiple decision tree models and voting on the prediction results of multiple decision tree models, thereby improving the generalization ability of the random forest model.

[0049] In this way, the random forest basic model effectively reduces the risk of overfitting and improves the generalization ability of the random forest model for memory usage patterns by building multiple decision tree models and integrating their prediction results. This method can extract key features from complex memory usage data and help identify factors that may cause system performance bottlenecks.

[0050] In an optional embodiment, the memory feature vector is input into multiple basic models in the stacked model to obtain multiple first prediction results, and the method also includes: initializing the predictor; in each iteration process, based on the predictor, calculating the residual of each training sample; based on the residual, training the weak learner; based on the learning rate, adding the weak learner to the predictor; repeating the aforementioned residual calculation, weak learner training and weak learner addition steps until the predictor training stop condition is met to obtain a prediction model; inputting the memory feature vector into the prediction model to generate a first prediction result.

[0051] In this embodiment, the memory bottleneck can be identified by the gradient boosting machine base model. The first prediction result generated based on the gradient boosting machine base model can be the result of a strong prediction model constructed by iteratively reducing the residual. The predictor training stop condition can be reaching a preset number of training rounds or a time limit. Based on the learning rate, adding a weak learner to the predictor can be achieved by the following formula:

[0052] F m (x) = F m-1 (x)+ηh m (x)

[0053] Among them, F m (x) is the predictor model of the mth iteration, F m-1 (x) is the predictor model of the m-1th iteration, η is the learning rate, h m (x) is the weak learner in the mth round.

[0054] In this way, the gradient boosting machine basic model reduces the model residual through iterative training. Each iteration focuses on reducing the residual of the previous iteration, and gradually builds a strong prediction model that can capture the nonlinear relationship in memory usage and provide more accurate predictions for the identification of memory bottlenecks.

[0055] In some optional embodiments, the memory feature vector is input into multiple basic models in the stacked model to obtain multiple first prediction results, and it also includes: initializing the deep neural network structure; inputting the training data into the deep neural network for forward calculation, passing information layer by layer through the deep neural network to obtain the network output; calculating the difference between the true value and the network output based on the loss function; calculating the gradient of the loss function with respect to the network parameters, and updating the network parameters based on the gradient inversely to minimize the loss function; repeating the steps of forward calculation, difference calculation and loss function minimization until the deep neural network training stopping condition is met to obtain the deep neural network model; inputting the memory feature vector into the deep neural network model to generate the first prediction result.

[0056] In this embodiment, the memory bottleneck can be identified by the deep neural network basic model. The first prediction result generated based on the deep neural network basic model can be obtained by training with a multi-layer perceptron structure and a back propagation algorithm. The deep neural network training stop condition can be reaching a preset number of training rounds or a time limit.

[0057] The deep neural network structure may include an input layer, multiple hidden layers, and an output layer. Activation functions can be used to process data between layers, such as ReLU or Sigmoid. The activation output of each layer of the deep neural network can be calculated using the following formula:

[0058] a (l) =f(W (l) a (l-1) +b (l) )

[0059] Among them, a (l) is the activation output of layer l, W (l) is the weight matrix of the lth layer, b (l) is the bias vector of the lth layer, and f is the activation function.

[0060] In this way, based on the basic model of deep neural network, through the multi-layer perceptron structure, and using the back propagation algorithm to train the network weights, the complex nonlinear relationship in the memory feature vector can be captured.

[0061] The random forest model, gradient boosting machine model and deep neural network model are used in combination as basic models, so that the identification of memory bottlenecks not only covers multiple dimensions of data, but also improves the accuracy and robustness of predictions, providing a solid data foundation for subsequent memory optimization management.

[0062] In some optional embodiments, multiple first prediction results are input into a meta-model in a stacking model, and a second prediction result is obtained by performing a secondary prediction on the multiple first prediction results based on the meta-model to obtain a second prediction result, including: inputting multiple first prediction results into a meta-model in the stacking model, and performing a weighted average on the multiple first prediction results based on the meta-model to generate a second prediction result.

[0063] In this embodiment, the first prediction result obtained based on the random forest basic model, the gradient boosting machine basic model and the deep neural network basic model can be used as a new feature vector and input into the meta-model in the stacking model. The meta-model performs a weighted combination on the prediction results of the three basic models to obtain a second prediction result.

[0064] Specifically, three basic models (RF, GBM, DNN) are trained, namely, the random forest basic model, the gradient boosting machine basic model, and the deep neural network basic model, to obtain multiple first prediction results. The first prediction results are input into the meta-model as features, and the meta-model is trained to combine the prediction results of the basic models. The second prediction result can be obtained by the following formula:

[0065]

[0066] in, is the second prediction result, g is the meta-model, This is the first prediction result based on the random forest basic model. This is the first prediction result based on the basic model of the gradient boosting machine. It is the first prediction result obtained based on the deep neural network basic model.

[0067] In this way, the accuracy and reliability of memory bottleneck identification are further enhanced by using the stacked ensemble learning method. By inputting the prediction results of the three basic models, random forest, gradient boosting machine, and deep neural network, into the meta-model as new features, the stacking method effectively integrates the advantages of different models, reduces the possible deviation of a single model, and improves the prediction ability of the ensemble model. This integration method can not only examine memory usage data from multiple perspectives, but also obtain more comprehensive and accurate memory bottleneck predictions by weighted combination of prediction results of different models.

[0068] In some optional embodiments, memory management is performed based on the second prediction result, including: if the second prediction result characterizes memory allocation and release frequency limiting system performance; allocating a preset memory pool, and dividing the preset memory pool into multiple first memory blocks; when allocating memory to a target program, allocating the first memory block to the target program; when the target program releases memory, releasing the first memory block to the preset memory pool.

[0069] In this embodiment, if the frequency of memory allocation and release has a significant impact on the performance of the operating system, it can be considered that the frequency of memory allocation and release limits the system performance. The memory pool technology can be used to adjust the memory allocation strategy to reduce the number of memory allocation and release. The memory allocator parameters can also be adjusted to optimize the memory allocation performance. The cache amount, that is, the cache size, can also be adjusted to make the cache memory more suitable for the operating requirements of the current system and reduce the performance loss caused by cache failure. Specifically, the impact of the current cache size on the system performance can be evaluated, and then the cache size can be adjusted according to the cache hit rate, access mode and importance of cached data of the application.

[0070] Get the reasons why the memory allocation and release frequency limits the system performance. Based on the reasons why the memory allocation and release frequency limits the system performance, perform memory monitoring, monitor the memory usage in real time, and perform memory management. Memory monitoring includes: memory leaks and memory fragmentation. Specifically, you can call malloc or free hooks to track memory allocation and release; regularly sample the memory status and calculate the memory usage rate. The memory usage rate MemoryUsageRate can be calculated by the following formula:

[0071] MemoryUsageRate=(UsedMemory / TotalMemory)*100%

[0072] Among them, UsedMemory is the amount of used memory, and TotalMemory is the total memory size.

[0073] During the memory leak detection process, a timestamp and size can be assigned to each memory allocation record; memory allocation records can be scanned regularly to obtain memory blocks that have not been released for a long time;

[0074] The memory leak score can be calculated using the following formula:

[0075] LeakScore = sum(size_i*age_i)

[0076] Among them, LeakScore is the memory leak score; size_i is the size of the i-th memory block; age_i is the duration from the allocation of the i-th memory block to the current time.

[0077] In this way, memory optimization management can be carried out in a timely manner based on the monitored memory usage. By real-time monitoring of memory usage, remaining amount, and frequency of memory allocation and release, potential problems such as memory leaks, insufficient memory, or over-allocation can be quickly identified.

[0078] In some optional implementations, performing memory management based on the second prediction result further includes: if the second prediction result indicates that the memory fragmentation rate limits system performance; moving the data in the memory to a continuous memory area.

[0079] In this embodiment, memory fragmentation monitoring can be used to calculate the size and distribution of free memory blocks after memory allocation and release; the fragmentation rate is used to evaluate the degree of memory fragmentation; and the memory fragmentation rate can be calculated by the following formula:

[0080] FragmentationRate=(SumSmallFragments / TotalFreeMemory)*100%

[0081] Among them, FragmentationRate is the memory fragmentation rate; SumSmallFragments is the sum of the sizes of all free memory blocks smaller than a certain threshold; TotalFreeMemory is the total free memory size.

[0082] The memory fragmentation rate is the ratio of the sum of all small memory fragments SumOfSmallFragments to the total free memory TotalFreeMemory, multiplied by 100% to get the percentage. This formula is used to measure the degree of memory fragmentation, that is, the degree to which free memory is divided into many small fragments. A high fragmentation rate may lead to inefficient memory allocation and affect system performance.

[0083] If the second prediction result indicates that the memory fragmentation rate limits system performance, memory cleanup can be performed regularly. Memory cleanup includes: memory compression and memory migration. Memory compression is to move the data in the memory to a continuous memory area. Memory migration is to migrate the frequently accessed data layer to a continuous area of ​​the physical memory. This can be achieved by writing a special memory management script or using the memory cleanup tool provided by the system.

[0084] In this way, memory utilization can be improved through memory defragmentation algorithms.

[0085] In some optional implementations, if the second prediction result indicates that the frequency of memory allocation and release limits system performance, memory can be reserved for preset tasks and system modules. Specifically, preset tasks and system modules that frequently allocate and release memory are determined, and memory is reserved for them through system configuration or memory management tools; the reserved memory can be achieved by setting a memory protection area or using memory locking technology.

[0086] In this way, it can be ensured that the preset tasks and system modules can obtain sufficient memory resources during operation, and that the operation of the preset tasks and system modules will not be affected by other tasks.

[0087] In some optional implementations, if the second prediction result indicates that the frequency of memory allocation and release limits system performance, the memory access pattern can be optimized, the memory access pattern of the application can be analyzed, and the data structure and algorithm can be optimized. A performance analysis tool can be used to monitor the memory access pattern of the application to find access hot spots and unreasonable memory access operations.

[0088] In this way, unnecessary memory access can be reduced, the frequency of memory allocation and release can be lowered, and memory management can be optimized.

[0089] In some optional implementations, if the second prediction result indicates that the frequency of memory allocation and release limits system performance, the virtual memory settings may be adjusted to adjust the size and strategy of the virtual memory according to system requirements.

[0090] The following formula can help you make decisions when adjusting virtual memory settings:

[0091] PageFaultRate=(PageFaults / TotalPageReferences)*100%

[0092] Among them, PageFaultRate is the page fault rate, PageFaults is the number of page faults that occur within a period of time; TotalPageReferences is the total number of page references in the same period of time.

[0093] EAT=(1-p)*memoryAccessTime+p*(pageFaultRate*diskAccessTime)

[0094] Among them, EAT is the effective access time, p is the probability of page access; memoryAccessTime is the memory access time; diskAccessTime is the disk access time.

[0095] The swap space usage calculation formula is:

[0096] SwapUsageRate=(UsedSwapSpace / TotalSwapSpace)*100%

[0097] Among them, SwapUsageRate is the swap space usage rate, UsedSwapSpace is the currently used swap space size; TotalSwapSpace is the total swap space size;

[0098] The swap space demand forecast calculation formula is:

[0099] PredictedSwapSpace=(HistoricalSwapUsageRate*TotalSystemMemory)+Safety Margin

[0100] Among them, PredictedSwapSpace is the swap space demand prediction, HistoricalSwapUsageRate is the swap space usage rate calculated based on historical data, TotalSystemMemory is the total memory size of the system; SafetyMargin is the additional swap space size reserved to deal with emergencies.

[0101] In this way, by setting up virtual memory, you can use disk space to expand the system's available memory, allowing the system to run more or larger applications, improving system performance and optimizing memory management.

[0102] In some optional implementations, if the second prediction result characterizes that the memory allocation and release frequency limits the system performance, memory protection can be performed on the preset memory area, the preset memory area that needs to be protected is identified, and then the memory protection function provided by the system is used to prevent these areas from being misoperated or modified by malicious code. In this way, misoperation or malicious code modification can be prevented, and stable operation of the system can be ensured.

[0103] In some optional implementations, the memory management effect can be regularly evaluated, the management strategy can be adjusted according to the system operation status, and the memory management effect can be continuously improved.

[0104] When collecting data related to memory usage and performance, you can compare and analyze the data before optimization. Based on the evaluation results, adjust the optimization strategy, including reconfiguring the memory allocation strategy, adjusting the cache size, or improving the memory protection mechanism, to ensure that memory management optimization measures can continuously and effectively improve system performance.

[0105] The embodiment of the present invention also provides a method for memory management. Figure 2 A schematic diagram of another method for memory management provided by an embodiment of the present invention is shown. Figure 2 As shown, another method for memory management provided by an embodiment of the present invention includes:

[0106] Step S201, analyzing the memory usage of the current system based on the memstat tool.

[0107] Step S202: Perform training and prediction based on the random forest basic model, the gradient boosting machine basic simulation, and the deep neural network basic model to obtain a first prediction result corresponding to the basic model type.

[0108] Step S203: Based on the stacking integration method, multiple first prediction results are input into the meta-model for prediction to obtain a second prediction result.

[0109] Step S204: based on the second prediction result, combined with system characteristics and application requirements, propose a memory allocation strategy and cache amount.

[0110] Step S205, based on the second prediction result, combined with system characteristics and application requirements, the memory usage is monitored in real time through the memory monitoring module, and the memory is regularly sorted.

[0111] Step S206, based on the second prediction result, combined with system characteristics and application requirements, memory reservation is performed to optimize the memory access mode and reduce unnecessary memory access.

[0112] Step S207, based on the second prediction result, combined with system characteristics and application requirements, memory protection is performed on the preset memory area to prevent misoperation or malicious code modification.

[0113] Step S208: regularly evaluate the memory management effect.

[0114] In this embodiment, a device for memory management is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0115] This embodiment provides a device for memory management. Figure 3 A schematic diagram of the structure of a device for memory management provided by an embodiment of the present invention is shown. Figure 3 As shown, including:

[0116] The vector acquisition module 301 is used to acquire memory usage information and generate a memory feature vector based on the memory usage information.

[0117] The first prediction module 302 is used to input the memory feature vector into multiple basic models in the stacking model to obtain multiple first prediction results, wherein the first prediction results include possible reasons why the system performance is limited due to memory usage.

[0118] The second prediction module 303 is used to input the multiple first prediction results into the meta-model in the stacking model, and perform secondary prediction on the multiple first prediction results based on the meta-model to obtain a second prediction result.

[0119] The memory management module 304 is used to perform memory management based on the second prediction result.

[0120] In some optional implementations, the first prediction module 302 includes:

[0121] The first unit of the first prediction module is used to create a memory usage data set based on memory usage information; perform random sampling based on the memory usage data set to obtain multiple memory usage data subsets; build a decision tree model based on the memory usage data subsets; input the memory feature vector into each decision tree model for integration to generate a first prediction result.

[0122] In some optional implementations, the first prediction module 302 further includes:

[0123] The second unit of the first prediction module is used to initialize the predictor; in each iterative process, based on the predictor, the residual of each training sample is calculated; based on the residual, the weak learner is trained; based on the learning rate, the weak learner is added to the predictor; the aforementioned residual calculation, weak learner training and weak learner addition steps are repeated until the predictor training stop condition is met to obtain a prediction model; the memory feature vector is input into the prediction model to generate a first prediction result.

[0124] In some optional implementations, the first prediction module 302 further includes:

[0125] The third unit of the first prediction module is used to initialize the deep neural network structure; input the training data into the deep neural network for forward calculation, pass information layer by layer through the deep neural network to obtain the network output; calculate the difference between the true value and the network output based on the loss function; calculate the gradient of the loss function with respect to the network parameters, and reversely update the network parameters based on the gradient to minimize the loss function; repeat the steps of forward calculation, difference calculation and loss function minimization until the deep neural network training stopping condition is met to obtain the deep neural network model; input the memory feature vector into the deep neural network model to generate the first prediction result.

[0126] In some optional implementations, the second prediction module 303 includes:

[0127] The first unit of the second prediction module is used to input the multiple first prediction results into the meta-model in the stacking model, and perform weighted averaging on the multiple first prediction results based on the meta-model to generate a second prediction result.

[0128] In some optional implementations, the memory management module 304 includes:

[0129] The first unit of the memory management module is used to limit system performance if the second prediction result characterizes memory allocation and release frequency; allocate a preset memory pool and divide the preset memory pool into multiple first memory blocks; when allocating memory to a target program, allocate the first memory block to the target program; when the target program releases memory, release the first memory block to the preset memory pool.

[0130] In some optional implementations, the memory management module 304 further includes:

[0131] The second unit of the memory management module is used to move the data in the memory to a continuous memory area if the second prediction result indicates that the memory fragmentation rate limits the system performance.

[0132] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0133] The device for memory management in this embodiment is presented in the form of a functional unit, where the unit refers to an application specific integrated circuit (ASIC) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0134] The embodiment of the present invention also provides a computer device having the above Figure 3 The device for memory management is shown.

[0135] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a graphical user interface on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0136] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0137] The aforementioned memory 20 stores instructions executable by at least one processor 10, so that the aforementioned at least one processor 10 executes the method shown in the above embodiment.

[0138] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0139] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0140] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0141] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (such as a light emitting diode) and a tactile feedback device (such as a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0142] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0143] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0144] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for memory management, characterized in that The method comprises: Acquire memory usage information, and generate a memory feature vector based on the memory usage information; Inputting the memory feature vector into multiple basic models in the stacking model to obtain multiple first prediction results, wherein the first prediction results include possible reasons why the system performance is limited due to memory usage; Inputting the plurality of first prediction results into a meta-model in the stacking model, and performing secondary prediction on the plurality of first prediction results based on the meta-model to obtain a second prediction result; Memory management is performed based on the second prediction result.

2. The method according to claim 1, characterized in that The step of inputting the memory feature vector into a plurality of basic models in a stacked model to obtain a plurality of first prediction results includes: Based on the memory usage information, create a memory usage data set; Performing random sampling based on the memory usage data set to obtain multiple memory usage data subsets; Building a decision tree model based on the memory usage data subset; The memory feature vector is input into each decision tree model for integration to generate the first prediction result.

3. The method according to claim 1 or 2, characterized in that: The step of inputting the memory feature vector into a plurality of basic models in a stacked model to obtain a plurality of first prediction results further includes: Initialize the predictor; In each iteration, based on the predictor, the residual of each training sample is calculated; Based on the residual, training a weak learner; adding the weak learner to the predictor based on a learning rate; Repeat the aforementioned residual calculation, weak learner training and weak learner addition steps until the predictor training stop condition is met to obtain a prediction model; The memory feature vector is input into the prediction model to generate the first prediction result.

4. The method according to claim 3, characterized in that The step of inputting the memory feature vector into a plurality of basic models in a stacked model to obtain a plurality of first prediction results further includes: Initialize the deep neural network structure; Input the training data into the deep neural network for forward calculation, pass information layer by layer through the deep neural network, and obtain the network output; Calculate the difference between the true value and the network output based on the loss function; Calculate the gradient of the loss function with respect to the network parameters, and reversely update the network parameters based on the gradient to minimize the loss function; Repeat the steps of forward calculation, difference calculation and loss function minimization until the deep neural network training stop condition is met to obtain a deep neural network model; The memory feature vector is input into the deep neural network model to generate the first prediction result.

5. The method according to claim 1, characterized in that The step of inputting the plurality of first prediction results into a meta-model in the stacking model, and performing secondary prediction on the plurality of first prediction results based on the meta-model to obtain a second prediction result includes: The plurality of first prediction results are input into a meta-model in the stacking model, and a weighted average is performed on the plurality of first prediction results based on the meta-model to generate the second prediction result.

6. The method according to claim 1, characterized in that The performing memory management based on the second prediction result includes: If the second prediction result indicates that the memory allocation and release frequency limits the system performance; Allocating a preset memory pool, and dividing the preset memory pool into a plurality of first memory blocks; In case of allocating memory to a target program, allocating the first memory block to the target program; When the target program releases memory, the first memory block is released to the preset memory pool.

7. The method according to claim 1, characterized in that The performing memory management based on the second prediction result further includes: If the second prediction result indicates that the memory fragmentation rate limits system performance; Move data in memory to a contiguous memory area.

8. A device for memory management, characterized in that The device comprises: A vector acquisition module, used to acquire memory usage information and generate a memory feature vector based on the memory usage information; A first prediction module, configured to input the memory feature vector into a plurality of basic models in a stacking model to obtain a plurality of first prediction results, wherein the first prediction results include possible reasons for limiting system performance due to memory usage; A second prediction module, configured to input the plurality of first prediction results into a meta-model in the stacking model, and perform secondary prediction on the plurality of first prediction results based on the meta-model to obtain a second prediction result; A memory management module is used to perform memory management based on the second prediction result.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for memory management according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for memory management according to any one of claims 1 to 7.

Images