Data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a target node to be counted; scanning the process container every other preset time interval to obtain a first scanning result of the process container; determining a second scanning result of the process container based on the first scanning result, and obtaining the total cold memory page data volume of the target node and the total memory page data volume of the target node; and obtaining a cold memory page proportion of the target node based on the total cold memory page data volume of the target node and the total memory page data volume of the target node, and determining a memory state of the target node. A process container of each service unit in a target node is detected, and a scanning command is issued regularly. After the scanning result is returned, the proportion of the cold memory pages in all the memory pages of the container level is calculated based on the scanning result, and real-time detection of the memory pages of the container level is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a method, device, electronic device, and storage medium for cold page ratio statistics. Background Art

[0002] Memory is a crucial component of computers, and the efficiency of a computer system is closely linked to it. More memory means higher efficiency, while less memory means lower efficiency. Therefore, it's crucial to monitor computer system memory in real time and clean it up promptly to ensure efficient operation.

[0003] In existing technology, Idle Page Tracking (Idle Page Tracking) technology is usually used to manage the memory of a computer system. This technology is used to detect and identify memory pages in each process in the system, and reclaim the corresponding memory pages or write their corresponding contents back to disk when conditions are met.

[0004] However, the above solution can only be used to detect memory pages at the process level. In some special scenarios (such as k8s (Kubernetes, an open source platform for managing containers) clusters, each memory page exists at the Cgroup (container) level. The existing technology does not support detection at the Cgroup level. Therefore, a new solution is needed to realize the detection of memory pages at the Cgroup level. Summary of the Invention

[0005] The purpose of this application is to solve at least one of the above technical deficiencies. The technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a data processing method, comprising:

[0007] Obtaining a target node to be counted; wherein the target node includes at least one service unit, and each service unit includes at least one process container for storing each process;

[0008] For each process container, the process container is scanned at a preset time interval to obtain a first scan result of the process container at the current scan time; the first scan result at the current scan time includes memory page data volumes corresponding to at least two heat levels; the memory pages correspond to the processes in the process container; the memory page data volumes include file page data volumes and anonymous page data volumes;

[0009] Determine the heat level with the lowest heat level among the at least two heat levels as the cold heat level, and determine a second scan result of the process container based on the first scan result; wherein the second scan result includes the amount of cold memory page data and the total amount of memory page data in the process container;

[0010] Obtaining the total cold memory page data volume and the total memory page data volume of the target node based on the second scan results of each process container;

[0011] A cold memory page ratio of the target node is obtained based on the total cold memory page data volume of the target node and the total memory page data volume of the target node, and a memory state of the target node at a current moment is determined based on the cold memory page ratio.

[0012] In an optional embodiment of the present application, scanning the process container to obtain a first scanning result of the process container at the current scanning time specifically includes:

[0013] Obtaining the first scanning result corresponding to the last scanning moment;

[0014] Scan the process container to obtain the access status of each memory page in the process container between the last scan time and the current scan time;

[0015] The first scanning result corresponding to the previous scanning moment is updated based on the access status of each memory page to obtain the first scanning result at the current scanning moment.

[0016] In an optional embodiment of the present application, updating the first scan result corresponding to the previous scan time based on the access status of each memory page to obtain the first scan result at the current scan time specifically includes:

[0017] Creating a first heat level in the first scan result at the previous scan time, and setting the first heat level to be the heat level with the highest heat in the first scan result at the previous scan time;

[0018] For each scanned memory page, if the access status of the memory page between the last scan time and the current scan time is not accessed, and the heat level of the memory page is the cold heat level, the memory page will be recycled;

[0019] If the access status of the memory page between the last scan time and the current scan time is accessed, the heat level of the memory page is adjusted to the first heat level;

[0020] When all memory pages in the cold and hot levels are recycled or their corresponding hot levels are adjusted, the cold and hot levels are deleted, and a first scanning result at the current moment is generated based on the remaining hot levels.

[0021] In an optional embodiment of the present application, the method further specifically includes:

[0022] If the memory page has not been accessed between the last scan time and the current scan time and the original heat level of the memory page is the cold heat level, and the memory page cannot be reclaimed, the heat level of the memory page is adjusted to the second heat level; wherein the second heat level is the heat level with the lowest heat except the cold heat level in the first scan result of the last scan time.

[0023] In an optional embodiment of the present application, if there are memory pages that have not been scanned;

[0024] After deleting the heat level with the lowest heat, the method further specifically includes:

[0025] Adjust the heat level of the memory page to the heat level with the lowest heat at the current moment.

[0026] In an optional embodiment of the present application, determining a second scan result of the process container based on the first scan result specifically includes:

[0027] The anonymous page data volume corresponding to the hot and cold levels is used as the cold anonymous page data volume in the process container, and the file page data volume of the hot and cold levels is used as the cold file page data volume in the process container;

[0028] The sum of the anonymous page data volumes of each heat level in the first scan result at the current scan moment is taken as the total anonymous page data volume in the process container, and the sum of the file page data volumes of each heat level in the first scan result is taken as the total file page data volume in the process container.

[0029] In an optional embodiment of the present application, the method further specifically includes:

[0030] For each scanning moment, the hot memory page data volume of the target node at the scanning moment is obtained based on the total file page data volume of the target node at the scanning moment, the total anonymous page data volume of the target node at the scanning moment, the total cold anonymous page data volume of the target node at the scanning moment, and the total cold file page data volume of the target node at the scanning moment;

[0031] A memory hot-cold portrait of the target node is generated based on the total memory page data volume, the total cold anonymous page data volume, the total cold file page data volume, and the hot memory page data volume of the target node at at least two scanning moments. The memory hot-cold portrait represents the changing trend of the memory status of the target node over time.

[0032] In an optional embodiment of the present application, creating a first heat level in a first scan result at a previous scan time specifically includes:

[0033] Creating a first heat level at a first predetermined position in a first scan result at a previous scan moment;

[0034] The cold heat levels are deleted, and the first scan result at the current moment is generated based on the remaining heat levels, specifically including:

[0035] Deleting the heat level at the second predetermined position and updating the remaining heat levels to obtain the first scanning result at the current moment;

[0036] The remaining heat levels are updated to obtain the first scan result at the current moment, specifically including:

[0037] The heat levels are sorted based on their heat levels to obtain a first scanning result; wherein the cold heat level is located at a second predetermined position in the first scanning result.

[0038] In a second aspect, an embodiment of the present application provides a data processing device, including:

[0039] A target node acquisition module is used to acquire a target node to be counted; wherein the target node includes at least one service unit, and each service unit includes at least one process container for storing each process;

[0040] A first scanning result acquisition module is configured to scan each process container at a preset time interval to obtain a first scanning result of the process container at the current scanning moment; the first scanning result at the current scanning moment includes memory page data volumes corresponding to at least two heat levels; the memory pages correspond to the processes in the process container; the memory page data volumes include file page data volumes and anonymous page data volumes;

[0041] A second scanning result acquisition module is configured to determine the least hot level of the at least two hot levels as the cold hot level, and determine a second scanning result of the process container based on the first scanning result; wherein the second scanning result includes the amount of cold memory page data and the total amount of memory page data in the process container;

[0042] A data volume statistics module is used to obtain the total cold memory page data volume and the total memory page data volume of the target node based on the second scan result of each process container;

[0043] The memory status acquisition module is used to obtain the cold memory page ratio of the target node based on the total cold memory page data volume of the target node and the total memory page data volume of the target node, and determine the memory status of the target node at the current moment based on the cold memory page ratio.

[0044] In an optional embodiment of the present application, the first scanning result acquisition module is specifically configured to:

[0045] Obtaining the first scanning result corresponding to the last scanning moment;

[0046] Scan the process container to obtain the access status of each memory page in the process container between the last scan time and the current scan time;

[0047] The first scanning result corresponding to the previous scanning moment is updated based on the access status of each memory page to obtain the first scanning result at the current scanning moment.

[0048] In an optional embodiment of the present application, the first scanning result acquisition module is further configured to:

[0049] Creating a first heat level in the first scan result at the previous scan time, and setting the first heat level to be the heat level with the highest heat in the first scan result at the previous scan time;

[0050] For each scanned memory page, if the access status of the memory page between the last scan time and the current scan time is not accessed, and the heat level of the memory page is the cold heat level, the memory page will be recycled;

[0051] If the access status of the memory page between the last scan time and the current scan time is accessed, the heat level of the memory page is adjusted to the first heat level;

[0052] When all memory pages in the cold and hot levels are recycled or their corresponding hot levels are adjusted, the cold and hot levels are deleted, and a first scanning result at the current moment is generated based on the remaining hot levels.

[0053] In an optional embodiment of the present application, the device further includes a heat level adjustment module, specifically configured to:

[0054] If the memory page has not been accessed between the last scan time and the current scan time and the original heat level of the memory page is the cold heat level, and the memory page cannot be reclaimed, the heat level of the memory page is adjusted to the second heat level; wherein the second heat level is the heat level with the lowest heat except the cold heat level in the first scan result of the last scan time.

[0055] In an optional embodiment of the present application, if there are memory pages that have not been scanned;

[0056] After deleting the heat level with the lowest heat, the heat level adjustment module is further used to:

[0057] Adjust the heat level of the memory page to the heat level with the lowest heat at the current moment.

[0058] In an optional embodiment of the present application, the second scanning result acquisition module is specifically configured to:

[0059] The anonymous page data volume corresponding to the hot and cold levels is used as the cold anonymous page data volume in the process container, and the file page data volume of the hot and cold levels is used as the cold file page data volume in the process container;

[0060] The sum of the anonymous page data volumes of each heat level in the first scan result at the current scan moment is taken as the total anonymous page data volume in the process container, and the sum of the file page data volumes of each heat level in the first scan result is taken as the total file page data volume in the process container.

[0061] In an optional embodiment of the present application, the device further includes a memory image generation module, specifically configured to:

[0062] For each scanning moment, the hot memory page data volume of the target node at the scanning moment is obtained based on the total file page data volume of the target node at the scanning moment, the total anonymous page data volume of the target node at the scanning moment, the total cold anonymous page data volume of the target node at the scanning moment, and the total cold file page data volume of the target node at the scanning moment;

[0063] A memory hot-cold portrait of the target node is generated based on the total memory page data volume, the total cold anonymous page data volume, the total cold file page data volume, and the hot memory page data volume of the target node at at least two scanning moments. The memory hot-cold portrait represents the changing trend of the memory status of the target node over time.

[0064] In an optional embodiment of the present application, the first scanning result acquisition module may also be used to:

[0065] Creating a first heat level at a first predetermined position in a first scan result at a previous scan moment;

[0066] The first scanning result acquisition module can also be used to:

[0067] Deleting the heat level at the second predetermined position and updating the remaining heat levels to obtain the first scanning result at the current moment;

[0068] The first scanning result acquisition module can also be used to:

[0069] The heat levels are sorted based on their heat levels to obtain a first scanning result; wherein the cold heat level is located at a second predetermined position in the first scanning result.

[0070] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory;

[0071] The processor executes the computer program to implement the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect.

[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect is implemented.

[0073] In a fifth aspect, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that when executed by the computer device, the method provided in the embodiment of the first aspect or any alternative embodiment of the first aspect is implemented.

[0074] The beneficial effects of the technical solution provided by the embodiments of the present application are:

[0075] By detecting the process container (Cgroup) of each service unit in the target node and issuing scan commands at regular intervals, the system can calculate the proportion of cold memory pages among all memory pages (including file pages and anonymous pages) at the Cgroup level based on the scan results, thus achieving real-time detection of memory pages at the Cgroup level. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0077] Figure 1 A schematic diagram of an application environment for a data processing method provided in an example;

[0078] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0079] Figure 3 A schematic diagram of a process for deploying scripts in a virtualization scenario in an example of an embodiment of the present application;

[0080] Figure 4 This is a flowchart of deploying a script in a k8s cluster scenario in an example of an embodiment of the present application;

[0081] Figure 5 This is an example diagram of a hot and cold image stored in an example of an embodiment of the present application;

[0082] Figure 6 A structural block diagram of a data processing device provided in an embodiment of the present application;

[0083] Figure 7A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0085] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B". When describing multiple (two or more) items, if the relationship between the multiple items is not clearly defined, the multiple items may refer to one, multiple or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" can be implemented as parameter A including A1 or A2 or A3, and can also be implemented as parameter A including at least two of the three items A1, A2, and A3.

[0086] The data processing method of the present application can be implemented based on big data. Big data refers to a collection of data that cannot be captured, managed, and processed by conventional software tools within a certain time frame. It is a massive, high-growth, and diversified information asset that requires new processing models to have stronger decision-making power, insight discovery, and process optimization capabilities. With the advent of the cloud era, big data has also attracted more and more attention. Big data requires special technologies to effectively process large amounts of data within a tolerable time frame. Technologies applicable to big data include large-scale parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

[0087] In the embodiments of the present application, determining the current memory state of a target node based on the cold memory page ratio can be implemented using artificial intelligence. Artificial Intelligence (AI) refers to theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, machine learning / deep learning, autonomous driving, and smart transportation.

[0088] Machine Learning (ML) is a multidisciplinary interdisciplinary field that involves probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory, and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by example. In the specific embodiments of this application, any data related to the object, such as data involved in the object's use of the application program, is involved. When the embodiments of this application are applied to specific products or technologies, the object's permission or consent must be obtained, and the collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if any of the above-mentioned data related to the object is involved in the embodiments of this application, this data must be obtained with the object's authorization and consent and in compliance with the relevant laws, regulations, and standards of the relevant countries and regions.

[0089] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0090] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on, or combine with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0091] In order to better illustrate and understand the solutions provided by the embodiments of the present application, some relevant technical terms involved in the embodiments of the present application are first introduced:

[0092] File pages: These are the memory pages in memory that correspond to the file contents on disk. When a file is read or mapped into memory, the operating system loads the file contents into the file pages in memory. These file pages can usually be swapped between memory and disk. That is, when memory is insufficient, the operating system can write unused file pages back to disk to free up memory space. At the same time, file pages can be shared by multiple processes, so that only one copy of the same file needs to be kept in memory, thereby improving memory utilization.

[0093] Anonymous pages: refers to memory pages in memory that do not directly correspond to the contents of files on disk. These anonymous pages are usually used to store data such as the process heap, stack, and global variables. Unlike file pages, the contents of anonymous pages cannot directly correspond to files on disk. Therefore, when memory is insufficient, the operating system needs to write the contents of anonymous pages to the swap partition (SwapPartition) or swap file (SwapFile) to free up memory space.

[0094] The data processing method provided in the embodiment of the present application can be executed by any computer device, and optionally, can be executed by a server, wherein the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0095] Figure 1 Schematic diagram of an application environment of the speech processing method provided in an embodiment of the present application, wherein the application environment may include a terminal 101 and a target node (server to be detected) 102.

[0096] like Figure 1As shown, when the cold memory page ratio of the target node needs to be calculated, the terminal 101 will send a scanning instruction to the target node 102. After the terminal 101 scans the target node 102, it obtains the first scanning result of the target node 102 at the current moment. Then, the terminal 101 determines the second scanning result corresponding to the target node 102 at the current moment based on the first scanning result, and calculates the cold memory page ratio of the target node 102 at the current moment based on the second scanning result. Finally, the memory status of the target node 102 at the current moment is determined based on the cold memory page ratio.

[0097] Those skilled in the art will appreciate that a server may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal may be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a laptop computer, a digital broadcast receiver, a MID (Mobile Internet Devices), a PDA (Personal Digital Assistant), a desktop computer, a smart home appliance, a vehicle-mounted terminal (such as a vehicle-mounted navigation terminal, a vehicle-mounted computer, etc.), a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected via wired or wireless communication, but are not limited thereto. The embodiments of the present invention may be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. The specific requirements may also be determined based on the actual application scenario and are not limited here.

[0098] The terminal may be part of a server, used to initiate a scan command to a target node and perform a scan, and may determine the memory status of the target node based on the scan results. The target node may be a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device (e.g., smart speaker), wearable electronic device (e.g., smart watch), vehicle-mounted terminal, smart home appliance (e.g., smart TV), AR (Augmented Reality, augmented reality) / VR (Virtual Reality, virtual display) device, etc. In an embodiment of the present application, it may also be a k8s cluster or a partially virtualized scenario, but is not limited thereto. The push terminal and the server may be directly or indirectly connected via wired or wireless communication, and this application is not limited thereto.

[0099] Figure 2A flow chart of a data processing method is provided for an embodiment of the present application. The execution subject of the method can be a terminal (such as a computer, a mobile phone, etc.). Figure 1 As shown, the method may include:

[0100] Step S101 : obtaining a target node to be counted; wherein the target node includes at least one service unit, and each service unit includes at least one process container for storing each process.

[0101] The target node can be a smart terminal or a server. In the embodiment of the present application, the target node is deployed in a virtualized scenario or a k8s cluster scenario. For any target node, the target node contains multiple Pods (i.e., service units for deploying various services), each of which contains multiple process containers, each of which stores the processes running each service.

[0102] Step S102: For each process container, the process container is scanned at a preset time interval to obtain a first scanning result of the process container at the current scanning moment; the first scanning result at the current scanning moment includes memory page data volumes corresponding to at least two heat levels; the memory pages correspond to each process in the process container; the memory page data volume includes file page data volume and anonymous page data volume.

[0103] Among them, the preset time interval can be freely set according to actual needs. The heat level can also be called a generation (Gen), which is used to characterize the heat of each memory page. Generally speaking, the closer the last access time is to the current time, the higher the corresponding heat. Memory pages correspond to each process in the process container and can be divided into file pages and anonymous pages. The amount of memory page data represents the total size of the memory pages belonging to this heat level. The first scan result consists of multiple heat levels, and each heat level can be a line. For example, it can be shown in Table 1 below.

[0104]

[0105] Table 1

[0106] In Table 1, the first row represents the number of the target node, and each row except the first row represents a heat level. The first column represents the number of the heat level. In the embodiment of the present application, generally speaking, the heat level with a smaller number corresponds to a smaller heat level. The second column represents the time (ms, milliseconds) from the time the heat level is generated to the current time. The third column represents the total data volume of the anonymous pages belonging to the heat level. The fourth column represents the total data volume of the file pages belonging to the heat level.

[0107] Specifically, in an embodiment of the present application, each process container needs to be scanned periodically. After each scan is completed, a first scan result at the current scan moment can be output through a first preset interface.

[0108] Step S103: Determine the heat level with the lowest heat level among the at least two heat levels as the cold heat level, and determine a second scan result of the process container based on the first scan result; wherein the second scan result includes the amount of cold memory page data and the total amount of memory page data in the process container.

[0109] The cold heat level is the heat level with the lowest heat level in the first scan result. Generally speaking, the heat level with the lowest number in the first scan result is the cold heat level. The second scan result can include four data: cold anonymous page data volume, cold file page data volume, total anonymous page data volume, and total file page data volume. The cold anonymous page data volume and cold file page data volume can be directly read from the cold heat level. The total anonymous page data volume and total file page data volume are obtained by adding the anonymous page data volume of each heat level or adding the file page data volume of each heat level.

[0110] Specifically, after the first scan result is output, the heat level with the lowest heat in the first scan result is used as the cold heat level, and then the second scan result is determined based on the amount of memory page data at each heat level.

[0111] Step S104: Obtain the total cold memory page data volume and the total memory page data volume of the target node based on the second scan results of each process container.

[0112] Specifically, since the target node is composed of multiple service units, and each service unit is composed of multiple process containers, the memory page data volume and cold memory page data volume of each process container can be added together to obtain the total memory page data volume and the total cold memory page data volume of the target node.

[0113] Step S105: obtaining the cold memory page ratio of the target node based on the total cold memory page data volume and the total memory page data volume of the target node, and determining the memory status of the target node at the current moment based on the cold memory page ratio.

[0114] Specifically, the total amount of cold memory page data of the target node is divided by the total amount of memory page data of the target node to obtain the cold memory page ratio of the target node. Then, based on the cold memory page ratio, the current memory status of the target node can be analyzed, and the target node can be adjusted based on the memory status.

[0115] It should be noted that the scanning process requires a script to perform scheduled scans. The script will execute the following logic: All pods (i.e., service units) on the target node are scanned sequentially, and a full page table scan is performed on the current service unit. This full page table scan is performed by calling the second preset interface (memory.full_scan). Pages that were not scanned between the previous scan time and the current scan time remain on the oldest Gen lru list (i.e., the lowest heat level). The amount of memory page data belonging to the cold heat level is output through the meomroy.lru_gen interface (the first preset interface mentioned above, which is used to output scan results). This is then divided by the total number of memory pages in the process container to obtain the ratio of the number of cold memory pages on the target node. The results for this pod are output to the corresponding log, and the next pod is scanned. After all pods on the node have been scanned, the node enters a "sleep" state at a preset specified time.

[0116] It should be noted that in the embodiments of the present application, for the convenience of deployment, the written scripts can be "packaged" to a certain extent for different application scenarios. For example, in a virtualization scenario (such as deployment on a virtual host or virtual child), the script can be packaged into a systemd (user-level process software under Linux (a system name)) service, while in a k8s scenario, the script can be packaged into a daemonset (daemon process set). The specific deployment process can refer to the following steps:

[0117] Figure 3 Schematic diagram of a script deployment solution in a virtualized scenario provided in an embodiment of the present application. Figure 3 As shown, by running a script packaged as systemd (i.e., Mglru-hot in the figure; if the packaged script exits due to an abnormal reason or after the node restarts, it can be quickly pulled up to continue the detection task), a scan command is regularly issued to the target node. By calling the second preset interface (i.e., the Per-cgroup scan interface in the figure, which includes the first preset interface and the second preset interface), a full scan (i.e., executing Mglru scan in the figure) is performed, and finally the amount of cold memory page data of the target node is returned through the first preset interface.

[0118] like Figure 4 As shown, Figure 4 This is a flow chart of a script deployment solution in a k8s cluster provided in an embodiment of the present application. Figure 4As shown, since there are multiple nodes in the k8s cluster, each node must be classified first. In the embodiment of the present application, affinity matching conditions can be configured in the daemonset yaml (a configuration file), and corresponding affinity labels can be added to the sampling nodes in the k8s cluster. In this way, the script will only be deployed in the sampling nodes with affinity labels, and a detection pod can be deployed on each sampling node to run the detection script in the detection pod. The advantage of this is that when the pods on these sampling nodes are scheduled, the detection script can export the memory profile corresponding to the pod.

[0119] For the target node (i.e., the sampling node), a script packaged as a daemontest (i.e., ds) (i.e., Mglru-hot in the figure) is run to periodically initiate a scan command to the target node. A full scan (i.e., an Mglru scan is executed in the figure) is performed by calling the second preset interface (i.e., the Per-cgroup scan interface in the figure, which includes the first and second preset interfaces). Finally, the amount of cold memory page data on the target node is returned through the first preset interface. For non-sampling nodes, no scan script is configured, so no operation is performed.

[0120] The solution provided by this application detects the process container (Cgroup) of each service unit in the target node and issues scan commands at regular intervals. When the scan results are returned, the proportion of cold memory pages (including file pages and anonymous pages) at the Cgroup level can be calculated based on the scan results, thus achieving real-time detection of memory pages at the Cgroup level.

[0121] In an optional embodiment of the present application, scanning the process container to obtain a first scanning result of the process container at the current scanning time specifically includes:

[0122] Obtaining the first scanning result corresponding to the last scanning moment;

[0123] Scan the process container to obtain the access status of each memory page in the process container between the last scan time and the current scan time;

[0124] The first scanning result corresponding to the previous scanning moment is updated based on the access status of each memory page to obtain the first scanning result at the current scanning moment.

[0125] The last scan time is the last scan time closest to the current time. The access status of the memory page can be divided into whether the memory page is accessed or not accessed between the last scan time and the current time.

[0126] Specifically, in an embodiment of the present application, the basic format of the first scan result at each moment is the same (as shown in Table 1 above). After each scan, only the data in the first scan result needs to be updated, and the basis for each update is based on the access status of each memory page from the previous scan moment to the current moment. If the memory page is accessed, then the memory page can be updated to a higher heat level. If the memory page is not accessed, then the memory page will be transferred to a lower heat level accordingly.

[0127] In an optional embodiment of the present application, updating the first scan result corresponding to the previous scan time based on the access status of each memory page to obtain the first scan result at the current scan time specifically includes:

[0128] Creating a first heat level in the first scan result at the previous scan time, and setting the first heat level to be the heat level with the highest heat in the first scan result at the previous scan time;

[0129] For each scanned memory page, if the access status of the memory page between the last scan time and the current scan time is not accessed, and the heat level of the memory page is the cold heat level, the memory page will be recycled;

[0130] If the access status of the memory page between the last scan time and the current scan time is accessed, the heat level of the memory page is adjusted to the first heat level;

[0131] When all memory pages in the cold and hot levels are recycled or their corresponding hot levels are adjusted, the cold and hot levels are deleted, and a first scanning result at the current moment is generated based on the remaining hot levels.

[0132] The first heat level is a heat level newly added based on the first scan result corresponding to the previous scan time, and the heat level is the highest relative to the heat levels in the first scan result corresponding to the previous scan time.

[0133] Specifically, after the scan is completed, if a certain memory page has not been accessed between the last scan time and the current scan time, it means that the heat of the memory page is low during this period, and the heat level of the memory page can be adjusted to a lower heat level. However, if the memory page itself is at a cold heat level and cannot be adjusted to a lower heat level, then the memory page is directly recycled. When all memory pages have been recycled or recovered, it means that this adjustment has been completed. At this time, the memory pages belonging to the cold heat level in the first scan result of the last scan time have all been adjusted or recycled. Therefore, the cold heat level can be directly deleted, and the heat level of each memory page at the current moment is counted and the first scan result of the last scan time is updated to obtain the first scan result at the current moment.

[0134] In an optional embodiment of the present application, the method further specifically includes:

[0135] If the memory page has not been accessed between the last scan time and the current scan time and the original heat level of the memory page is the cold heat level, and the memory page cannot be reclaimed, the heat level of the memory page is adjusted to the second heat level; wherein the second heat level is the heat level with the lowest heat except the cold heat level in the first scan result of the last scan time.

[0136] Specifically, when a memory page needs to be recycled but cannot be recycled, the heat level of the memory page needs to be adjusted. Specifically, since the memory page actually still belongs to the cold heat level, and the cold heat level at the last scanning moment needs to be deleted at the current moment, the heat level of the memory page can be adjusted to the second heat level at the last scanning moment, that is, the coldest heat level except the cold heat level.

[0137] In an optional embodiment of the present application, if there are memory pages that have not been scanned;

[0138] After deleting the heat level with the lowest heat, the method further specifically includes:

[0139] Adjust the heat level of the memory page to the heat level with the lowest heat at the current moment.

[0140] Specifically, during the scanning process, some memory pages may not be scanned. In this case, the heat level of these memory pages can be adjusted to the current cold heat level (ie, the second heat level corresponding to the previous scanning moment).

[0141] In an optional embodiment of the present application, determining a second scan result of the process container based on the first scan result specifically includes:

[0142] The anonymous page data volume corresponding to the hot and cold levels is used as the cold anonymous page data volume in the process container, and the file page data volume of the hot and cold levels is used as the cold file page data volume in the process container;

[0143] The sum of the anonymous page data volumes of each heat level in the first scan result at the current scan moment is taken as the total anonymous page data volume in the process container, and the sum of the file page data volumes of each heat level in the first scan result is taken as the total file page data volume in the process container.

[0144] Specifically, in an embodiment of the present application, memory pages at the cold and hot levels can generally be regarded as cold memory that needs to be counted. Therefore, the anonymous page data volume and file page data volume corresponding to the cold and hot levels in the first scan result can represent the cold anonymous page data volume and cold file page data volume of the process container. At the same time, for subsequent calculations, it is also necessary to count the total memory page data volume. Specifically, the anonymous pages of each hot level can be added to obtain the total anonymous page data volume of the process container, and the file pages of each hot level can be added to obtain the total file page data volume of the process container. Finally, the cold anonymous page data volume, cold file page data volume, total anonymous page data volume and total file page data volume are integrated to obtain the second scan result.

[0145] In an optional embodiment of the present application, the method further specifically includes:

[0146] For each scanning moment, the hot memory page data volume of the target node at the scanning moment is obtained based on the total file page data volume of the target node at the scanning moment, the total anonymous page data volume of the target node at the scanning moment, the total cold anonymous page data volume of the target node at the scanning moment, and the total cold file page data volume of the target node at the scanning moment;

[0147] A memory hot-cold portrait of the target node is generated based on the total memory page data volume, the total cold anonymous page data volume, the total cold file page data volume, and the hot memory page data volume of the target node at at least two scanning moments. The memory hot-cold portrait represents the changing trend of the memory status of the target node over time.

[0148] The hot memory page may be a memory page other than a cold memory page, or may be a memory page with the highest heat level in the first scan result corresponding to the current scan time, which is not limited in this embodiment of the present application. The hot and cold memory portrait may be drawn using a Python script.

[0149] Specifically, in an embodiment of the present application, if the hot memory page is a memory page other than a cold memory page, the total memory page data volume of the target node (that is, the sum of the total file page data volume of the target node at the scanning time and the total anonymous page data volume of the target node at the scanning time) can be subtracted from the total cold memory page data volume of the target node (that is, the sum of the total cold anonymous page data volume of the target node at the scanning time and the total cold file page data volume of the target node at the scanning time) to obtain the hot memory page; if the hot memory page is the memory page with the highest heat level in the first scanning result corresponding to the current scanning time, the second heat level in the first scanning result can be directly read to obtain it.

[0150] For memory hot and cold profiles, you can first select a certain number of scan time samples, and then use Python scripts to draw memory hot and cold profiles based on the data corresponding to these scan time samples. Figure 5 For example, in Figure 5 In the figure, the horizontal axis represents the change of time, and the vertical axis represents the data volume of memory pages. The top curve in the figure (i.e., total memory used) represents the changing trend of the total memory page data volume over time. The second row of curves (i.e., cold file memory) represents the changing trend of the total cold file page data volume over time. The third row of curves (i.e., cold anon memory) represents the changing trend of the total cold anonymous page data volume over time. The fourth row of curves (i.e., hot memory) represents the changing trend of the hot memory page data volume over time.

[0151] In an optional embodiment of the present application, creating a first heat level in a first scan result at a previous scan time specifically includes:

[0152] Creating a first heat level at a first predetermined position in a first scan result at a previous scan moment;

[0153] The cold heat levels are deleted, and the first scan result at the current moment is generated based on the remaining heat levels, specifically including:

[0154] Deleting the heat level at the second predetermined position and updating the remaining heat levels to obtain the first scanning result at the current moment;

[0155] The remaining heat levels are updated to obtain the first scan result at the current moment, specifically including:

[0156] The heat levels are sorted based on their heat levels to obtain a first scanning result; wherein the cold heat level is located at a second predetermined position in the first scanning result.

[0157] The first predetermined position may be after the last row in the first scan result, and the second predetermined position may be the heat level in the first row in the first scan result.

[0158] Specifically, in the embodiment of the present application, the heat levels are generally arranged from small to large in order from top to bottom. Since the first heat level is the heat level with the highest heat, the first heat level is generally directly created at the first predetermined position (i.e., the last row in the first scan result). When it is necessary to delete the cold heat level, the heat level in the first row (i.e., at the second predetermined position) can be directly deleted to obtain the first scan result. And when the remaining heat levels are sorted and updated, the cold heat level at the current moment is set at the second predetermined position.

[0159] Figure 6 A structural block diagram of a data processing device provided in an embodiment of the present application is shown in FIG. Figure 6As shown, the data processing device 600 may include: a target node acquisition module 601, a first scan result acquisition module 602, a second scan result acquisition module 603, a data volume statistics module 604 and a memory status acquisition module 605, wherein:

[0160] The target node acquisition module 601 is used to acquire the target node to be counted; wherein the target node includes at least one service unit, and each service unit includes at least one process container for storing each process;

[0161] The first scanning result acquisition module 602 is configured to scan each process container at a preset time interval to obtain a first scanning result of the process container at the current scanning moment; the first scanning result at the current scanning moment includes memory page data volumes corresponding to at least two heat levels; the memory pages correspond to the processes in the process container; the memory page data volumes include file page data volumes and anonymous page data volumes;

[0162] The second scanning result acquisition module 603 is configured to determine the heat level with the lowest heat level among the at least two heat levels as the cold heat level, and determine a second scanning result of the process container based on the first scanning result; wherein the second scanning result includes the amount of cold memory page data and the total amount of memory page data in the process container;

[0163] The data volume statistics module 604 is used to obtain the total cold memory page data volume and the total memory page data volume of the target node based on the second scan result of each process container;

[0164] The memory status acquisition module 605 is used to obtain the cold memory page ratio of the target node based on the total cold memory page data volume and the total memory page data volume of the target node, and determine the memory status of the target node at the current moment based on the cold memory page ratio.

[0165] The solution provided by this application detects the process container (Cgroup) of each service unit in the target node and issues scan commands at regular intervals. When the scan results are returned, the proportion of cold memory pages (including file pages and anonymous pages) at the Cgroup level can be calculated based on the scan results, thus achieving real-time detection of memory pages at the Cgroup level.

[0166] In an optional embodiment of the present application, the first scanning result acquisition module is specifically configured to:

[0167] Obtaining the first scanning result corresponding to the last scanning moment;

[0168] Scan the process container to obtain the access status of each memory page in the process container between the last scan time and the current scan time;

[0169] The first scanning result corresponding to the previous scanning moment is updated based on the access status of each memory page to obtain the first scanning result at the current scanning moment.

[0170] In an optional embodiment of the present application, the first scanning result acquisition module is further configured to:

[0171] Creating a first heat level in the first scan result at the previous scan time, and setting the first heat level to be the heat level with the highest heat in the first scan result at the previous scan time;

[0172] For each scanned memory page, if the access status of the memory page between the last scan time and the current scan time is not accessed, and the heat level of the memory page is the cold heat level, the memory page will be recycled;

[0173] If the access status of the memory page between the last scan time and the current scan time is accessed, the heat level of the memory page is adjusted to the first heat level;

[0174] When all memory pages in the cold and hot levels are recycled or their corresponding hot levels are adjusted, the cold and hot levels are deleted, and a first scanning result at the current moment is generated based on the remaining hot levels.

[0175] In an optional embodiment of the present application, the device further includes a heat level adjustment module, specifically configured to:

[0176] If the memory page has not been accessed between the last scan time and the current scan time and the original heat level of the memory page is the cold heat level, and the memory page cannot be reclaimed, the heat level of the memory page is adjusted to the second heat level; wherein the second heat level is the heat level with the lowest heat except the cold heat level in the first scan result of the last scan time.

[0177] In an optional embodiment of the present application, if there are memory pages that have not been scanned;

[0178] After deleting the heat level with the lowest heat, the heat level adjustment module is further used to:

[0179] Adjust the heat level of the memory page to the heat level with the lowest heat at the current moment.

[0180] In an optional embodiment of the present application, the second scanning result acquisition module is specifically configured to:

[0181] The anonymous page data volume corresponding to the hot and cold levels is used as the cold anonymous page data volume in the process container, and the file page data volume of the hot and cold levels is used as the cold file page data volume in the process container;

[0182] The sum of the anonymous page data volumes of each heat level in the first scan result at the current scan moment is taken as the total anonymous page data volume in the process container, and the sum of the file page data volumes of each heat level in the first scan result is taken as the total file page data volume in the process container.

[0183] In an optional embodiment of the present application, the device further includes a memory image generation module, specifically configured to:

[0184] For each scanning moment, the hot memory page data volume of the target node at the scanning moment is obtained based on the total file page data volume of the target node at the scanning moment, the total anonymous page data volume of the target node at the scanning moment, the total cold anonymous page data volume of the target node at the scanning moment, and the total cold file page data volume of the target node at the scanning moment;

[0185] A memory hot-cold portrait of the target node is generated based on the total memory page data volume, the total cold anonymous page data volume, the total cold file page data volume, and the hot memory page data volume of the target node at at least two scanning moments. The memory hot-cold portrait represents the changing trend of the memory status of the target node over time.

[0186] In an optional embodiment of the present application, the first scanning result acquisition module may also be used to:

[0187] Creating a first heat level at a first predetermined position in a first scan result at a previous scan moment;

[0188] The first scanning result acquisition module can also be used to:

[0189] Deleting the heat level at the second predetermined position and updating the remaining heat levels to obtain the first scanning result at the current moment;

[0190] The first scanning result acquisition module can also be used to:

[0191] The heat levels are sorted based on their heat levels to obtain a first scanning result; wherein the cold heat level is located at a second predetermined position in the first scanning result.

[0192] Reference below Figure 7 , which shows an electronic device suitable for implementing the embodiments of the present application (for example, Figure 2 The electronic devices in the embodiments of the present application may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), wearable devices, and fixed terminals such as digital TVs and desktop computers. Figure 7The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0193] The electronic device includes: a memory and a processor, wherein the memory is used to store a program for executing the methods described in each of the above method embodiments; and the processor is configured to execute the program stored in the memory. The processor here may be referred to as the processing device 701 described below, and the memory may include at least one of the read-only memory (ROM) 702, random access memory (RAM) 703, and storage device 708 described below, as shown below:

[0194] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0195] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0196] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

[0197] It should be noted that the computer-readable storage medium mentioned above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0198] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0199] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0200] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0201] Obtaining a target node to be counted; wherein the target node includes at least one service unit, and each service unit includes at least one process container for storing each process;

[0202] For each process container, the process container is scanned at a preset time interval to obtain a first scan result of the process container at the current scan time; the first scan result at the current scan time includes memory page data volumes corresponding to at least two heat levels; the memory pages correspond to the processes in the process container; the memory page data volumes include file page data volumes and anonymous page data volumes;

[0203] Determine the heat level with the lowest heat level among the at least two heat levels as the cold heat level, and determine a second scan result of the process container based on the first scan result; wherein the second scan result includes the amount of cold memory page data and the total amount of memory page data in the process container;

[0204] Obtaining the total cold memory page data volume and the total memory page data volume of the target node based on the second scan results of each process container;

[0205] A cold memory page ratio of the target node is obtained based on the total cold memory page data volume of the target node and the total memory page data volume of the target node, and a memory state of the target node at a current moment is determined based on the cold memory page ratio.

[0206] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0207] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0208] The modules or units described in the embodiments of this application may be implemented in software or hardware. The name of a module or unit does not, in some cases, limit the unit itself. For example, the first constraint acquisition module may also be described as a "module for acquiring a first constraint."

[0209] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0210] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0211] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0212] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: include: Obtaining a target node to be counted; wherein the target node includes at least one service unit, and each service unit includes at least one process container for storing each process; For each process container, the process container is scanned at a preset time interval to obtain a first scan result of the process container at the current scan time; the first scan result at the current scan time includes memory page data volumes corresponding to at least two heat levels; the memory pages correspond to the processes in the process container; the memory page data volumes include file page data volumes and anonymous page data volumes; Determine the heat level with the lowest heat level among the at least two heat levels as the cold heat level, and determine a second scan result of the process container based on the first scan result; wherein the second scan result includes the amount of cold memory page data and the total amount of memory page data in the process container; Obtaining the total cold memory page data volume of the target node and the total memory page data volume of the target node based on the second scan result of each process container; The cold memory page ratio of the target node is obtained based on the total cold memory page data volume of the target node and the total memory page data volume of the target node, and the memory state of the target node at the current moment is determined based on the cold memory page ratio.

2. The method according to claim 1, wherein Scanning the process container to obtain a first scanning result of the process container at a current scanning time includes: Obtaining the first scanning result corresponding to the last scanning moment; Scan the process container to obtain access status of each memory page in the process container between the last scanning time and the current scanning time; The first scanning result corresponding to the previous scanning moment is updated based on the access status of each memory page to obtain the first scanning result at the current scanning moment.

3. The method according to claim 2, characterized in that The updating of the first scanning result corresponding to the previous scanning moment based on the access status of each memory page to obtain the first scanning result of the current scanning moment includes: Creating a first heat level in the first scan result at the previous scan time, and setting the first heat level as the heat level with the highest heat in the first scan result at the previous scan time; For each scanned memory page, if the access status of the memory page between the last scan time and the current scan time is not accessed, and the heat level of the memory page is the cold heat level, the memory page is recycled; If the access status of the memory page between the last scanning moment and the current scanning moment is accessed, adjusting the heat level of the memory page to the first heat level; When all memory pages in the cold and hot levels are recycled or their corresponding hot levels are adjusted, the cold and hot levels are deleted, and a first scanning result at the current moment is generated based on the remaining hot levels.

4. The method according to claim 3, wherein The method further comprises: If the memory page has not been accessed between the last scan time and the current scan time and the original heat level of the memory page is the cold heat level, and the memory page cannot be recycled, the heat level of the memory page is adjusted to a second heat level; wherein the second heat level is the heat level with the lowest heat except the cold heat level in the first scan result of the last scan time.

5. The method according to claim 3, characterized in that If there are memory pages that have not been scanned; After deleting the heat level with the lowest heat, the method further includes: The heat level of the memory page is adjusted to the heat level with the lowest heat at the current moment.

6. The method according to claim 1, wherein Determining a second scanning result of the process container based on the first scanning result includes: The anonymous page data volume corresponding to the cold / hot level is used as the cold anonymous page data volume in the process container, and the file page data volume of the cold / hot level is used as the cold file page data volume in the process container; The sum of the anonymous page data volumes of each heat level in the first scan result at the current scan moment is used as the total anonymous page data volume in the process container, and the sum of the file page data volumes of each heat level in the first scan result is used as the total file page data volume in the process container.

7. The method according to claim 1, wherein The method further comprises: For each scanning moment, obtaining the hot memory page data volume of the target node at the scanning moment based on the total file page data volume of the target node at the scanning moment, the total cold anonymous page data volume of the target node at the scanning moment, and the total cold file page data volume of the target node at the scanning moment; A memory hot and cold portrait of the target node is generated based on the total memory page data volume of the target node, the total cold anonymous page data volume of the target node, the total cold file page data volume of the target node, and the hot memory page data volume at at least two scanning moments; wherein the memory hot and cold portrait represents the changing trend of the memory status of the target node over time.

8. The method according to claim 3, characterized in that The creating a first heat level in the first scan result at the previous scan time includes: Creating the first heat level at a first predetermined position in the first scan result at the last scan moment; The step of deleting the cold / hot level and generating a first scanning result at the current moment based on the remaining hot levels includes: Deleting the heat level at the second predetermined position and updating the remaining heat levels to obtain the first scanning result at the current moment; The updating of the remaining heat levels to obtain the first scanning result at the current moment specifically includes: The heat levels are sorted based on their heat levels to obtain the first scanning result; wherein the cold heat level is located at a second predetermined position in the first scanning result.

9. A cold page ratio statistics device, characterized in that: include: A target node acquisition module is used to acquire a target node to be counted; wherein the target node includes at least one service unit, and each service unit includes at least one process container for storing each process; A first scanning result acquisition module is configured to scan each process container at a preset time interval to obtain a first scanning result of the process container at a current scanning moment; the first scanning result at the current scanning moment includes memory page data volumes corresponding to at least two heat levels, the memory pages corresponding to the processes in the process container; the memory page data volumes include file page data volumes and anonymous page data volumes; a second scanning result acquisition module, configured to determine the least hot level of the at least two hot levels as the cold hot level, and determine a second scanning result of the process container based on the first scanning result; wherein the second scanning result includes the amount of cold memory page data and the total amount of memory page data in the process container; a data volume statistics module, configured to obtain a total cold memory page data volume and a total memory page data volume of the target node based on the second scan result of each process container; A memory status acquisition module is used to obtain the cold memory page ratio of the target node based on the total cold memory page data volume of the target node and the total memory page data volume of the target node, and determine the memory status of the target node at the current moment based on the cold memory page ratio.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, characterized in that The computer product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images