Container resource management method and device, electronic equipment and computer program product

Abstract

The invention discloses a container resource management method and device, electronic equipment and a computer program product, and relates to the technical field of cloud computing, and the method comprises the steps: collecting basic performance index data of a target container, application layer index data and access frequency data of a container file through a monitoring tool container, and performing cold and hot marking in combination with a preset data access threshold, periodically updating the cold and hot state of the current storage data in the memory to obtain a dynamic data classification result, and migrating the newly marked cold data from the memory to a disk based on the dynamic data classification result under the condition that the target container is in the first power consumption mode. And under the condition that the target container is switched to the second power consumption mode, recovering the cold data stored in the disk to the memory based on the dynamic data classification result. According to the resource management method and device, the technical problem that resource waste is easily caused due to the fact that cold and hot data feature analysis in the container operation environment is less in a resource management method in the related technology is solved.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing or other related fields, and in particular to a container resource management method and device, an electronic device, and a computer program product. Background Art

[0002] With the rise of container technology, containers consume less resources on the host machine and can achieve functions such as rapid deployment and cross-platform migration. They have gradually replaced virtual machines and become the mainstream virtualization technology on cloud platforms. Compared with virtual machines, containers start faster and have higher resource utilization. At present, containers have been widely used in cloud computing, mobile communications, finance, education, medical care and other fields. The rise of container technology has also driven the development of container cluster technology. Container cluster technology can reasonably schedule resources on a single host machine, thereby improving resource utilization. Container cluster technology can reasonably allocate resources according to the performance requirements of the container runtime, allocate resources to containers with high performance requirements, improve container execution efficiency, and enhance user experience.

[0003] In related technologies, the performance of containers can be improved by dynamically optimizing the container operating environment according to the performance requirements of the container runtime. However, the resource requirements of containers may change significantly during different periods of business application operation. During low-traffic periods, containers may consume too much memory resources, resulting in resource waste; while during high-traffic periods, if resources are insufficiently allocated, the response speed and processing power of the application may be affected. In other words, the hot and cold data characteristics of the container operating environment are not taken into account, which can easily lead to resource waste.

[0004] The performance and resource requirements of container applications change dynamically, and the resource management methods in related technologies cannot adapt to such changes in a timely manner. Especially in scenarios where business traffic changes dramatically, how to achieve dynamic adjustment of resources to cope with mode switching with different power consumption requirements.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a container resource management method and device, an electronic device and a computer program product to at least solve the technical problem that the resource management method in the related art is prone to waste of resources due to less analysis of the characteristics of cold and hot data in the container operating environment.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a container resource management method is provided, including: through a monitoring tool container running in a cluster, basic performance indicator data, application layer indicator data and access frequency data of a target container are collected, and the basic performance indicator data and the access frequency data of the container file are marked as hot and cold in combination with a preset data access threshold, wherein after obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through a user-mode memory paging strategy; the hot and cold states of the data currently stored in the memory are regularly updated to obtain a dynamic data classification result; when the target container is in a first power consumption mode, the newly marked cold data is migrated from the memory to the disk based on the dynamic data classification result; when the target container is switched to a second power consumption mode, the cold data stored in the disk is restored to the memory based on the dynamic data classification result, wherein the container application activity in the second power consumption mode is higher than that in the first power consumption mode.

[0008] Optionally, the step of collecting basic performance indicator data, application layer indicator data and access frequency data of container files of the target container through a monitoring tool container running in the cluster includes: associating the target container to be monitored through the monitoring tool container running in the cluster, and using a lightweight indicator exporter to monitor the basic performance indicator data of the target container, and collecting the indicator data of the application layer through a custom script to obtain the application layer indicator data, wherein the basic performance indicator data includes at least one of the following: CPU usage, memory occupancy, disk I / O rate, network throughput, container startup time, and the application layer indicator data includes at least one of the following: request response time, error rate, throughput; collecting the access frequency data of files in the target container through the monitoring tool container, wherein the access frequency data includes at least one of the following: the number of read and write operations of the file, and the access time interval.

[0009] Optionally, the step of associating the target container to be monitored through a monitoring tool container running in the cluster includes: writing an image file, and using the image file to deploy binary files, dependent libraries and initial configuration files of multiple tools in an application container engine to obtain the monitoring tool container, wherein the multiple tools include: a predetermined monitoring tool, a predetermined alarm tool and a predetermined visualization tool; automatically starting the multiple tools through a startup script in the monitoring tool container, linking the target container using a service discovery function, and marking the target container for monitoring, wherein the monitoring mark is used to mark the container of the target container to be monitored.

[0010] Optionally, the step of using a lightweight indicator exporter to monitor the basic performance indicator data of the target container includes: obtaining a data capture strategy and a pre-set data collection frequency; according to the data collection frequency, using a lightweight indicator exporter to obtain the basic performance indicator data of the target container based on the data capture strategy; storing the basic performance indicator data of the target container in a time series database, and performing data visualization at the same time.

[0011] Optionally, in combination with a preset data access threshold, the step of marking the basic performance indicator data and the access frequency data of the container file as hot or cold includes: analyzing the access frequency data of the container file collected by the monitoring tool container, identifying the container file whose access times exceed the preset access frequency threshold in the near window time period, and marking the container file as hot data, wherein the near window time period refers to the time period from the current time point as the tracing point to the preset historical window; identifying the container file that has not been accessed in the preset historical time period or the access times are lower than the preset access frequency threshold, and marking the container file as cold data.

[0012] Optionally, the step of regularly updating the hot and cold status of the data currently stored in the memory to obtain a dynamic data classification result includes: obtaining a data update cycle, and executing an automation script according to the data update cycle, wherein the automation script is used to re-evaluate the access frequency data of the container file stored in the memory to obtain a data evaluation result; based on the data evaluation result, determining a change in the access frequency of the container file; based on the change in the access frequency of the container file, adjusting the hot and cold markings of the container file to obtain a dynamic data classification result, wherein the dynamic data classification result includes at least: newly marked cold data and hot data.

[0013] Optionally, when the target container is in the first power consumption mode, the step of migrating the newly marked cold data from the memory to the disk based on the dynamic data classification result includes: detecting the activity of the application running in the target container, and determining that the target container is in the first power consumption mode when the activity is lower than a preset activity threshold; when the target container is in the first power consumption mode, paging the newly marked cold data from the memory to the disk through a user-mode memory paging strategy and an asynchronous I / O read / write strategy, wherein, in the process of paging the newly marked cold data from the memory to the disk, data migration is performed using a data checksum and a sequential write mechanism.

[0014] Optionally, when the target container switches to the second power consumption mode, the step of restoring the cold data stored in the disk to the memory based on the dynamic data classification result includes: detecting the activity of the application running in the target container, and determining that the target container is in the second power consumption mode when the activity is greater than a preset activity threshold; when the target container is in the second power consumption mode, using a pre-fetch locking technology to pre-load the cold data to be transmitted, so as to restore the cold data stored in the disk to the memory, wherein, when restoring the cold data stored in the disk to the memory, using a data encryption technology to encrypt and transmit the cold data stored in the disk to be transmitted.

[0015] According to another aspect of an embodiment of the present invention, a container resource management device is also provided, including: a data collection unit, which is used to collect basic performance indicator data, application layer indicator data and access frequency data of container files of a target container through a monitoring tool container running in a cluster, and to mark the basic performance indicator data and the access frequency data of the container files as hot and cold in combination with a preset data access threshold, wherein after obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through a user-mode memory paging policy; a state update unit, which is used to periodically update the hot and cold states of the data currently stored in the memory to obtain a dynamic data classification result; a first migration unit, which is used to migrate the newly marked cold data from the memory to the disk based on the dynamic data classification result when the target container is in the first power consumption mode; and a second migration unit, which is used to restore the cold data stored in the disk to the memory based on the dynamic data classification result when the target container is switched to the second power consumption mode, wherein the activity of the container application in the second power consumption mode is higher than that in the first power consumption mode.

[0016] Optionally, the data collection unit includes: a container association module, which is used to associate the target container to be monitored through a monitoring tool container running in the cluster, and use a lightweight indicator exporter to monitor the basic performance indicator data of the target container, and collect the indicator data of the application layer through a custom script to obtain the application layer indicator data, wherein the basic performance indicator data includes at least one of the following: CPU usage, memory usage, disk I / O rate, network throughput, container startup time, and the application layer indicator data includes at least one of the following: request response time, error rate, throughput; an access frequency collection module, which is used to collect the access frequency data of files in the target container through the monitoring tool container, wherein the access frequency data includes at least one of the following: the number of read and write operations of the file, and the access time interval.

[0017] Optionally, the container association module includes: an image file writing submodule, used to write an image file, and use the image file to deploy the binary files, dependent libraries and initial configuration files of multiple tools in the application container engine respectively to obtain the monitoring tool container, wherein the multiple tools include: a predetermined monitoring tool, a predetermined alarm tool and a predetermined visualization tool; a script startup submodule, used to automatically start the multiple tools through the startup script in the monitoring tool container, use the service discovery function to link the target container, and monitor the target container, wherein the monitoring mark is used to mark the container of the target container to be monitored.

[0018] Optionally, the container association module also includes: a collection frequency acquisition submodule, used to obtain a data capture strategy and a pre-set data collection frequency; a performance indicator data acquisition submodule, used to obtain basic performance indicator data of the target container based on the data capture strategy according to the data collection frequency using a lightweight indicator exporter; a performance indicator data storage submodule, used to store the basic performance indicator data of the target container in a time series database, and perform data visualization at the same time.

[0019] Optionally, the data collection unit also includes: an access frequency data analysis module, which is used to analyze the access frequency data of the container file collected by the monitoring tool container, identify the container file whose access times exceed the preset access frequency threshold in the near window time period, and mark the container file as hot data, wherein the near window time period refers to the time period from the current time point as the tracing point to the predetermined historical window; a container file identification module, which is used to identify the container file that has not been accessed within a predetermined historical time period or the access times are lower than the preset access frequency threshold, and mark the container file as cold data.

[0020] Optionally, the status update unit includes: an update cycle acquisition module, used to acquire a data update cycle, and execute an automated script according to the data update cycle, wherein the automated script is used to re-evaluate the access frequency data of the container file stored in the memory to obtain a data evaluation result; an access frequency change determination module, used to determine the change in access frequency of the container file based on the data evaluation result; and a cold and hot mark adjustment module, used to adjust the cold and hot marks of the container file based on the change in access frequency of the container file to obtain a dynamic data classification result, wherein the dynamic data classification result includes at least: newly marked cold data and hot data.

[0021] Optionally, the first migration unit includes: a first detection module, used to detect the activity of the application running in the target container, and determine that the target container is in the first power consumption mode when the activity is lower than a preset activity threshold; a first migration module, used to migrate the newly marked cold data from the memory to the disk by paging through the user-mode memory paging strategy and the asynchronous I / O read / write strategy when the target container is in the first power consumption mode, wherein, in the process of paging the newly marked cold data from the memory to the disk, data migration is performed using a data check code and a sequential write mechanism.

[0022] Optionally, the second migration unit includes: a second detection module, used to detect the activity of the application running in the target container, and determine that the target container is in the second power consumption mode when the activity is greater than a preset activity threshold; a second migration module, used to use a pre-fetch locking technology to pre-load the cold data to be transmitted when the target container is in the second power consumption mode, so as to restore the cold data stored in the disk to the memory, wherein, when restoring the cold data stored in the disk to the memory, a data encryption technology is used to encrypt and transmit the cold data stored in the disk to be transmitted.

[0023] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned container resource management methods.

[0024] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned container resource management methods.

[0025] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of any one of the above container resource management methods are implemented.

[0026] In the present disclosure, by running a monitoring tool container in a cluster, basic performance indicator data, application layer indicator data and access frequency data of container files of a target container are collected, and combined with a preset data access threshold, the basic performance indicator data and the access frequency data of container files are marked as hot and cold. After obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through the user-mode memory paging strategy. The hot and cold states of the data currently stored in the memory are updated regularly to obtain a dynamic data classification result. When the target container is in a first power consumption mode, the newly marked cold data is migrated from the memory to the disk based on the dynamic data classification result. When the target container is switched to a second power consumption mode, the cold data stored in the disk is restored to the memory based on the dynamic data classification result, wherein the activity of the container application in the second power consumption mode is higher than that in the first power consumption mode.

[0027] In the present disclosure, the hot and cold data of the container can be automatically identified, and the memory usage of the container can be dynamically adjusted according to the changes in business traffic and resource requirements, so as to achieve smooth switching between the low power consumption mode at night (i.e., the first power consumption mode mentioned above) and the high performance mode during the day (i.e., the second power consumption mode mentioned above), thereby optimizing the overall resource utilization of the data center and reducing operating costs. At the same time, by quickly migrating cold data to the memory, the high response speed and stable operation of the application can be ensured, and the requirements of financial transactions for high reliability and high performance can be met, thereby solving the technical problem that the resource management method in the related technology is prone to waste of resources due to less analysis of the characteristics of hot and cold data in the container operating environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0029] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a container resource management method is shown;

[0030] Figure 2 is a flow chart of an optional container resource management method according to an embodiment of the present invention;

[0031] Figure 3 is a flow chart of a container resource management method based on memory information disk storage technology according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of an optional container resource management device according to an embodiment of the present invention;

[0033] Figure 5It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] It should be noted that the container resource management method and device in the present disclosure can be used in the field of cloud computing. When the corresponding memory information of the container that needs to be compressed is stored in the disk based on cloud computing to achieve the low power consumption mode of the container, it can also be used in any field other than the field of cloud computing. When the corresponding memory information of the container that needs to be compressed is stored in the disk based on cloud computing to achieve the low power consumption mode of the container, the application field of the container resource management method and device in the present disclosure is not limited.

[0037] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation portals are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0038] It should be noted that in the present disclosure, when collecting and analyzing customer information, corresponding operation entrances are provided for users to choose to agree or reject the automated decision results; if the user chooses to reject, the expert decision-making process is entered.

[0039] The following embodiments of the present invention can be applied to various container resource management systems / applications / devices. The present invention can use various container monitoring tools to obtain the access frequency of each file in the container, thereby determining the compression strategy of the container. At the same time, the memory information disk storage technology is used to store the corresponding memory information of the container that needs to be compressed to the disk, thereby realizing the low power consumption mode of the container.

[0040] The present invention is described in detail below in conjunction with various embodiments.

[0041] Embodiment 1

[0042] According to an embodiment of the present invention, an embodiment of a container resource management method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] The container resource management method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a container resource management method. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more ( Figure 1 The computer system 102 includes a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU (Microcontroller Unit) or a programmable logic device FPGA (Field Programmable Gate Array) and other processing devices), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0044] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0045] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the container resource management method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned container resource management method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0047] The display may be, for example, a touch screen liquid crystal display (LCD), which enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0048] Under the above operating environment, this application provides Figure 2 The container resource management method shown. Figure 2 The figure is a flowchart of an optional container resource management method according to an embodiment of the present invention.

[0049] It should be noted that this embodiment can be run in a cloud computing environment, and the data center of a financial institution often needs to run thousands of containers to support different applications and services. The resource consumption of these containers will vary significantly during the day and night, weekdays and weekends, and during specific events (such as promotions). The technical solution provided in this embodiment can automatically identify the hot and cold data of the container, dynamically adjust the memory usage of the container according to changes in business traffic and resource requirements, and achieve smooth switching between low-power mode at night and high-performance mode during the day, thereby optimizing the overall resource utilization of the data center and reducing operating costs. At the same time, it can effectively reduce the resource occupation of the container during non-peak hours, reduce hardware costs and energy consumption, and during business peaks, such as Double 11, Spring Festival and other holidays, by quickly migrating cold data to memory, ensure high response speed and stable operation of the application, and meet the high reliability and high performance requirements of financial transactions.

[0050] Furthermore, this embodiment can effectively identify which data is critical to the current analysis task (hot data) and which is historical data or infrequently accessed data (cold data). By saving cold data to disk during low-activity periods to reduce memory usage, and restoring cold data to memory during high-activity periods, data processing efficiency can be significantly improved, storage costs can be reduced, and rapid response and processing capabilities of analysis tasks can be ensured. For scenarios where resource allocation needs to be dynamically adjusted according to actual needs, such as various online businesses, Internet of Things applications, etc., the technical solution of the present application can provide a flexible resource management mechanism. When demand is low, by storing cold data on disk, resource consumption can be reduced and operating costs can be reduced; and when demand is peak, by quickly restoring cold data to memory, it can be ensured that the system has sufficient resources to cope with high traffic, ensuring business continuity and user experience.

[0051] like Figure 2 As shown, the container resource management method includes the following steps:

[0052] Step S201, by running a monitoring tool container in the cluster, collects basic performance indicator data, application layer indicator data and access frequency data of the target container, and combines the preset data access threshold to perform hot and cold marking on the basic performance indicator data and the access frequency data of the container file. After obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through the user-mode memory paging strategy.

[0053] Here, the collection of basic performance indicator data is not limited to traditional CPU usage, memory usage, disk I / O rate, network throughput and container startup time, but also includes more detailed indicators such as CPU waiting time, context switching frequency, memory paging rate, disk read and write latency, network latency and packet loss rate. These indicators can more comprehensively reflect the running status of the container.

[0054] The collection of application layer indicator data covers the entire process of request processing, including but not limited to request response time, request error rate, request throughput, request resource consumption (CPU and memory), request type (read / write / query), and request success rate. Through custom scripts interacting with the target application, more detailed operating status can be captured, such as transaction success rate, service availability, average response time, etc., to evaluate the health of the application and user service quality.

[0055] Among them, the step of collecting basic performance indicator data, application layer indicator data and access frequency data of container files of the target container through a monitoring tool container running in the cluster includes: associating the target container to be monitored through the monitoring tool container running in the cluster, and using a lightweight indicator exporter to monitor the basic performance indicator data of the target container, and collecting the indicator data of the application layer through a custom script to obtain the application layer indicator data, wherein the basic performance indicator data includes at least one of the following: CPU utilization, memory usage, disk I / O rate, network throughput, container startup time, and the application layer indicator data includes at least one of the following: request response time, error rate, and throughput; collecting the access frequency data of files in the target container through the monitoring tool container, wherein the access frequency data includes at least one of the following: the number of read and write operations of the file, and the access time interval.

[0056] In this embodiment, the monitoring tool container can automatically associate the target container to be monitored through the service discovery mechanism of Kubernetes. Optionally, the step of associating the target container to be monitored through the monitoring tool container running in the cluster includes: writing an image file, using the image file to deploy the binary files, dependent libraries and initial configuration files of multiple tools in the application container engine respectively, and obtaining the monitoring tool container, wherein the multiple tools include: a predetermined monitoring tool, a predetermined alarm tool and a predetermined visualization tool; automatically starting multiple tools through the startup script in the monitoring tool container, linking the target container using the service discovery function, and marking the target container for monitoring, wherein the monitoring mark is used to mark the container of the target container to be monitored.

[0057] Here, in the image stage of building the monitoring tool container, the binary files, dependent libraries and initial configuration files of various monitoring tools can be integrated through customized Dockerfile. In addition, the container image also contains various lightweight indicator exporters and custom scripts for collecting specific application layer indicator data. Among them, Dockerfile is a text file used to build a Docker image, which contains a series of instructions and configurations for defining the image building steps and environment settings. For example, by writing Dockerfile, multiple tools (tools used to collect container operation status data and performance indicators) can be packaged into a Docker image, so that they can be easily deployed and run in a Docker container, simplifying the deployment process and improving the portability and scalability of the application. At the same time, the integrated preset alarm tool can provide real-time alarms for abnormal performance indicators based on preset rules and thresholds.

[0058] In the image building phase, in addition to the integrated monitoring tool binary files, all dependent libraries and initial configuration files are also included. The dependent libraries are used to ensure that the monitoring tool can run stably, and the initial configuration file contains the default configuration of the tool and the association rules of the target container. The association rules can include the crawling target list, crawling interval, data compression and filtering strategy of the monitoring alarm tool, as well as the plug-in configuration, data source configuration and preset dashboard layout of the data visualization tool. By integrating these configurations, the monitoring tool container can automatically identify and associate the target container to be monitored without additional manual configuration.

[0059] It should also be noted that inside the monitoring tool container, there is a startup script that is used to automatically load and configure all integrated monitoring tools when the container starts. The startup script reads the initial configuration file inside the container and sets the operating parameters of the tool, such as the capture port of the monitoring alarm tool and the listening address of the data visualization tool. In addition, the startup script can also dynamically adjust the configuration of the tool according to the operating status of the cluster, such as reducing the capture frequency of the monitoring alarm tool when resources are tight to reduce the performance impact on the target container. In addition, the startup script of the monitoring tool container also includes a policy update mechanism that can dynamically update the monitoring policy according to changes in application requirements, system load, and data access patterns. For example, when a surge in the business volume processed by the target container is detected, the monitoring tool will increase the frequency of data collection accordingly, increase the depth and breadth of monitoring indicators, and ensure that performance bottlenecks or anomalies can be discovered and responded to in a timely manner.

[0060] The monitoring tool container uses the service discovery function of Kubernetes to automatically discover and associate target containers within the cluster. By setting specific tags, the monitoring tool can identify which containers are its monitoring targets, so as to collect performance indicators and monitor them. Monitoring tags are a kind of metadata, which can be a specific identifier added to the container label, or a rule defined in the configuration file of the monitoring tool to specify which containers or services need to be monitored and which data needs to be recorded. Through service discovery and monitoring tags, the monitoring tool container can intelligently select monitoring objects, avoid mismonitoring of non-target containers, and improve the efficiency and accuracy of monitoring.

[0061] When collecting data, the monitoring tool container will optimize data transmission and storage strategies, reduce network overhead, and improve data security. For example, by setting a reasonable crawling interval to avoid excessive access to the target container, and using data compression and filtering technology to reduce the amount of data stored and transmitted.

[0062] Alternatively, step S201 is a step of using a lightweight indicator exporter to monitor basic performance indicator data of a target container, including: obtaining a data capture strategy and a pre-set data collection frequency; according to the data collection frequency, using a lightweight indicator exporter to obtain basic performance indicator data of the target container based on the data capture strategy; storing the basic performance indicator data of the target container in a time series database, and performing data visualization at the same time.

[0063] In this embodiment, the data capture strategy includes but is not limited to the collection of performance indicators, the conditions under which the data is collected, data processing and filtering rules, etc. The collection frequency is set according to the volatility, importance and impact of the indicators on the business. For example, the CPU usage and memory usage may be set to be collected every 5 seconds to quickly respond to changes in resource usage; while the disk I / O rate and network throughput may be set to be collected every 30 seconds or every minute, because these indicators change relatively slowly, and frequent collection may bring additional performance overhead.

[0064] The lightweight indicator exporter continuously monitors the basic performance indicators of the target container based on the set collection frequency and crawling strategy. The collected data includes but is not limited to CPU usage, memory usage, disk I / O rate, network throughput, etc. During the collection process, the exporter will also perform preliminary processing and filtering on the data to remove outliers and irrelevant data to ensure the accuracy and reliability of the data. In addition, in order to reduce the overhead of network transmission and storage, the exporter may compress or aggregate the data.

[0065] The collected performance indicator data is stored in a time series database, which can efficiently store and query a large amount of time series data and is an ideal choice for monitoring and alarm systems. At the same time, through integration with visualization tools, the stored performance data can be displayed in the form of charts, dashboards, etc., so that users can intuitively understand the operating status and performance trends of the container, conduct in-depth analysis and locate problems.

[0066] Optionally, in combination with a preset data access threshold, the step of marking the basic performance indicator data and the access frequency data of the container file as hot or cold includes: analyzing the access frequency data of the container file collected by the monitoring tool container, identifying the container file whose access times in the near window time period exceed the preset access frequency threshold, and marking the container file as hot data, wherein the near window time period refers to the time period from the current time point as the tracing point to the preset historical window; identifying the container file that has not been accessed in the preset historical time period or the access times are lower than the preset access frequency threshold, and marking the container file as cold data.

[0067] In this embodiment, the monitoring tool container collects data such as the number of file read and write operations and access time intervals by monitoring the access patterns of files inside the target container. The data is analyzed regularly to evaluate the access frequency and pattern of files. In particular, the focus is on identifying whether the access pattern of files presents periodic hot spots or long-term cold spot characteristics to optimize resource allocation strategies.

[0068] Based on the collected file access frequency data, the system will determine hot and cold data according to the preset access frequency threshold. The preset access frequency threshold can be the average access frequency based on historical data statistics, or a threshold that is dynamically adjusted according to business needs and resource conditions. The near window time period, that is, the time period traced back from the current time point, is used to evaluate the recent access frequency of files. This time period is usually set to a few minutes to a few hours to reflect the immediate access status of the file. By counting the number of file accesses in the near window time period, files above the threshold are identified and marked as hot data, and files below the threshold are marked as cold data.

[0069] It should be noted that the marking strategy for hot and cold data in this embodiment is not only based on the access frequency of files, but also takes into account factors such as the importance, size, and type of files. For example, even if some files are accessed less frequently, they will still be marked as hot data if they are key components of an application or contain sensitive information to ensure fast access and security. In addition, the strategy may also include time-based marking rules, that is, as the life cycle of a file changes, its hot and cold status may change, and it needs to be re-evaluated and marked regularly.

[0070] It should be noted that the marking of hot and cold data in this embodiment is not static, but is updated regularly according to the dynamic changes in file access patterns. This can be achieved by setting up regular tasks (such as every hour or every day). When the task is executed, the file access frequency data will be re-analyzed and its hot and cold status will be adjusted. This dynamic update mechanism ensures the accuracy of data status and the effectiveness of resource allocation strategies, and can respond to changes in business needs in a timely manner.

[0071] By integrating various monitoring tools, this embodiment can achieve comprehensive monitoring of container performance indicators, and by optimizing data collection frequency and strategy, it reduces unnecessary performance overhead and improves the efficiency of monitoring tools. In addition, the visual display of monitoring data makes it easier for operation and maintenance personnel to quickly understand the container status, perform troubleshooting and performance optimization.

[0072] Step S202, regularly updating the hot and cold status of the data currently stored in the memory to obtain a dynamic data classification result.

[0073] By dynamically analyzing and adjusting the hot and cold states of data, the system can intelligently move cold data with low access frequency from memory to disk storage, thereby freeing up valuable memory resources. At night or during business off-peak hours, this mechanism can effectively reduce the memory usage of containers, reduce hardware costs, and reduce energy consumption, thereby optimizing resource allocation and saving costs. During business peak hours or when there is high traffic access, the system can quickly identify and restore hot data to memory to ensure that high-frequency access data can receive a low-latency response. This mechanism can significantly improve data access speed and application response time, improve application throughput, and thus enhance user experience, especially in scenarios such as financial transactions, online shopping, and data queries, where application performance requirements are particularly stringent.

[0074] Optionally, step S202 includes: obtaining a data update cycle, executing an automated script according to the data update cycle, wherein the automated script is used to re-evaluate access frequency data of container files stored in the memory to obtain data evaluation results; based on the data evaluation results, determining changes in access frequency to the container files; based on the changes in access frequency to the container files, adjusting cold and hot markings for the container files to obtain dynamic data classification results, wherein the dynamic data classification results at least include: newly marked cold data and hot data.

[0075] It should be noted that in this embodiment, the automation script needs to be executed according to the data update cycle. The setting of the data update cycle here needs to consider various factors, including but not limited to the busyness of the business, the stability of the file access mode, and the precision requirements of resource management. For example, in an environment where the business traffic changes frequently, the data update cycle may be set to a shorter period (such as every 1 hour) to quickly respond to changes in the file access mode; in a scenario where the file access mode is relatively stable, it can be set to a longer period (such as every 24 hours) to reduce unnecessary evaluation overhead and system resource consumption. In this embodiment, the automation script is usually composed of a series of instructions, which are intended to read the file access frequency data stored in the memory, analyze the data, evaluate the hot and cold status of the file, and generate data evaluation results. The execution of the automation script depends on a predetermined time scheduling mechanism to ensure automatic operation at the beginning of each data update cycle without manual intervention.

[0076] It should also be noted that the data evaluation results are generated after the execution of the automated script, and include the latest evaluation of the access frequency of each file. During the evaluation process, the script will calculate key indicators such as the number of file accesses and access time intervals in the most recent data update cycle, and compare them with the preset cold and hot data determination thresholds. If the access frequency of a file increases significantly and exceeds the hot data threshold, it will be re-evaluated as hot data even if it was previously marked as cold data; conversely, if the access frequency of a file decreases and is below the cold data threshold, it will be re-marked as cold data even if it was previously hot data.

[0077] The dynamic data classification results are generated based on data evaluation and access frequency change analysis. The results include newly labeled cold data and hot data, as well as their detailed information, such as file name, file size, access frequency, and last access time. The adjustment of hot and cold tags is based on the dynamic data classification results. Specifically, the newly labeled hot data will be kept in memory first to support high-frequency access needs, while the newly labeled cold data will be gradually moved from memory to disk storage to reduce memory usage and improve system resource utilization. This process requires precise scheduling strategies and efficient memory-disk data migration mechanisms to ensure optimal resource allocation without affecting container application performance. The dynamic update of hot and cold data status is not only a data classification process, but also a key basis for adjusting resource management strategies.

[0078] Based on the newly generated dynamic data classification results, the system can intelligently adjust the resource allocation of the container, such as storing more cold data on disk during low business hours to reduce memory usage and power consumption; and quickly restore cold data to memory during business peak hours to support high concurrent access requirements and improve application performance. This dynamic adjustment strategy can achieve optimal resource utilization in different business scenarios, thereby improving the operating efficiency and cost-effectiveness of the overall system.

[0079] Step S203 : when the target container is in the first power consumption mode, migrate the newly marked cold data from the memory to the disk based on the dynamic data classification result.

[0080] For data migration of the target container in the first power consumption mode, optionally, step S203 includes: detecting the activity of the application running in the target container, and determining that the target container is in the first power consumption mode when the activity is lower than a preset activity threshold; when the target container is in the first power consumption mode, paging the newly marked cold data from the memory to the disk through the user-mode memory paging strategy and the asynchronous I / O read / write strategy, wherein, in the process of paging the newly marked cold data from the memory to the disk, data migration is performed using a data checksum and a sequential write mechanism.

[0081] The activity of the application running in the target container is continuously monitored through the monitoring tool container, including but not limited to indicators such as request response time, request frequency, CPU and memory usage. When it is detected that the application activity is lower than the preset activity threshold, the system determines that the target container enters a low power consumption state, that is, the first power consumption mode. Among them, the setting of the preset activity threshold in this embodiment needs to comprehensively consider business needs and resource management strategies to ensure efficient use of resources without affecting key services.

[0082] When the target container is in the first power consumption mode, the system performs cold data migration through the user-mode memory paging strategy. The core of the user-mode memory paging technology is to implement the management and replacement of memory pages in user space without the frequent intervention of the operating system kernel, thereby reducing the overhead of system calls and improving the efficiency of data migration. In the specific implementation, the system will use this technology to migrate the newly marked cold data from the memory to the disk in pages.

[0083] During the data migration process, an asynchronous I / O read / write strategy is adopted, that is, when writing data to the disk, it does not wait for the completion of the I / O operation, but allows other operations to continue, thereby avoiding the delay caused by the I / O operation and improving the concurrency and speed of data migration. In addition, in order to avoid possible damage or loss of data during the process of writing to the disk, the system generates a data checksum during data migration to ensure the integrity and accuracy of the data. The sequential write mechanism further optimizes the disk I / O efficiency. By writing data sequentially to the disk, it reduces the disk seek time and fragmentation, and accelerates the data migration process.

[0084] Step S204: when the target container is switched to the second power consumption mode, the cold data stored in the disk is restored to the memory based on the dynamic data classification result, wherein the activity of the container application in the second power consumption mode is higher than that in the first power consumption mode.

[0085] For restoring data to the memory when the target container switches to the second power consumption mode, optionally, step S204 includes: detecting the activity of the application running in the target container, and determining that the target container is in the second power consumption mode when the activity is greater than a preset activity threshold; when the target container is in the second power consumption mode, using a pre-fetch locking technology to pre-load cold data to be transmitted, so as to restore the cold data stored in the disk to the memory, wherein, when restoring the cold data stored in the disk to the memory, using a data encryption technology to encrypt and transmit the cold data stored in the disk to be transmitted.

[0086] The system continuously monitors the activity of applications running in the target container. When the activity exceeds the preset activity threshold, indicating an increase in business load or high traffic access, the target container will switch from low power mode to high performance mode, i.e., the second power mode. The setting of the activity threshold here also needs to consider business characteristics and resource requirements to ensure that the mode transition can respond to business changes in a timely manner and provide sufficient performance support. When the target container enters the second power mode, the system uses pre-fetch locking technology to restore cold data stored on the disk to memory. Pre-fetch locking technology can predict which data will be frequently accessed and load this data into memory in advance, while locking hot data to ensure its priority in memory, reducing data access latency and improving application response speed and throughput.

[0087] In the process of restoring data from disk to memory, in order to ensure data security, the system uses data encryption technology to encrypt and transmit the cold data stored in the disk. The selection of encryption technology needs to comprehensively consider the efficiency, security and compatibility of the encryption algorithm with the existing system architecture to ensure that the data transmission speed will not be significantly reduced during the encryption process. After the encrypted data is transferred to the target container memory, it will be decrypted to ensure the complete availability of the data.

[0088] Through the above implementation steps, during the data migration and recovery process, this application adopts data verification code, sequential writing mechanism and data encryption technology to effectively prevent data damage or leakage during transmission and storage, and ensure the security and integrity of data, especially in scenarios involving sensitive business data.

[0089] This embodiment can automatically detect application activity and adjust the hot and cold states of data through automated scripts and intelligent analysis algorithms to achieve dynamic allocation and optimization of resources. This mechanism reduces the need for manual intervention, improves operation and maintenance efficiency, and also ensures the real-time and accuracy of resource adjustment strategies.

[0090] Through the above steps, the basic performance indicator data, application layer indicator data and access frequency data of the container file of the target container can be collected by running the monitoring tool container in the cluster, and the basic performance indicator data and the access frequency data of the container file can be marked as hot and cold in combination with the preset data access threshold. After obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through the user-mode memory paging policy. The hot and cold states of the data currently stored in the memory are regularly updated to obtain the dynamic data classification result; when the target container is in the first power consumption mode, the newly marked cold data is migrated from the memory to the disk based on the dynamic data classification result, and when the target container is switched to the second power consumption mode, the cold data stored in the disk is restored to the memory based on the dynamic data classification result, wherein the container application activity in the second power consumption mode is higher than that in the first power consumption mode. In this embodiment, the cold and hot data of the container can be automatically identified, and the memory usage of the container can be dynamically adjusted according to changes in business traffic and resource requirements, so as to achieve smooth switching between low power consumption mode at night (i.e., the first power consumption mode mentioned above) and high performance mode during the day (i.e., the second power consumption mode mentioned above), thereby optimizing the overall resource utilization of the data center and reducing operating costs. At the same time, by quickly migrating cold data to memory, the high response speed and stable operation of the application can be ensured, and the requirements of financial transactions for high reliability and high performance can be met, thereby solving the technical problem that the resource management method in the related technology is prone to waste of resources due to less analysis of the characteristics of cold and hot data in the container operating environment.

[0091] By intelligently analyzing the access patterns of data in the container memory, identifying and marking hot and cold data, this embodiment can migrate cold data from memory to disk during periods of low business traffic, thereby reducing memory resource usage, enabling low-power mode operation of the container, and effectively saving costs and energy. When business traffic surges, hot data can be quickly restored to memory to ensure high-speed application response and meet performance requirements during peak hours. During business peaks, by quickly restoring cold data stored on disk to memory, this application can significantly improve data access speed and application response time, increase throughput, and thus improve user experience, especially in scenarios such as financial transactions and real-time data analysis that have strict requirements on latency and throughput, ensuring efficient operation of applications.

[0092] Another optional specific implementation is described in detail below.

[0093] Figure 3 is a flow chart of a container resource management method based on memory information disk storage technology according to an embodiment of the present invention. Figure 3 As shown, including:

[0094] Step 1: Add logging or monitoring tools to collect metrics and analyze data access patterns.

[0095] Deploy the container running the monitoring tool and the target container in the cluster. In the image building phase, integrate the binary files, dependent libraries, and initial configuration files of various monitoring tools into the container for the monitoring tool container, and automatically start the tool and associated services through the startup script. Then use the service discovery function to enable the monitoring tool container to automatically discover the target container and mark the container to be monitored through Kubernetes labels. Run a lightweight indicator exporter in the monitored container, and use the scheduling strategy to schedule the monitoring tool to the same node as the monitored container to reduce network overhead. Then configure reasonable CPU and memory resource restrictions, optimize the sampling frequency and data compression function, and filter non-essential indicators to improve efficiency. This ensures low-overhead and high-efficiency container monitoring without affecting the performance of the application container.

[0096] Step 2: Metrics collection: Capture and record all data access mode metrics for the monitored sources. The key metrics collected mainly include performance metrics: CPU usage, memory usage, disk I / O rate, network throughput, and container startup time, etc.; application layer metrics: request response time, error rate, throughput, etc. These metrics will directly obtain basic performance data through the running lightweight exporter, and capture key metrics of the container and application layers in combination with custom scripts or specific APIs. The specific metric collection frequency will be set according to the importance of the metric. For example, basic performance metrics (CPU, memory) can be set to be collected every 10 seconds, while metrics with lower frequency changes (disk I / O, network throughput) can be set to be collected every 30 seconds.

[0097] Step 3: Data access pattern analysis. According to the collected indicator data, analyze the situation of each data, and mark the data according to the defined ratio of hot and cold data, marking hot data and cold data. Specifically, the criteria for determining hot and cold data will be defined based on factors such as access frequency, time interval, and data type. Hot data usually refers to data with an access frequency higher than a preset threshold within a certain time window. For example, data that has been accessed more than 100 times in the last 10 minutes is marked as hot data. Cold data refers to data with an access frequency lower than the threshold or has not been accessed for a long time (such as the past 24 hours). In addition, the access pattern of the data should be comprehensively considered, such as the behavioral characteristics of periodic hot spots (frequent access in a short period of time and then gradually cooling down) and long-term static cold data (almost no longer accessed), and judgments should be made by monitoring access logs or the timestamps of data requests.

[0098] Step 4: Dynamically classify hot and cold data, determine the specific dynamic classification strategy and implement it, and regularly update the hot and cold status of the data according to the strategy. The dynamic classification strategy will be periodically updated according to the hot and cold status of the data. Specifically, the core rules of the dynamic strategy will be based on factors such as the specific threshold of the access frequency set manually, the access time interval, and the stage of the data life cycle, combined with the hot and cold marking of the data in step three to implement classification. During implementation, step three is executed through pre-set periodic tasks (such as every 1 hour or custom time interval) to analyze the access logs and real-time performance indicators of the stored data to determine whether the current status of the data has changed. For example, for data whose access frequency has dropped significantly in the past 10 minutes and has not reached the hot data threshold, it will be re-marked as cold data; for cold data that has been frequently accessed recently, it will be converted to hot data.

[0099] Step 5: Data storage strategy implementation, first determine whether the target container is in low power mode, if so, use the data storage strategy to store memory information into disk, if not, use the data storage strategy to store disk data into memory.

[0100] Before executing this step five, you need to determine the data storage strategy, such as user-mode fast memory paging technology. The core principle is to dynamically bind the hot and cold states of data to the storage medium through the user-mode memory management mechanism, and use the heterogeneous performance of memory and disk to improve data storage efficiency. The working principle is to keep hot data in memory first and page cold data to disk for storage based on the access frequency and hot and cold marks of memory pages. When hot data in memory needs to be accessed, the data page is directly mapped to user-mode memory through the user-mode memory mapping mechanism to provide access in a low-latency manner; for cold data, it is loaded from disk to memory through asynchronous I / O or direct file read and write calls. In terms of calling methods, the interface provided by the operating system is used to monitor the access status of data pages in real time, and the distribution of data between memory and disk is dynamically adjusted through customized paging strategies.

[0101] In order to ensure that hot data can be quickly returned to the memory, this implementation adopts pre-fetching and locking technology (such as asynchronously loading the predicted required pages in the background). In order to ensure efficient storage of cold data, the design adopts sequential write and compression formats to reduce disk I / O overhead, thereby achieving efficient reading. ), according to the previously determined hot and cold data, when the application activity is low and the traffic is small, such as at night, the data storage strategy is used to save the cold data in the container to the disk, and the low power consumption of the container is achieved while ensuring the survival of the application; when the application activity is high and there is a large amount of traffic, such as during the Double Eleven period, the data storage strategy is used to return the data previously saved on the disk to the memory, thereby achieving high-performance operation of the container and ensuring the handling of large traffic data. At the same time, during this conversion process, it is planned to adopt a variety of measures to ensure the security and integrity of the data.

[0102] It should be noted that when writing to disk, verification technology is used to generate data checksums to ensure the integrity of the data after writing, and a sequential write or transactional write mechanism is used to prevent data loss or damage caused by interruptions. Secondly, during the process of copying from the file system to the memory, the data integrity is verified by reading the checksum, and if necessary, the damaged data page is reloaded. In addition, data encryption technology is combined to protect data security during storage and transmission to avoid unauthorized access or tampering. If data transmission involves remote storage, reliability can be further improved through a redundancy mechanism to ensure the security and integrity of data exchange between memory and disk.

[0103] Through the above implementation, the present invention can use memory information disk storage technology to implement the tidal mode of the container. When in low power consumption, the container can consume less memory resources. When the business traffic is small, the container enters the low power consumption mode to prevent waste of resources.

[0104] At the same time, the user in this implementation has a high degree of autonomy and can customize the reduction ratio of container resources. According to the application situation, it can be ensured that the container can accept small-flow business requests when it is in low-power mode.

[0105] The following is a detailed description in conjunction with another embodiment.

[0106] Embodiment 2

[0107] A container resource management device provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1.

[0108] Figure 4 is a schematic diagram of an optional container resource management device according to an embodiment of the present invention, such as Figure 4 As shown, the container resource management device may include: a data collection unit 41 , a state update unit 42 , a first migration unit 43 , and a second migration unit 44 .

[0109] Among them, the data collection unit 41 is used to collect the basic performance indicator data, application layer indicator data and access frequency data of the target container through the monitoring tool container running in the cluster, and combine the preset data access threshold to perform hot and cold marking on the basic performance indicator data and the access frequency data of the container file. After obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through the user-mode memory paging strategy.

[0110] The state updating unit 42 is used to periodically update the hot and cold states of the data currently stored in the memory to obtain dynamic data classification results.

[0111] The first migration unit 43 is configured to migrate the newly marked cold data from the memory to the disk based on the dynamic data classification result when the target container is in the first power consumption mode.

[0112] The second migration unit 44 is used to restore the cold data stored in the disk to the memory based on the dynamic data classification result when the target container is switched to the second power consumption mode, wherein the activity of the container application in the second power consumption mode is higher than that in the first power consumption mode.

[0113] The above-mentioned container resource management device can collect basic performance indicator data, application layer indicator data and access frequency data of container files of the target container through the monitoring tool container running in the cluster, and mark the basic performance indicator data and the access frequency data of container files as hot and cold in combination with the preset data access threshold. After obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through the user-mode memory paging strategy. The hot and cold states of the data currently stored in the memory are regularly updated through the state update unit 42 to obtain the dynamic data classification result. When the target container is in the first power consumption mode, the newly marked cold data is migrated from the memory to the disk based on the dynamic data classification result through the first migration unit 43. When the target container is switched to the second power consumption mode, the cold data stored in the disk is restored to the memory based on the dynamic data classification result through the second migration unit 44, wherein the activity of the container application in the second power consumption mode is higher than that in the first power consumption mode. In this embodiment, the cold and hot data of the container can be automatically identified, and the memory usage of the container can be dynamically adjusted according to changes in business traffic and resource requirements, so as to achieve smooth switching between low power consumption mode at night (i.e., the first power consumption mode mentioned above) and high performance mode during the day (i.e., the second power consumption mode mentioned above), thereby optimizing the overall resource utilization of the data center and reducing operating costs. At the same time, by quickly migrating cold data to memory, the high response speed and stable operation of the application can be ensured, and the requirements of financial transactions for high reliability and high performance can be met, thereby solving the technical problem that the resource management method in the related technology is prone to waste of resources due to less analysis of the characteristics of cold and hot data in the container operating environment.

[0114] Optionally, the data collection unit includes: a container association module, which is used to associate the target container to be monitored through a monitoring tool container running in the cluster, and use a lightweight indicator exporter to monitor the basic performance indicator data of the target container, and collect the indicator data of the application layer through a custom script to obtain the application layer indicator data, wherein the basic performance indicator data includes at least one of the following: CPU utilization, memory usage, disk I / O rate, network throughput, container startup time, and the application layer indicator data includes at least one of the following: request response time, error rate, throughput; an access frequency collection module, which is used to collect the access frequency data of files in the target container through the monitoring tool container, wherein the access frequency data includes at least one of the following: the number of read and write operations of the file, and the access time interval.

[0115] Optionally, the container association module includes: an image file writing submodule, which is used to write an image file, and use the image file to deploy the binary files, dependent libraries and initial configuration files of multiple tools in the application container engine to obtain a monitoring tool container, wherein the multiple tools include: a predetermined monitoring tool, a predetermined alarm tool and a predetermined visualization tool; a script startup submodule, which is used to automatically start multiple tools through the startup script in the monitoring tool container, use the service discovery function to link the target container, and monitor the target container, wherein the monitoring mark is used to mark the container of the target container to be monitored.

[0116] Optionally, the container association module also includes: a collection frequency acquisition submodule, which is used to obtain a data capture strategy and a pre-set data collection frequency; a performance indicator data acquisition submodule, which is used to obtain basic performance indicator data of the target container based on the data capture strategy according to the data collection frequency using a lightweight indicator exporter; a performance indicator data storage submodule, which is used to store the basic performance indicator data of the target container in a time series database and perform data visualization at the same time.

[0117] Optionally, the data collection unit also includes: an access frequency data analysis module, which is used to analyze the access frequency data of the container files collected by the monitoring tool container, identify the container files whose access times exceed the preset access frequency threshold in the near window time period, and mark the container files as hot data, wherein the near window time period refers to the time period from the current time point as the tracing point to the predetermined historical window; a container file identification module, which is used to identify the container files that have not been accessed in the predetermined historical time period or the access times are lower than the preset access frequency threshold, and mark the container files as cold data.

[0118] Optionally, the status update unit includes: an update cycle acquisition module, used to obtain a data update cycle, and execute an automated script according to the data update cycle, wherein the automated script is used to re-evaluate the access frequency data of the container file stored in the memory to obtain a data evaluation result; an access frequency change determination module, used to determine the change in access frequency of the container file based on the data evaluation result; and a cold and hot mark adjustment module, used to adjust the cold and hot marks of the container file based on the change in access frequency of the container file to obtain a dynamic data classification result, wherein the dynamic data classification result includes at least: newly marked cold data and hot data.

[0119] Optionally, the first migration unit includes: a first detection module, used to detect the activity of the application running in the target container, and determine that the target container is in the first power consumption mode when the activity is lower than a preset activity threshold; a first migration module, used to migrate the newly marked cold data from the memory to the disk by paging through the user-mode memory paging strategy and the asynchronous I / O read / write strategy when the target container is in the first power consumption mode, wherein in the process of paging the newly marked cold data from the memory to the disk, data migration is performed using a data checksum and a sequential write mechanism.

[0120] Optionally, the second migration unit includes: a second detection module, used to detect the activity of the application running in the target container, and determine that the target container is in the second power consumption mode when the activity is greater than a preset activity threshold; a second migration module, used to use a pre-fetch locking technology to pre-load the cold data to be transmitted when the target container is in the second power consumption mode, so as to restore the cold data stored in the disk to the memory, wherein, when restoring the cold data stored in the disk to the memory, a data encryption technology is used to encrypt and transmit the cold data stored in the disk to be transmitted.

[0121] The container resource management device may also include a processor and a memory. The data acquisition unit 41, the status update unit 42, the first migration unit 43, the second migration unit 44, etc. are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.

[0122] The above processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the management of various container resources in the cloud platform in the financial scenario can be achieved by adjusting the kernel parameters.

[0123] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0124] Embodiment 3

[0125] An embodiment of the present application may provide an electronic device, Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (only one is shown) processor 502, memory 504, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0126] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0127] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: through the monitoring tool container running in the cluster, the basic performance indicator data, application layer indicator data and access frequency data of the container file of the target container are collected, and the basic performance indicator data and the access frequency data of the container file are marked as hot and cold in combination with the preset data access threshold, wherein, after obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through the user-mode memory paging strategy; the hot and cold states of the data currently stored in the memory are regularly updated to obtain the dynamic data classification result; when the target container is in the first power consumption mode, the newly marked cold data is migrated from the memory to the disk based on the dynamic data classification result; when the target container is switched to the second power consumption mode, the cold data stored in the disk is restored to the memory based on the dynamic data classification result, wherein the container application activity in the second power consumption mode is higher than that in the first power consumption mode.

[0128] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: associate the target container to be monitored through the monitoring tool container running in the cluster, and use the lightweight indicator exporter to monitor the basic performance indicator data of the target container, and collect the indicator data of the application layer through a custom script to obtain the application layer indicator data, wherein the basic performance indicator data includes at least one of the following: CPU usage, memory occupancy, disk I / O rate, network throughput, container startup time, and the application layer indicator data includes at least one of the following: request response time, error rate, throughput; collect the access frequency data of the files in the target container through the monitoring tool container, wherein the access frequency data includes at least one of the following: the number of read and write operations of the file, and the access time interval.

[0129] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: write an image file, and use the image file to deploy the binary files, dependent libraries and initial configuration files of multiple tools in the application container engine to obtain a monitoring tool container, wherein the multiple tools include: a predetermined monitoring tool, a predetermined alarm tool and a predetermined visualization tool; automatically start the multiple tools through the startup script in the monitoring tool container, use the service discovery function to link the target container, and monitor the target container, wherein the monitoring mark is used to mark the container of the target container to be monitored.

[0130] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain the data capture strategy and the pre-set data collection frequency; according to the data collection frequency, use the lightweight indicator exporter to obtain the basic performance indicator data of the target container based on the data capture strategy; store the basic performance indicator data of the target container in the time series database, and perform data visualization at the same time.

[0131] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: analyze the access frequency data of the container files collected by the monitoring tool container, identify the container files whose access times exceed the preset access frequency threshold in the near window time period, and mark the container files as hot data, wherein the near window time period refers to the time period from the current time point as the tracing point to the predetermined historical window; identify the container files that have not been accessed in the predetermined historical time period or the access times are lower than the preset access frequency threshold, and mark the container files as cold data.

[0132] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a data update cycle, and execute an automated script according to the data update cycle, wherein the automated script is used to re-evaluate the access frequency data of the container file stored in the memory to obtain a data evaluation result; based on the data evaluation result, determine the change in the access frequency of the container file; based on the change in the access frequency of the container file, adjust the cold and hot markings of the container file to obtain a dynamic data classification result, wherein the dynamic data classification result at least includes: newly marked cold data and hot data.

[0133] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: detect the activity of the application running in the target container, and determine that the target container is in the first power consumption mode when the activity is lower than the preset activity threshold; when the target container is in the first power consumption mode, migrate the newly marked cold data from the memory to the disk by paging through the user state memory paging strategy and the asynchronous I / O read / write strategy, wherein, in the process of migrating the newly marked cold data from the memory to the disk by paging, use the data check code and the sequential write mechanism to perform data migration.

[0134] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: detect the activity of the application running in the target container, and determine that the target container is in the second power consumption mode when the activity is greater than a preset activity threshold; when the target container is in the second power consumption mode, use a pre-fetch locking technology to pre-load the cold data to be transmitted, so as to restore the cold data stored in the disk to the memory, wherein, when restoring the cold data stored in the disk to the memory, use a data encryption technology to encrypt and transmit the cold data stored in the disk to be transmitted.

[0135] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone, a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, etc. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations are shown.

[0136] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0137] Embodiment 4

[0138] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the container resource management method provided in the first embodiment.

[0139] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the container resource management method of any one of the above-mentioned embodiments.

[0140] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0141] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the container resource management method described in each embodiment of the present application.

[0142] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the container resource management method described in each embodiment of the present application are implemented.

[0143] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0144] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0146] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0147] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0148] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0149] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A container resource management method, characterized in that: include: By running a monitoring tool container in the cluster, basic performance indicator data, application layer indicator data, and access frequency data of container files of the target container are collected, and the basic performance indicator data and the access frequency data of the container files are marked as hot and cold in combination with a preset data access threshold. After obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through the user state memory paging strategy; Regularly updating the hot and cold status of the data currently stored in the memory to obtain dynamic data classification results; When the target container is in the first power consumption mode, migrating the newly marked cold data from the memory to the disk based on the dynamic data classification result; When the target container is switched to a second power consumption mode, the cold data stored in the disk is restored to the memory based on the dynamic data classification result, wherein the activity of the container application in the second power consumption mode is higher than that in the first power consumption mode.

2. The method according to claim 1, characterized in that The steps of collecting basic performance indicator data, application layer indicator data, and container file access frequency data of the target container by running the monitoring tool container in the cluster include: The target container to be monitored is associated with a monitoring tool container running in the cluster, and a lightweight indicator exporter is used to monitor the basic performance indicator data of the target container, and the indicator data of the application layer is collected through a custom script to obtain the application layer indicator data, wherein the basic performance indicator data includes at least one of the following: CPU usage, memory usage, disk I / O rate, network throughput, container startup time, and the application layer indicator data includes at least one of the following: request response time, error rate, and throughput; Access frequency data of files in the target container is collected through the monitoring tool container, wherein the access frequency data includes at least one of the following: the number of read and write operations of the file and the access time interval.

3. The method according to claim 2, characterized in that The step of associating the target container to be monitored by the monitoring tool container running in the cluster includes: Compile an image file, and use the image file to deploy binary files, dependent libraries, and initial configuration files of multiple tools in an application container engine to obtain the monitoring tool container, wherein the multiple tools include: a predetermined monitoring tool, a predetermined alarm tool, and a predetermined visualization tool; The multiple tools are automatically started by a startup script in the monitoring tool container, the target container is linked by using a service discovery function, and the target container is marked for monitoring, wherein the monitoring mark is used to mark the container of the target container to be monitored.

4. The method according to claim 2, characterized in that: The steps of using a lightweight indicator exporter to monitor basic performance indicator data of the target container include: Obtain data capture strategies and pre-set data collection frequencies; According to the data collection frequency, using a lightweight indicator exporter to obtain basic performance indicator data of the target container based on the data capture strategy; The basic performance indicator data of the target container is stored in a time series database, and the data is visualized.

5. The method according to claim 1, characterized in that The step of marking the basic performance indicator data and the access frequency data of the container file as hot or cold in combination with a preset data access threshold comprises: Analyze the access frequency data of the container file collected by the monitoring tool container, identify the container file whose access times exceed the preset access frequency threshold in the near window time period, and mark the container file as hot data, wherein the near window time period refers to the time period from the current time point as the tracing point to the predetermined historical window; A container file that has not been accessed within a predetermined historical period or has been accessed less than the preset access frequency threshold is identified, and the container file is marked as cold data.

6. The method according to claim 1, characterized in that The step of regularly updating the hot and cold status of the data currently stored in the memory to obtain a dynamic data classification result comprises: Obtaining a data update cycle, and executing an automation script according to the data update cycle, wherein the automation script is used to re-evaluate access frequency data of the container file stored in the memory to obtain a data evaluation result; Determine, based on the data evaluation result, a change in the access frequency to the container file; Based on the change in the access frequency of the container file, the cold and hot markings of the container file are adjusted to obtain a dynamic data classification result, wherein the dynamic data classification result at least includes: newly marked cold data and hot data.

7. The method according to claim 1, characterized in that When the target container is in the first power consumption mode, the step of migrating the newly marked cold data from the memory to the disk based on the dynamic data classification result includes: detecting the activity of the application running in the target container, and determining that the target container is in the first power consumption mode when the activity is lower than a preset activity threshold; When the target container is in the first power consumption mode, the newly marked cold data is paged and migrated from the memory to the disk through the user-mode memory paging strategy and the asynchronous I / O read / write strategy, wherein in the process of paging the newly marked cold data from the memory to the disk, data verification code and sequential write mechanism are used to perform data migration.

8. The method according to claim 1, characterized in that When the target container is switched to the second power consumption mode, the step of restoring the cold data stored in the disk to the memory based on the dynamic data classification result includes: detecting the activity of the application running in the target container, and determining that the target container is in the second power consumption mode when the activity is greater than a preset activity threshold; When the target container is in the second power consumption mode, a pre-fetch locking technology is used to pre-load the cold data to be transmitted, so as to restore the cold data stored in the disk to the memory, wherein, when restoring the cold data stored in the disk to the memory, a data encryption technology is used to encrypt and transmit the cold data stored in the disk to be transmitted.

9. A container resource management device, characterized in that: include: A data collection unit is used to collect basic performance indicator data, application layer indicator data and access frequency data of container files of a target container by running a monitoring tool container in a cluster, and to mark the basic performance indicator data and the access frequency data of container files as hot or cold in combination with a preset data access threshold, wherein after obtaining the marking result, the hot data in the marking result is retained in the memory, and the cold data in the marking result is paged to the disk through a user-mode memory paging strategy; A state updating unit, used for regularly updating the hot and cold states of the data currently stored in the memory to obtain a dynamic data classification result; A first migration unit, configured to migrate the newly marked cold data from the memory to the disk based on the dynamic data classification result when the target container is in the first power consumption mode; The second migration unit is used to restore the cold data stored in the disk to the memory based on the dynamic data classification result when the target container is switched to a second power consumption mode, wherein the activity of the container application in the second power consumption mode is higher than that in the first power consumption mode.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the container resource management method described in any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the container resource management method according to any one of claims 1 to 8 are implemented.

Images