Container cloud storage volume performance optimization method and device, electronic equipment and program product

Abstract

This invention discloses a method, apparatus, electronic device, and program product for optimizing the performance of container cloud storage volumes. The method includes: acquiring real-time IO requests of a target cloud storage volume through an IO monitoring agent and uploading these requests to an intelligent analysis center; generating IO request timing data based on the real-time IO requests through the intelligent analysis center and inputting this data into a pre-trained access data block prediction model to obtain the target data block to be accessed; generating a preheating task based on the target cloud storage volume and the target data block to be accessed through the intelligent analysis center and distributing the preheating task to a preheating engine; and loading the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer through the preheating task. This invention reduces the access latency of container cloud storage volumes and improves the cache utilization of container cloud storage volumes, thereby enhancing the IO performance of container cloud storage volumes. It can be widely applied in the field of cloud computing technology.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a method, apparatus, electronic device, and program product for optimizing the performance of container cloud storage volumes. Background Technology

[0002] In container cloud platforms (such as Kubernetes), stateful applications (such as databases and middleware) typically persist data by mounting cloud storage volumes (such as Ceph RBD volumes) via the CSI interface. These distributed storage systems typically employ a tiered storage architecture, using high-performance media (such as SSDs) as a caching layer and large-capacity but lower-performance media (such as HDDs) as a capacity layer.

[0003] The current industry standard is a passive caching strategy. When an application in a container initiates an I / O request (such as reading a data block), the storage system first checks whether the data block is already in the SSD cache: if it is (cache hit), the data is returned directly from the SSD with extremely low latency; if it is not (cache miss), the data block must be read from the HDD capacity layer, loaded into the SSD cache, and then returned to the application. This process generates high latency (typically from a few milliseconds to tens of milliseconds), which seriously affects the performance of data-intensive applications.

[0004] Existing container cloud storage volume caching solutions have the following problems: 1) High Cold Start Latency: For newly created volumes or volumes that have not been accessed for a long time, their data is stored in the slow HDD layer. When an application accesses any data for the first time, it will encounter a cache miss, resulting in high latency, which affects application startup speed and user experience.

[0005] 2) Inability to handle sudden I / O surges: When the application's operating mode changes (such as a full table scan of the database or the start of a log analysis task), a large number of sequential read requests will be generated. Passive caching strategies cannot detect this change in advance and can only load data from the HDD one by one after the requests arrive, resulting in consistently low application performance during this period.

[0006] 3) Cache Pollution: Passive strategies may load some data that is accessed occasionally but is no longer needed into the resource-constrained cache, crowding out the space for truly hot data and reducing the overall cache hit rate.

[0007] The above problems urgently need to be addressed. Summary of the Invention

[0008] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0009] Therefore, one objective of this invention is to provide a method for optimizing the performance of container cloud storage volumes. This method reduces the access latency of container cloud storage volumes and improves the cache utilization of container cloud storage volumes, thereby enhancing the IO performance of container cloud storage volumes.

[0010] Another objective of this invention is to provide a container cloud storage volume performance optimization device.

[0011] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include: On one hand, embodiments of the present invention provide a method for optimizing the performance of container cloud storage volumes, including the following steps: The IO monitoring agent obtains the real-time IO requests of the target cloud storage volume and uploads the real-time IO requests to the intelligent analysis center. The intelligent analysis center generates IO request time series data based on the real-time IO request, and inputs the IO request time series data into a pre-trained access data block prediction model to obtain the target data block to be accessed. The intelligent analysis center generates a preheating task based on the target cloud storage volume and the target data block to be accessed, and sends the preheating task to the preheating engine. The preheating engine loads the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task.

[0012] Furthermore, in one embodiment of the present invention, the IO monitoring agent and the preheating engine are both deployed on various working nodes of the container cloud platform, and the intelligent analysis center is centrally deployed on the container cloud platform.

[0013] Furthermore, in one embodiment of the present invention, the step of generating IO request timing data based on the real-time IO request specifically includes: The real-time I / O request is parsed to obtain the corresponding logical block address, I / O size, operation type, and timestamp; I / O request segments are generated based on the logical block address, the I / O size, and the operation type. The I / O request segments within a preset time window are then time-sequentially processed based on the timestamp to generate the I / O request timing data.

[0014] Furthermore, in one embodiment of the present invention, the access data block prediction model is trained through the following steps: Obtain historical IO requests for the sample cloud storage volume, and determine the IO request time series sample for the first historical period and the actual access data block label for the second historical period based on the historical IO requests; The timing samples of the IO requests are input into a pre-built long-term memory neural network to obtain the predicted data blocks to be accessed; The loss value is determined based on the predicted data block to be accessed and the actual data block to be accessed; The parameters of the long-term memory neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained access data block prediction model.

[0015] Furthermore, in one embodiment of the present invention, the step of generating a preheating task based on the target cloud storage volume and the target data block to be accessed, and sending the preheating task to the preheating engine, specifically includes: The target working node is determined based on the target cloud storage volume, and the target data address is determined based on the target data block to be accessed. Determine the prediction confidence of the target data block to be accessed, and determine the preheating priority based on the prediction confidence; The preheating task is generated based on the target data address and the preheating priority, and the preheating task is sent to the preheating engine corresponding to the target working node.

[0016] Furthermore, in one embodiment of the present invention, the step of loading the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task specifically includes: The target data address and the preheating priority are determined based on the preheating task; Generate a corresponding IO read request based on the target data address, and add the IO read request to the preheating queue according to the preheating priority; The IO read requests are executed sequentially according to the preheating queue, so that the target data block to be accessed is loaded from the capacity layer of the target cloud storage volume to the cache layer.

[0017] Furthermore, in one embodiment of the present invention, the container cloud storage volume performance optimization method further includes the following steps: Monitor whether the target data block to be accessed is accessed within a preset time period; When the target data block to be accessed is accessed within a preset time period, a positive sample is generated based on the IO request timing data and the target data block to be accessed. If the target data block to be accessed is not accessed within a preset time period, the target data block to be accessed is removed from the cache layer of the target cloud storage volume, and a negative sample is generated based on the IO request time series data and the target data block to be accessed. The access data block prediction model is incrementally learned based on the positive or negative samples.

[0018] On the other hand, embodiments of the present invention provide a container cloud storage volume performance optimization device, comprising: The IO request acquisition module is used to acquire the real-time IO requests of the target cloud storage volume through the IO monitoring agent and upload the real-time IO requests to the intelligent analysis center; The access data block prediction module is used to generate IO request time series data based on the real-time IO request through the intelligent analysis center, and input the IO request time series data into the pre-trained access data block prediction model to obtain the target data block to be accessed. The preheating task distribution module is used to generate a preheating task through the intelligent analysis center based on the target cloud storage volume and the target data block to be accessed, and to distribute the preheating task to the preheating engine; The data block preheating and loading module is used to load the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task through the preheating engine.

[0019] On the other hand, embodiments of the present invention provide an electronic device, which includes a memory, a processor, a computer program stored in the memory and executable on the processor, and a data bus for implementing communication between the processor and the memory. When the computer program is executed by the processor, it implements the container cloud storage volume performance optimization method as described above.

[0020] On the other hand, embodiments of the present invention also provide a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more computer programs, which can be executed by one or more processors to implement the container cloud storage volume performance optimization method as described above.

[0021] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the container cloud storage volume performance optimization method as described above.

[0022] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention: This invention, in its embodiment, acquires real-time IO requests from a target cloud storage volume through an IO monitoring agent and uploads these requests to an intelligent analysis center. The intelligent analysis center generates IO request timing data based on the real-time IO requests and inputs this data into a pre-trained access data block prediction model to obtain the target data block to be accessed. The intelligent analysis center then generates a preheating task based on the target cloud storage volume and the target data block to be accessed, and distributes this preheating task to a preheating engine. The preheating engine then loads the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task. This invention predicts the data block to be accessed based on the real-time IO requests of the container cloud storage volume, generates corresponding preheating tasks, and loads the data block to be accessed from the capacity layer to the cache layer via the preheating engine. This allows applications to directly read data from the cache layer when requesting access to preheated data blocks, reducing access latency, improving cache utilization, and ultimately enhancing the IO performance of the container cloud storage volume. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating the steps of a container cloud storage volume performance optimization method provided in an embodiment of the present invention; Figure 2 A flowchart of step S102 provided in an embodiment of the present invention; Figure 3 A flowchart illustrating one step of training an access data block prediction model according to an embodiment of the present invention; Figure 4 A flowchart of step S103 provided in an embodiment of the present invention; Figure 5 A flowchart of step S104 provided in an embodiment of the present invention; Figure 6 Another flowchart of the container cloud storage volume performance optimization method provided in this embodiment of the invention; Figure 7 This is a schematic diagram of the container cloud storage volume performance optimization device provided in an embodiment of the present invention; Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention; Figure 9This is a schematic diagram of the structure of the storage medium provided in an embodiment of the present invention. Detailed Implementation

[0025] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. It should be noted that although functional modules are divided in the system schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system schematic diagram or the order in the flowchart. The step numbers in the following embodiments are only set for ease of explanation and do not limit the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0027] The following explains some of the terms used in the embodiments of the present invention.

[0028] Container Cloud Platform: A cloud service platform built on container technology, used to deploy, run and manage containerized applications, typically providing orchestration (such as Kubernetes), storage, networking and other capabilities.

[0029] Cloud Storage Volume: A block device provided by a distributed storage system (such as Ceph, AWS EBS, AzureDisk) that can be mounted onto compute instances (such as virtual machines or containers) for persistent data storage.

[0030] IOPS (Input / Output Operations Per Second): A key metric for measuring the performance of storage devices.

[0031] Access Pattern Recognition Model: A lightweight machine learning model proposed in this invention for analyzing historical I / O request time-series data and predicting data blocks that may be accessed subsequently.

[0032] Preheating Engine: The core module of this invention, responsible for generating preheating tasks based on the output of the access pattern recognition model, and preloading the predicted data blocks from the underlying slow storage medium (such as HDD) into the high-speed cache (such as SSD cache layer or memory cache).

[0033] Kubernetes CSI (Container Storage Interface): A standard interface for container storage provided by Kubernetes, allowing storage vendors to develop plugins and integrate them into the Kubernetes environment.

[0034] The container cloud storage volume performance optimization method provided in this application can be applied to terminals, servers, or software running on terminals or servers. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a driver status monitoring method based on machine vision, but is not limited to the above forms.

[0035] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0036] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data for the proper functioning of the embodiments of this application obtained.

[0037] like Figure 1 The diagram shown is a flowchart of one step of the container cloud storage volume performance optimization method provided in an embodiment of the present invention. (Refer to...) Figure 1 This invention provides a method for optimizing the performance of container cloud storage volumes, specifically including the following steps: S101. Obtain the real-time IO requests of the target cloud storage volume through the IO monitoring agent and upload the real-time IO requests to the intelligent analysis center; S102. The intelligent analysis center generates IO request time series data based on real-time IO requests, and inputs the IO request time series data into a pre-trained access data block prediction model to obtain the target data block to be accessed. S103. The intelligent analysis center generates a preheating task based on the target cloud storage volume and the target data block to be accessed, and sends the preheating task to the preheating engine. S104. The preheating engine loads the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task.

[0038] Specifically, the embodiments of the present invention are deployed in a container cloud platform and mainly include the following components: 1) IO Monitoring Agent: Deployed as a DaemonSet on each worker node, it transparently intercepts and monitors all IO requests sent to a specified cloud storage volume, collects IO sequences (such as LBA, operation type, timestamp) and reports them.

[0039] 2) Intelligent Analysis Center: Centralized deployment, containing an access data block prediction model. This model receives IO sequences reported by each Agent, performs real-time analysis, predicts the subsequent access patterns of each volume, and outputs a predicted list of data blocks.

[0040] 3) Warm-up Engine: Also deployed as a DaemonSet on worker nodes. It receives warm-up tasks (containing volume IDs and a list of data blocks to be warmed up) from the analytics center and actively loads data blocks from the capacity layer to the cache layer by calling the storage system's specific warm-up API or simulating read requests.

[0041] It can be understood that the embodiments of the present invention predict the data blocks to be accessed based on the real-time IO requests of the container cloud storage volume, thereby generating corresponding preheating tasks and loading the data blocks to be accessed from the capacity layer of the container cloud storage volume to the cache layer through the preheating engine. This allows the application to directly read data from the cache layer when requesting access to the preheated data blocks, reducing the access latency of the container cloud storage volume, improving the cache utilization of the container cloud storage volume, and thus improving the IO performance of the container cloud storage volume.

[0042] As an optional implementation, the IO monitoring agent and the preheating engine are deployed on various working nodes of the container cloud platform, while the intelligent analysis center is centrally deployed on the container cloud platform.

[0043] Specifically, when an application Pod in a container accesses a mounted cloud storage volume, the IO monitoring agent on the corresponding worker node captures the IO request, buffers it, and uploads it in batches to the intelligent analysis center.

[0044] like Figure 2 The diagram shown is a flowchart of step S102 provided in an embodiment of the present invention. (Refer to...) Figure 2 As an optional implementation, IO request timing data is generated based on real-time IO requests, specifically including: S1021. Parse the real-time IO request to obtain the corresponding logical block address, IO size, operation type and timestamp; S1022. Generate IO request segments based on logical block address, IO size and operation type, and perform time-series processing on IO request segments within a preset time window based on timestamps to generate IO request time-series data.

[0045] Specifically, the intelligent analysis center parses real-time IO requests to obtain the corresponding logical block address, IO size, operation type, and timestamp. It generates and maintains an IO sequence within a time window for each cloud storage volume, which includes IO request segments composed of logical block address, IO size, and operation type.

[0046] After obtaining the timing data of the IO request, it is input into a pre-trained access data block prediction model. The model learns the current access pattern (e.g., it is reading a large file in a skipping manner with stride=2MB) and predicts the N data blocks that are most likely to be accessed next (e.g., LBA: 20480, 22528, 24576), thus obtaining the target data block to be accessed.

[0047] like Figure 3 The diagram shown is a flowchart illustrating one step of training an access data block prediction model according to an embodiment of the present invention. (Refer to...) Figure 3 As an optional implementation, the access data block prediction model is trained through the following steps: S201. Obtain the historical IO requests of the sample cloud storage volume, and determine the IO request time series sample of the first historical period and the actual access data block label of the second historical period based on the historical IO requests. S202. Input the timing sample of the IO request into the pre-built long-term memory neural network to obtain the predicted data block to be accessed; S203. Determine the loss value based on the predicted data block to be accessed and the actual data block to be accessed; S204. Update the parameters of the long-term memory neural network using the backpropagation algorithm based on the loss value to obtain the trained access data block prediction model.

[0048] Specifically, the historical IO requests of the sample cloud storage volume are obtained, and the time-series samples of IO requests in the first historical period and the actual access data block labels in the second historical period are determined based on the historical IO requests. The time-series samples of IO requests are input into a pre-constructed long-term memory neural network to obtain the predicted data blocks to be accessed. The loss value is determined based on the predicted data blocks to be accessed and the actual access data block labels. The parameters of the long-term memory neural network are updated based on the loss value using the backpropagation algorithm to complete one iteration of training. When the number of iterations reaches a preset threshold, or the loss value is lower than the preset threshold, training is stopped, and the trained access data block prediction model is obtained.

[0049] In some alternative embodiments, the access block prediction model can also be implemented using other lightweight prediction algorithms, such as Markov chains and heuristic-based rule engines (for specific known applications, such as .ibd file access patterns in MySQL).

[0050] like Figure 4 The diagram shown is a flowchart of step S103 provided in an embodiment of the present invention. (Refer to...) Figure 4 As an optional implementation, a preheating task is generated based on the target cloud storage volume and the target data block to be accessed, and the preheating task is sent to the preheating engine, which specifically includes: S1031. Determine the target working node based on the target cloud storage volume, and determine the target data address based on the target data block to be accessed; S1032. Determine the prediction confidence of the target data block to be accessed, and determine the preheating priority based on the prediction confidence. S1033. Generate a preheating task based on the target data address and preheating priority, and send the preheating task to the preheating engine corresponding to the target working node.

[0051] Specifically, the intelligent analysis center determines the preheating priority of the target data block to be accessed based on the prediction confidence level, ensuring that data blocks with a high probability of being accessed are loaded into the cache layer in advance; the target data block to be accessed and the preheating priority are encapsulated into a preheating task and sent to the preheating engine of the working node where the target cloud storage volume is located.

[0052] like Figure 5 The diagram shown is a flowchart of step S104 provided in an embodiment of the present invention. (Refer to...) Figure 5 As an optional implementation, the target data block to be accessed is loaded from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task, specifically including: S1041. Determine the target data address and preheating priority based on the preheating task; S1042. Generate the corresponding IO read request according to the target data address, and add the IO read request to the preheating queue according to the preheating priority; S1043. Execute IO read requests sequentially according to the preheating queue, so that the target data block to be accessed is loaded from the capacity layer of the target cloud storage volume to the cache layer.

[0053] Specifically, after receiving a task, the preheating engine generates a corresponding read I / O request (or calls the storage system's preheating interface). These requests are sent to the storage system, forcing the target data block to be loaded from the HDD layer to the SSD cache layer. Later, when the application in the container actually accesses these preheated data blocks, the request hits directly in the SSD cache layer, achieving low-latency access.

[0054] In this embodiment of the invention, different preheating tasks have corresponding preheating priorities. The preheating engine maintains a preheating queue in real time based on the preheating priority and executes IO read requests sequentially from the queue, thereby ensuring that data blocks that are likely to be accessed are loaded into the cache layer in advance.

[0055] like Figure 6 The diagram shown is a flowchart of one step of the container cloud storage volume performance optimization method provided in an embodiment of the present invention. (Refer to...) Figure 6 As an optional implementation, the container cloud storage volume performance optimization method further includes the following steps: S301. Monitor whether the target data block to be accessed is accessed within a preset time period; S302. When the target data block to be accessed is accessed within a preset time period, a positive sample is generated based on the IO request timing data and the target data block to be accessed. S303. When the target data block to be accessed has not been accessed within a preset time period, the target data block to be accessed is removed from the cache layer of the target cloud storage volume, and negative samples are generated based on the IO request time series data and the target data block to be accessed. S304. Incrementally learn the access data block prediction model based on positive or negative samples.

[0056] Specifically, after the preheating is completed, the system continuously monitors whether the target data block to be accessed is accessed within a preset time period. If the target data block to be accessed is accessed within the preset time period, a positive sample is generated based on the IO request time series data and the target data block to be accessed. If the target data block to be accessed is not accessed within the preset time period, the target data block to be accessed is removed from the cache layer of the target cloud storage volume, and a negative sample is generated based on the IO request time series data and the target data block to be accessed. All positive samples and the negative samples are collected according to a preset period to form an incremental learning dataset. Based on this dataset, the access data block prediction model is incrementally learned, thereby continuously improving the prediction performance of the access data block prediction model.

[0057] In some optional embodiments, embodiments of the present invention can also work in conjunction with the Kubernetes scheduler and perform collaborative warm-up based on application tags, as follows: 1) Users define a specific label "apptype: redis" for the Pod that deploys a database (such as Redis).

[0058] 2) The intelligent analysis center learned that Pods with the tag "apptype: redis" will always sequentially read a file named "dump.rdb" during the initial startup of their storage volume.

[0059] 3) When the Kubernetes scheduler decides to schedule a new Pod with "apptype: redis" to a node, it will emit an event.

[0060] 4) The preheating engine listens for this event. Once detected, it immediately queries the Intelligent Analysis Center for the "startup preheating template" corresponding to the application type (i.e., the data block range of the "dump.rdb" file that needs to be preheated).

[0061] 5) The preheating engine performs a preheating task before the Pod actually starts and mounts the volume, loading all the data blocks of the "dump.rdb" file into the cache.

[0062] 6) After the Pod starts, almost 100% of all data accesses are cached and hit, achieving "second-level" startup.

[0063] It is understood that the embodiments of the present invention predict the data blocks to be accessed based on the real-time IO requests of the container cloud storage volume, thereby generating corresponding preheating tasks and loading the data blocks to be accessed from the capacity layer of the container cloud storage volume to the cache layer through the preheating engine. This allows the application to directly read data from the cache layer when requesting access to the preheated data blocks, reducing the access latency of the container cloud storage volume, improving the cache utilization of the container cloud storage volume, and thus improving the IO performance of the container cloud storage volume.

[0064] Furthermore, this embodiment of the invention proactively preheats the cache, completing high-latency operations that miss cache hits in the background in advance, eliminating application-aware access latency; through intelligent prediction, it avoids bandwidth waste and cache pollution caused by blind prefetching, achieving high preheating accuracy and significantly improving efficiency; and through collaboration with cloud-native platforms, it achieves end-to-end performance optimization from the infrastructure layer to the application layer, resulting in a significant improvement in user experience.

[0065] like Figure 7 The diagram shown is a structural schematic of the container cloud storage volume performance optimization device provided in an embodiment of the present invention. (Refer to...) Figure 7 This invention provides a container cloud storage volume performance optimization device, comprising: The IO request acquisition module is used to acquire real-time IO requests of the target cloud storage volume through the IO monitoring agent and upload the real-time IO requests to the intelligent analysis center; The access data block prediction module is used to generate IO request time series data based on real-time IO requests through the intelligent analysis center, and input the IO request time series data into the pre-trained access data block prediction model to obtain the target data block to be accessed. The preheating task distribution module is used to generate preheating tasks based on the target cloud storage volume and the target data block to be accessed through the intelligent analysis center, and distribute the preheating tasks to the preheating engine. The data block preheating and loading module is used to load the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task through the preheating engine.

[0066] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0067] This invention also provides an electronic device, comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned container cloud storage volume performance optimization method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0068] like Figure 8 The diagram shown is a hardware structure schematic of an electronic device provided in an embodiment of the present invention. (Refer to...) Figure 8 This invention provides an electronic device, comprising: The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The memory 802 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 802 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 using the container cloud storage volume performance optimization method of the embodiments of this invention. The 803 input / output interface is used to implement information input and output. The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804); The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0069] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0070] like Figure 9 The diagram shown is a structural schematic of the storage medium provided in an embodiment of the present invention. (Refer to...) Figure 9 The present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs 901, which can be executed by one or more processors to implement the above-mentioned container cloud storage volume performance optimization method.

[0071] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0072] This invention also discloses a computer program product, including a computer program that, when executed by a processor, implements the above-described container cloud storage volume performance optimization method.

[0073] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0074] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0075] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0076] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0077] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0078] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0079] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0081] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0082] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0083] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0084] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0085] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for optimizing the performance of container cloud storage volumes, characterized in that, Includes the following steps: The IO monitoring agent obtains the real-time IO requests of the target cloud storage volume and uploads the real-time IO requests to the intelligent analysis center. The intelligent analysis center generates IO request time series data based on the real-time IO request, and inputs the IO request time series data into a pre-trained access data block prediction model to obtain the target data block to be accessed. The intelligent analysis center generates a preheating task based on the target cloud storage volume and the target data block to be accessed, and sends the preheating task to the preheating engine. The preheating engine loads the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task.

2. The method for optimizing the performance of container cloud storage volumes according to claim 1, characterized in that, Both the IO monitoring agent and the preheating engine are deployed on various working nodes of the container cloud platform, while the intelligent analysis center is centrally deployed on the container cloud platform.

3. The method for optimizing the performance of container cloud storage volumes according to claim 1, characterized in that, The step of generating IO request timing data based on the real-time IO request specifically includes: The real-time I / O request is parsed to obtain the corresponding logical block address, I / O size, operation type, and timestamp; I / O request segments are generated based on the logical block address, the I / O size, and the operation type. The I / O request segments within a preset time window are then time-sequentially processed based on the timestamp to generate the I / O request timing data.

4. The method for optimizing the performance of container cloud storage volumes according to claim 1, characterized in that, The access data block prediction model is trained through the following steps: Obtain historical IO requests for the sample cloud storage volume, and determine the IO request time series sample for the first historical period and the actual access data block label for the second historical period based on the historical IO requests; The timing samples of the IO requests are input into a pre-built long-term memory neural network to obtain the predicted data blocks to be accessed; The loss value is determined based on the predicted data block to be accessed and the actual data block to be accessed; The parameters of the long-term memory neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained access data block prediction model.

5. The method for optimizing the performance of container cloud storage volumes according to claim 1, characterized in that, The step of generating a preheating task based on the target cloud storage volume and the target data block to be accessed, and then sending the preheating task to the preheating engine, specifically includes: The target working node is determined based on the target cloud storage volume, and the target data address is determined based on the target data block to be accessed. Determine the prediction confidence of the target data block to be accessed, and determine the preheating priority based on the prediction confidence; The preheating task is generated based on the target data address and the preheating priority, and the preheating task is sent to the preheating engine corresponding to the target working node.

6. The method for optimizing the performance of container cloud storage volumes according to claim 5, characterized in that, The step of loading the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task specifically includes: The target data address and the preheating priority are determined based on the preheating task; Generate a corresponding IO read request based on the target data address, and add the IO read request to the preheating queue according to the preheating priority; The IO read requests are executed sequentially according to the preheating queue, so that the target data block to be accessed is loaded from the capacity layer of the target cloud storage volume to the cache layer.

7. A method for optimizing the performance of container cloud storage volumes according to any one of claims 1 to 6, characterized in that, The container cloud storage volume performance optimization method also includes the following steps: Monitor whether the target data block to be accessed is accessed within a preset time period; When the target data block to be accessed is accessed within a preset time period, a positive sample is generated based on the IO request timing data and the target data block to be accessed. If the target data block to be accessed is not accessed within a preset time period, the target data block to be accessed is removed from the cache layer of the target cloud storage volume, and a negative sample is generated based on the IO request time series data and the target data block to be accessed. The access data block prediction model is incrementally learned based on the positive or negative samples.

8. A container cloud storage volume performance optimization device, characterized in that, include: The IO request acquisition module is used to acquire the real-time IO requests of the target cloud storage volume through the IO monitoring agent and upload the real-time IO requests to the intelligent analysis center; The access data block prediction module is used to generate IO request time series data based on the real-time IO request through the intelligent analysis center, and input the IO request time series data into the pre-trained access data block prediction model to obtain the target data block to be accessed. The preheating task distribution module is used to generate a preheating task through the intelligent analysis center based on the target cloud storage volume and the target data block to be accessed, and to distribute the preheating task to the preheating engine; The data block preheating and loading module is used to load the target data block to be accessed from the capacity layer of the target cloud storage volume to the cache layer according to the preheating task through the preheating engine.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, a computer program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the computer program is executed by the processor, it implements the container cloud storage volume performance optimization method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the container cloud storage volume performance optimization method as described in any one of claims 1 to 7.

Images