Speed limiting method and device for data reading and writing, electronic equipment and storage medium

Abstract

The invention discloses a data read-write speed limiting method and device, electronic equipment and a storage medium, and the data read-write speed limiting method comprises the steps: when a target page memory generated by a target process in a control group Cgroup v1 of a first version is written back, determining a target controller corresponding to the target page memory based on a first corresponding relation, the target page memory is generated when the target process performs cache reading and writing, and after the Cgroup v1 is pre-configured, write-back limitation of cache reading and writing is started; determining a target subsystem corresponding to the target controller based on a second corresponding relation; on the basis of the target subsystem, the write-back speed of writing back the data in the target page memory to the target memory is limited, the limitation that the write-back speed of the page memory cannot be limited in the Cgroup v1 is broken through, and the speed limit of cache read-write in the Cgroup v1 is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a data reading and writing speed limiting method, device, electronic device and storage medium. Background Art

[0002] At present, the first version of control groups (Control groups v1, Cgroup v1) is used in a large number of Linux kernels. When the subsystem (blkio cgroup) in Cgroup v1 limits the write-back speed of block devices, it only has a certain speed limit effect on direct IO. However, for buffered IO, the blkio cgroup's block device write-back speed limit function will fail. Summary of the invention

[0003] In view of the above problems, the present application proposes a data reading and writing speed limiting method, device, electronic device and storage medium.

[0004] In a first aspect, an embodiment of the present application provides a method for limiting the speed of data reading and writing, the method comprising: when writing back a target page memory generated by a target process in a first version of a control group Cgroup v1, determining a target controller corresponding to the target page memory based on a first correspondence relationship, the target page memory being generated when the target process performs cache reading and writing, the Cgroup v1 being pre-configured to enable write-back restrictions on cache reading and writing, the first correspondence relationship being determined based on a controller memory cgroup of the control group to which the target process belongs; determining a target subsystem corresponding to the target controller based on a second correspondence relationship, the second correspondence relationship being determined based on a subsystem controller blkcg_css of the control group to which the target process belongs when generating the target page memory, the target subsystem being a subsystem blkio cgroup corresponding to the blkcg_css; and limiting a write-back speed of data written back to the target page memory to the target memory based on the target subsystem.

[0005] In a second aspect, an embodiment of the present application provides a speed limiting device for reading and writing data, the device comprising: a target controller determination module, used to determine, when writing back a target page memory generated by a target process in a first version of a control group Cgroup v1, a target controller corresponding to the target page memory based on a first correspondence relationship, the target page memory being generated when the target process performs cache reading and writing, the Cgroup v1 being pre-configured to enable write-back restrictions on cache reading and writing, the first correspondence relationship being determined based on a controller memorycgroup of the control group to which the target process belongs; a target subsystem determination module, used to determine, based on a second correspondence relationship, a target subsystem corresponding to the target controller, the second correspondence relationship being determined based on a subsystem controller blkcg_css of the control group to which the target process belongs when generating the target page memory, the target subsystem being a subsystem blkio cgroup corresponding to the blkcg_css; a speed limiting module, used to limit a write-back speed of data written back to the target page memory to the target memory based on the target subsystem.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the data reading and writing speed limiting method provided in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the speed limiting method for reading and writing data provided in the first aspect above.

[0008] The solution provided by the present application, when the write-back restriction of cache reading and writing is enabled for Cgroup v1, the target controller to which the target process currently belongs is obtained based on the first correspondence between the controller and the control group to which the target process belongs, and then the target subsystem to which the target process currently belongs is determined based on the second correspondence between the controller and the subsystem controller, and finally the write-back speed of writing data in the target page memory back to the target memory is realized according to the target subsystem. By limiting the write-back speed of the target page memory by determining the target subsystem corresponding to the target process when the write-back restriction of cache reading and writing is enabled for Cgroup v1, the limitation that the write-back speed of the page memory cannot be limited in Cgroup v1 is broken, and the speed limit of cache reading and writing in Cgroup v1 is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram showing the internal management hierarchy of the control group of the first version is shown.

[0011] Figure 2 A schematic flow chart of a method for limiting the speed of data reading and writing provided in an embodiment of the present application is shown.

[0012] Figure 3 A schematic flow chart of a method for limiting the speed of data reading and writing provided in another embodiment of the present application is shown.

[0013] Figure 4 A schematic diagram of the process of establishing a first corresponding relationship and a second corresponding relationship provided in another embodiment of the present application is shown.

[0014] Figure 5 A schematic flow chart of a method for limiting the speed of data reading and writing provided in yet another embodiment of the present application is shown.

[0015] Figure 6 A schematic diagram of the kernel flow of enabling the control group write-back function of the Linux kernel is shown.

[0016] Figure 7 The figure shows a flow chart of limiting the write-back speed of Buffered IO in the Linux kernel.

[0017] Figure 8 A structural block diagram of a speed limiting device for reading and writing data provided in an embodiment of the present application is shown.

[0018] Fig. 9 A structural block diagram of an electronic device provided by an embodiment of the present application for executing a speed limiting method for reading and writing data according to an embodiment of the present application is shown.

[0019] Fig.10 A storage medium provided by an embodiment of the present application and used to store or carry program codes for implementing a speed limiting method for reading and writing data according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0021] See also Figure 1 , Figure 1 Figure 1 shows a schematic diagram of the internal management hierarchy of the first version of the control group. Figure 1 In the cgroup, each level of subsystems or controllers are independent of each other, for example, the CPU subsystem for tracking CPU usage, the controller for managing and limiting the memory resource usage of processes or process groups (memory cgroup), and the blkio cgroup subsystem for managing and controlling device block input and output. Among them, cpuacct manages and limits the use of memory resources of this subsystem by processes or process groups. The memory cgroup is used to manage and limit the use of system memory resources by processes or process groups. The memory cgroup provides a mechanism to monitor and control the memory allocation, use, and release of processes to achieve resource management and performance optimization. The blkio cgroup is used to limit and monitor the input and output operations of block devices. The blkio cgroup allows system administrators to limit the I / O operations of different processes or process groups to ensure fair allocation of system resources and performance optimization. By using the blkio cgroup, users can assign specific I / O bandwidth and I / O priority to each cgroup. This prevents a process or process group from occupying too much disk bandwidth, thereby affecting the performance of other processes.

[0022] Due to the design problem of Cgroup v1, when the dirty pages generated by Buffered IO are written back in units of pages, they can only be traced back to the memory cgroup to which the page memory belongs, but the associated blkio cgroup is unknown, so the write-back cannot be speed-limited. When the memory cgroup and the blkio cgroup do not cooperate, the blkio cgroup determined by the memory cgroup is the root block IO control group (root blkio cgroup, root blkcg), and the root blkcg does not limit the speed of Buffered IO write-back by default, which means that the writing of dirty pages in Cgroup v1 will not be speed-limited. When using Buffered IO to write large files, a large number of dirty pages will be generated quickly, but the dirty page write-back is slow, resulting in excessive memory usage. Under normal circumstances, when the dirty page write-back speed is much lower than the dirty page generation speed, the process should be forced to sleep and the dirty page generation speed should be rate-limited.

[0023] In response to the problems in the background technology, the inventors have proposed a data reading and writing speed limit method, device, electronic device and storage medium provided in the embodiments of the present application. When the write-back restriction of cache reading and writing is enabled for Cgroup v1, the target controller to which the target process currently belongs is obtained based on the first correspondence between the controller and the control group where the target process is located, and then the target subsystem to which the target process currently belongs is determined based on the second correspondence between the controller and the subsystem controller, and finally the write-back speed when writing data in the target page memory back to the target memory is realized according to the target subsystem. By limiting the write-back speed of the target page memory by determining the target subsystem corresponding to the target process when the write-back restriction of cache reading and writing is enabled for Cgroup v1, the limitation that the write-back speed of the page memory cannot be limited in Cgroup v1 is broken, and the speed limit of cache reading and writing in Cgroup v1 is realized.

[0024] The following will describe in detail the data reading and writing speed limiting method, device, electronic device and storage medium provided in the embodiments of the present application in conjunction with the accompanying drawings.

[0025] See also Figure 2 , Figure 2 The flow chart of the data reading and writing speed limiting method provided by the embodiment of the present application is shown. The data reading and writing speed limiting method can be applied to the following Figure 8 The data reading and writing speed limiting device 300 and the electronic device 100 equipped with the data reading and writing speed limiting device 300 are shown. Fig. 9 ).

[0026] The data reading and writing speed limiting method may specifically include the following steps:

[0027] Step S110: when writing back a target page memory generated by a target process in a first version of a control group Cgroup v1, determining a target controller corresponding to the target page memory based on a first correspondence, wherein the target page memory is generated when the target process performs cache reading and writing, and the Cgroup v1 is pre-configured to enable write-back restrictions on cache reading and writing, and the first correspondence is determined according to the controller memorycgroup of the control group to which the target process belongs.

[0028] Among them, the control group (Cgroup) is a function in the Linux kernel that is used to limit resources and manage process groups. Cgroup v1 is the first version of Cgroup. In Cgroup v1, each Cgroup is a directory that can contain child Cgroups and processes. By setting parameters in the files in the Cgroup directory, resource restrictions can be imposed on the process groups in the Cgroup, such as the central processing unit (CPU), memory, disk input / output (I / O), etc.

[0029] Buffered input / output (Buffered IO) uses the Linux kernel's buffer to transfer data. In Buffered IO, data is first read into the kernel's buffer and then copied from the buffer to the application's memory space, or data is first read from the application's memory space into the kernel's buffer and then copied from the kernel's buffer to the application. Due to the use of buffers, Buffered IO can provide higher performance and throughput because data can be processed in batches in memory, reducing frequent access to block devices (devices that provide persistent data storage, such as hard drives, solid-state drives (SSDs), or flash devices). Buffered IO can also perform functions such as random access and file positioning.

[0030] Furthermore, the buffer is a portion of the memory space used to temporarily store data transmitted from one place to another. In short, the buffer is a reserved memory space used to buffer input or output data. The buffer can be a page storage area or a cache file. In the embodiment of the present application, the buffer is preferably a page memory.

[0031] Since the subsystems in Cgroup v1 are independent of each other, when Buffered IO is used for a process or process group in Cgroup v1, the page memory generated by Buffered IO is written back in units of pages, and can only be traced back to the memory cgroup corresponding to the page memory. The write-back speed of the page memory cannot be limited by the memory cgroup. Therefore, in order to implement Cgroup v1 to limit the write-back speed of Buffered IO page memory, it is necessary to enable Cgroup v1 to limit the write-back speed of Buffered IO. The enabling method can be to modify the Linux kernel so that Cgroup v1 enables the write-back limit for Buffered IO.

[0032] In the embodiment of the present application, the first correspondence relationship refers to the correspondence relationship between the control group where the target process is located and the controller. When the corresponding target page memory is generated in the target process, the controller corresponding to the control group where the target process is located is used as the target controller corresponding to the target page memory.

[0033] Step S120: Based on a second corresponding relationship, determine a target subsystem corresponding to the target controller, wherein the second corresponding relationship is determined based on a subsystem controller blkcg_css of a control group to which the target process belongs when generating the target page memory, and the target subsystem is a subsystem blkio cgroup corresponding to the blkcg_css.

[0034] Among them, the subsystem controller blkcg_css (Block IO Controller Group Control Set) is a subsystem of the subsystem blkio cgroup, which is used to manage and configure blkio cgroup. blkcg_css allows users to create, delete, and manage block I / O control groups, as well as assign resource limits and parameters to these control groups. In Linux, one blkcg_css corresponds to one blkio cgroup.

[0035] In the embodiment of the present application, after the blkcg_css corresponding to the target controller is determined according to the second corresponding relationship, the target subsystem corresponding to the target controller can be determined according to the blkcg_css.

[0036] Step S130: Based on the target subsystem, limiting the writing speed of writing back the data in the target page memory to the target memory.

[0037] In the embodiment of the present application, the subsystem blkio cgroup can be used to limit and monitor the I / O operation of the block device. Therefore, the administrator can limit the write back of the target page memory according to the restriction information in the blkio cgroup.

[0038] For example, when writing back large files at high speed, the write-back speed of the block device is limited in combination with the memory limit. For example, the memory limit is 100MB, and the write-back limit of the block device is 10MBps. When writing a 1GB large file, because the process generates dirty pages too quickly, it is limited by the kernel dirty page balance (balance_dirty_pages_ratelimited) and then pauses, waiting for the write-back thread to write back to release the dirty pages. The process will always be suppressed by the memory speed limit, and will be paused after generating dirty pages too quickly, so it takes a long time to exit, and the write-back thread writing back dirty pages is also subject to the precise speed limit of block IO, which almost degenerates Buffered IO into Direct IO.

[0039] The solution provided in this embodiment, when enabling write-back restriction of cache read and write for Cgroup v1, obtains the target controller to which the target process currently belongs based on the first correspondence between the controller and the control group to which the target process belongs, and then determines the target subsystem to which the target process currently belongs based on the second correspondence between the controller and the subsystem controller, and finally implements the write-back speed when writing data in the target page memory back to the target memory according to the target subsystem. By limiting the write-back speed of the target page memory by determining the target subsystem corresponding to the target process when enabling write-back restriction of cache read and write for Cgroup v1, the limitation that the write-back speed of the page memory cannot be limited in Cgroup v1 is broken, and the versatility of the application of Cgroup v1 is improved. Moreover, since the second correspondence is established in real time when the target page memory is generated, the flexibility and accuracy of establishing the second correspondence are improved.

[0040] See also Figure 3 , Figure 3 A schematic flow chart of a method for limiting the speed of data reading and writing provided in another embodiment of the present application is shown.

[0041] Step S210: When the target process performs cache reading and writing, the target page memory is generated, and memory accounting is performed on the target page memory to record the correspondence between the target process and the target controller as the first correspondence.

[0042] Among them, memory accounting is used to track and record the amount of memory used by processes in each cgroup. Therefore, memory accounting can be used to track and record the amount of memory used by the target page memory. Then, the target controller corresponding to the target process can be adjusted according to the amount of memory used by the target page memory. Finally, when the memory accounting of the target page memory is submitted, the target controller corresponding to the target process is determined, and then the corresponding relationship between the target process and the target controller is determined as the first corresponding relationship.

[0043] Step S220: When the target process performs cache reading and writing, the target page memory is generated, and the target control group to which the target process currently belongs is determined.

[0044] Step S230: Determine the memory cgroup to which the target process currently belongs and the blkcg_css to which the target process currently belongs according to the control group identifier corresponding to the target control group.

[0045] The control group identifier includes but is not limited to a controller identifier, a subsystem identifier, a subsystem controller identifier and other identifiers.

[0046] Since in Cgroup v1, resource control and management are performed on process groups in units of control groups, the subsystem and / or subsystem controller corresponding to the target process can be determined according to the control group identifier corresponding to the target control group to which the target process belongs.

[0047] Exemplarily, taking the target process p as an example, the control group of the target process p is obtained, the controller identifier corresponding to blkcg_css is io_cgrp_id, and the controller identifier corresponding to the memory cgroup is memory_cgrp_id. The corresponding subsystem is obtained according to the controller identifier and the kernel's call function or query function. For example, the function is the task_css function, and the blkcg_css corresponding to the target process p is obtained according to task_css(p, io_cgrp_id), and the memcg_css corresponding to the target process p is obtained according to task_css(p, memory_cgrp_id). Since in Cgroup v1, memcg_css corresponds to memorycgroup, the memory cgroup corresponding to the target process p is determined according to memcg_css.

[0048] Step S240: determining the memory cgroup to which the target process currently belongs as the target controller, and determining the blkcg_css to which the target process currently belongs as the target subsystem controller, and establishing a correspondence between the target controller and the target subsystem controller as the second correspondence.

[0049] In an embodiment of the present application, after determining the memory cgroup and blkcg_css to which the target process currently belongs, in order to determine the blkio cgroup corresponding to the memory cgroup to which the target process currently belongs to implement write-back restriction on the target process, a correspondence between the memory cgroup and blkcg_css is first established as a second correspondence, and then the blkio cgroup corresponding to the memory cgroup is determined according to the second correspondence.

[0050] In some embodiments, the establishment of the second correspondence further includes: in response to a modification of the memory cgroup to which the target process currently belongs, determining the modified memory cgroup as the target controller. In response to a modification of the blkcg_css to which the target process belongs, obtaining the modified blkcg_css as the target subsystem controller. Determining the modified memory cgroup as the target controller, and determining the modified blkcg_css as the target subsystem controller, and re-establishing the second correspondence.

[0051] In an implementation manner of the present application, when the memory cgroup and blkcg_css corresponding to the target process are modified by the administrator, after the target page memory is generated when the target process performs cache reading and writing, the modified memorycgroup and blkcg_css can be used as the new target controller and target subsystem controller, and finally the second corresponding relationship can be re-established based on the new target controller and target subsystem controller.

[0052] In other feasible implementations, when only one of the memory cgroup and blkcg_css corresponding to the target process is changed by the administrator, after the target page memory is generated when the target process performs cache reading and writing, the modified subsystem controller or controller will be determined as the new target subsystem controller or controller to re-establish the second corresponding relationship, thereby achieving flexible correspondence between the memory cgroup and blkcg_css after the target process generates the target page memory, avoiding improper write-back restrictions caused by the memory cgroup only corresponding to a single blkcg_css, and improving the accuracy of the write-back restrictions on the target page memory.

[0053] Step S250: when writing back a target page memory generated by a target process in a first version of a control group Cgroup v1, determining a target controller corresponding to the target page memory based on a first correspondence, wherein the target page memory is generated when the target process performs cache reading and writing, and the Cgroup v1 is pre-configured to enable write-back restrictions on cache reading and writing, and the first correspondence is determined according to the controller memorycgroup of the control group to which the target process belongs.

[0054] For the detailed description of step S250, please refer to step S110 in the aforementioned embodiment, which will not be repeated here.

[0055] Step S260: Based on the second corresponding relationship, determine the target subsystem controller corresponding to the target controller.

[0056] Step S270: Determine the target subsystem corresponding to the target controller according to the corresponding relationship between the target subsystem controller and the target subsystem.

[0057] In the embodiment of the present application, the subsystem controller corresponds to one subsystem. Therefore, after the target subsystem controller corresponding to the target controller is determined, the target subsystem corresponding to the target controller can be determined according to the target subsystem controller.

[0058] Step S280: Based on the target subsystem, limiting the write-back speed of the data in the target page memory to the target memory.

[0059] For the detailed description of step S280, please refer to step S130 in the aforementioned embodiment, which will not be repeated here.

[0060] For example, see Figure 4 , Figure 4 A schematic diagram of the establishment process of the first correspondence and the second correspondence provided by another embodiment of the present application is shown. When the process pid_x uses Buffered IO to write a file, a dirty page is generated. At this time, memory accounting is performed on the dirty page to record the system memory consumed when the process pid_x uses Buffered IO to write a file. Then the memory cgroup and blkcg_css to which the process pid_x currently belongs are obtained, and the second correspondence is established according to the memory cgroup and blkcg_css. At this time, the memory accounting of pid_x is submitted, and the first correspondence is determined according to the submitted memory accounting.

[0061] The solution provided in this embodiment establishes a second corresponding relationship between the target controller and the target subsystem controller in real time when the target page memory is generated, thereby improving the accuracy of the second corresponding relationship, and further improving the accuracy of the target subsystem determination, making it more accurate to limit the write back speed of the target page memory according to the target subsystem.

[0062] See also Figure 5 , Figure 5 A schematic flow chart of a data reading and writing speed limiting method provided by another embodiment of the present application is shown. The data reading and writing speed limiting method may specifically include the following steps:

[0063] Step S310: Obtain the index node information of the target index node corresponding to the target process.

[0064] In Linux, the index node is used to store file attribute information, including but not limited to the number of bytes of the file, the read, write and execute permissions of the file, the control group information of the file, and other file attribute information.

[0065] In the embodiment of the present application, the index node information of the target index node corresponding to the target process is obtained in order to obtain the relevant file attribute information corresponding to the target process.

[0066] In some implementations, obtaining the index node information of the target index node corresponding to the target process includes: obtaining dirty page count information of the target memory in the target index node without enabling the memory cgroup and the blkio cgroup of the Cgroup v1. Obtaining control group writeback information of the target memory. Obtaining control group writeback information corresponding to the file system in the target index node. Determining the index node information based on the dirty page count information of the target memory, the control group writeback information of the target memory, and the control group writeback information corresponding to the file system.

[0067] In the related art, due to the design of Cgroup v1, the control group write-back function in Cgroup v1 is directly disabled when the memory cgroup and blkio cgroup of Cgroup v1 are enabled. Therefore, when the control group write-back control group function is enabled for Cgroup v1, it is not necessary to enable the memory cgroup and blkiocgroup of Cgroup v1. In the implementation manner of the present application, the method of not enabling the memory cgroup and blkiocgroup of Cgroup v1 can be to directly delete the related functions involved in enabling the memory cgroup and blkiocgroup of Cgroup v1, and compile the deleted Linux kernel to ensure that the memory cgroup and blkio cgroup of Cgroup v1 are not enabled.

[0068] A dirty page refers to a page that has been modified in the buffer but has not been refreshed to the memory or database. The dirty page count information refers to whether the memory supports dirty page counting.

[0069] The file system includes but is not limited to various types of files, data in the files, and information such as the structure of the file system. All files, directories, soft links, and file protection information seen by all Linux users and programs are stored in the file system.

[0070] The control group writeback function can implement writeback restrictions on dirty pages. The control group writeback information of the storage refers to whether the target storage supports the control group writeback function, and the control group writeback information corresponding to the file system refers to whether the file system where the target process is located supports the control group writeback function.

[0071] In the implementation manner of the present application, when the memory cgroup and blkio cgroup of Cgroup v1 are not enabled and determined, it is also determined whether the control group writeback function can be enabled for Cgroup v1 according to the dirty page count information of the target memory corresponding to the target process, the control group writeback information of the memory, and the control group writeback information corresponding to the file system. For example, when the memory cgroup and blkio cgroup of Cgroup v1 are not enabled and determined, when the dirty page count information indicates that the memory supports dirty page count, the control group writeback information of the memory indicates that the memory supports the control group writeback function, and the control group writeback information corresponding to the file system also indicates that the file system supports the control group writeback function, the control group writeback function is enabled for Cgroup v1. If any of the dirty page count information, the control group writeback information of the memory, or the control group writeback information corresponding to the file system indicates that the control group writeback function is not supported, the control group writeback function is not enabled for Cgroup v1. Since, in Linux, writing the data in the dirty page back to the memory is performed periodically, when judging whether the control group writeback function is enabled for Cgroup v1, the judgment may be performed each time the control group writeback is performed, so as to improve the timeliness and accuracy of the judgment result.

[0072] For example, see Figure 6 , Figure 6 The figure shows the kernel flow diagram of enabling the control group writeback function of the Linux kernel. It is necessary to determine whether the memory cgroup and blkio cgroup of Cgroup v1 in the enabling function (inode_cgwb_enabled) of the control group writeback function of the index node are enabled in the default hierarchy. Only when the memory cgroup and the blkio cgroup are both no, will it continue to determine whether the block device supports the dirty page counting function, whether the block device supports the control group writeback function, and whether the file system supports the control group writeback function. Then the control group writeback function of the index node can be enabled. Therefore, in order to enable it, it is necessary to ensure that the memory cgroup and blkio cgroup of Cgroupv1 in the enabling function of the control group writeback function of the index node are enabled in the default hierarchy. The enabling determination step of the memorycgroup and blkio cgroup in the default hierarchy in the enabling function can be deleted, such as Figure 6 The determination step in the dotted box is performed, and the memory cgroup and blkio cgroup of Cgroup v1 are not enabled to ensure that the memory cgroup and blkio cgroup of Cgroup v1 are enabled.

[0073] In one feasible way, when Cgroup v1 enables the control group writeback function, the number of dirty pages corresponding to the current system can be obtained, and whether to continue to generate dirty pages can be determined based on the number of dirty pages. The number of dirty pages can reflect the I / O load of the current storage or database. A high number of dirty pages means that the current system is experiencing a large number of write operations, and these operations have not been persistently written to the storage or database. If the number of dirty pages remains high for a long time, it is easy to cause system performance degradation and even data loss risks. When the system is at a high number of dirty pages, the generation of dirty pages can be suspended to avoid increasing the system burden. When the system is at a low number of dirty pages, the generation of dirty pages will not affect the system, so the normal generation of dirty pages can be maintained. Before the generation of dirty pages, the current number of dirty pages in the system is judged to ensure the normal operation of the system.

[0074] Step S320: when the index node information indicates that the target index node has a control group write-back function enabled, determining that the target process has a cache read-write write-back restriction enabled.

[0075] Step S330: If it is determined that the target process has enabled write-back restriction for cache reads and writes, when writing back the target page memory generated by the target process in the first version of the control group Cgroup v1, a target controller corresponding to the target page memory is determined based on the first corresponding relationship.

[0076] In the embodiment of the present application, after determining that the target process has enabled the write-back restriction of cache read and write according to the index node information corresponding to the target process,

[0077] Step S340: Based on a second correspondence, determine a target subsystem corresponding to the target controller, wherein the second correspondence is determined based on the subsystem controller blkcg_css of the control group to which the target process belongs when generating the target page memory, and the target subsystem is the subsystem blkio cgroup corresponding to the blkcg_css.

[0078] Step S350: Based on the target subsystem, limiting the write-back speed of the data in the target page memory to the target memory.

[0079] For detailed description of steps S340 to S350 , please refer to steps S120 to S130 in the aforementioned embodiment, which will not be repeated here.

[0080] In the embodiment of the present application, the write-back restriction of cache reading and writing of Cgroup v1 is enabled according to the file attribute information in the index node. It only needs to modify the file attribute information (kernel information) to enable the write-back restriction of Cgroup v1, thereby allowing a large number of existing low-version Linux servers to support the IO speed limit function of block devices, thereby improving the versatility of Cgroup v1 use.

[0081] See also Figure 7 , Figure 7 The flowchart of limiting the write-back speed of Buffered IO in the Linux kernel is shown. In combination with the above-mentioned embodiment scheme, when the write-back speed of Buffered IO is written back in the Linux kernel, the write-back process of the kernel creates a control group write-back subsystem cgroup writeback corresponding to the write-back process, and the memory cgroup to which the write-back process currently belongs can be determined according to the write-back process, and the blkcg_css to which the write-back process currently belongs can be obtained. Then the version of the current control group is judged. If the version of the control group is the first version, the blkio cgroup corresponding to the write-back process is determined according to blkcg_css and memorycgroup, and the control group write-back subsystem is speed-limited according to the parameters of the blkio cgroup, and cgroup writeback is bound to blkcg_css, and then in the subsequent write-back process, the parameters of the blkio cgroup are continued to be used to limit the write-back speed. If the version of the control group is the second version, the write-back speed of the Buffered IO is limited according to the write-back speed limiting logic of the control group of the second version.

[0082] See also Figure 8, which shows a structural block diagram of a data reading and writing speed limiting device 300 provided in an embodiment of the present application. The data reading and writing speed limiting device 300 is applied to the electronic device 100, and the data reading and writing speed limiting device 300 includes: a target controller determining module 310, which is used to determine the target controller corresponding to the target page memory based on a first corresponding relationship when writing back the target page memory generated by the target process in the first version of the control group Cgroup v1, the target page memory is generated when the target process performs cache reading and writing, the Cgroup v1 is pre-configured to enable the write-back restriction of cache reading and writing, and the first corresponding relationship is determined according to the controller memory cgroup of the control group to which the target process belongs; a target subsystem determining module 320, which is used to determine the target subsystem corresponding to the target controller based on a second corresponding relationship, the second corresponding relationship is determined based on the subsystem controller blkcg_css of the control group to which the target process belongs when generating the target page memory, and the target subsystem is the subsystem blkio cgroup corresponding to the blkcg_css; a speed limiting module 330, which is used to limit the write-back speed of the data written back to the target page memory to the target memory based on the target subsystem.

[0083] In some embodiments of the present application, the speed limiting device 300 for data reading and writing further includes: a target control group determination module, which is used to generate the target page memory when the target process performs cache reading and writing, and determine the target control group to which the target process currently belongs; an information determination module, which is used to determine the memory cgroup to which the target process currently belongs and the blkcg_css to which the target process currently belongs according to the control group identifier corresponding to the target control group; a second corresponding relationship establishment module, which is used to determine the memory cgroup to which the target process currently belongs as the target controller, and determine the blkcg_css to which the target process currently belongs as the target subsystem controller, and establish a corresponding relationship between the target controller and the target subsystem controller as the second corresponding relationship.

[0084] In some embodiments of the present application, the speed limiting device 300 for reading and writing data further includes: a target controller modification module, used to determine the modified memorycgroup as the target controller in response to the modification of the memory cgroup to which the target process currently belongs; a target subsystem controller modification module, used to obtain the modified blkcg_css as the target subsystem controller in response to the modification of the blkcg_css to which the target process belongs; a second correspondence re-establishment module, used to determine the modified memory cgroup as the target controller, and determine the modified blkcg_css as the target subsystem controller, and re-establish the second correspondence.

[0085] In some embodiments of the present application, the speed limiting device 300 for data reading and writing also includes: a first correspondence recording module, which is used to generate the target page memory when the target process performs cache reading and writing, and perform memory accounting on the target page memory to record the correspondence between the target process and the target controller as the first correspondence.

[0086] In some embodiments of the present application, the target subsystem determination module 320 includes: a target subsystem controller determination unit, used to determine the target subsystem controller corresponding to the target controller based on the second correspondence relationship; a target subsystem determination unit, used to determine the target subsystem corresponding to the target controller according to the correspondence relationship between the target subsystem controller and the target subsystem.

[0087] In some embodiments of the present application, the data reading and writing speed limit device 300 also includes: an index node information acquisition module, which is used to obtain the index node information of the target index node corresponding to the target process; a write back restriction enabling module, which is used to determine that the target process has enabled the write back restriction of cache reading and writing when the index node information indicates that the target index node has enabled the control group write back function; the target controller determination module 310 also includes: a target controller determination unit, which is used to determine the target controller corresponding to the target page memory based on the first corresponding relationship when writing back the target page memory generated by the target process in the first version of the control group Cgroup v1 if it is determined that the target process has enabled the write back restriction of cache reading and writing.

[0088] In some embodiments of the present application, the index node information acquisition module includes: a dirty page count information acquisition unit of the target memory, a speed limit module for reading and writing image evaluation data, which is used to obtain the dirty page count information of the target memory in the target index node without enabling the memorycgroup and blkio cgroup of the Cgroup v1; a control group write back information acquisition unit of the target memory, which is used to obtain the control group write back information of the target memory; a control group write back information acquisition unit of the file system, which is used to obtain the control group write back information corresponding to the file system in the target index node; an index node information determination unit, which is used to determine the index node information based on the dirty page count information of the target memory, the control group write back information of the target memory, and the control group write back information corresponding to the file system.

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0090] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0091] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0092] A structural block diagram of an electronic device provided in an embodiment of the present application. Please refer to Fig. 9 , which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 can be an electronic device such as a computer, a server, etc. that can run an application. The electronic device 100 in the present application may include one or more of the following components: a processor 101, a memory 102, and one or more applications, wherein the one or more applications may be stored in the memory 102 and configured to be executed by one or more processors 101, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0093] The processor 101 may include one or more processing cores. The processor 101 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 102, and calling data stored in the memory 102. Optionally, the processor 101 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 101 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 101, but may be implemented separately through a communication chip.

[0094] The memory 102 may include a random access memory (RAM) or a read-only memory (ROM). The memory 102 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0095] Please refer to Fig.10 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 200 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0096] The computer-readable storage medium 200 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 200 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 200 has storage space for program code 210 that performs any method steps of the above method. These program codes can be read from or written to one or more computer program products. The program code 210 can be compressed, for example, in an appropriate form.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data reading and writing speed limiting method, characterized in that: The method comprises: When writing back a target page memory generated by a target process in a first version of a control group Cgroup v1, determining a target controller corresponding to the target page memory based on a first correspondence, the target page memory is generated when the target process performs cache reading and writing, the Cgroup v1 is pre-configured to enable write-back restriction of cache reading and writing, and the first correspondence is determined according to a controller memory cgroup of the control group to which the target process belongs; Based on a second corresponding relationship, determining a target subsystem corresponding to the target controller, wherein the second corresponding relationship is determined based on a subsystem controller blkcg_css of a control group to which the target process belongs when generating the target page memory, and the target subsystem is a subsystem blkio cgroup corresponding to the blkcg_css; Based on the target subsystem, a write-back speed of the data in the target page memory to the target memory is limited.

2. The method according to claim 1, characterized in that When writing back the target page memory generated by the target process in the first version of the control group Cgroupv1, before determining the target controller corresponding to the target page memory based on the first corresponding relationship, the method further includes: When the target process performs cache reading and writing, the target page memory is generated, and the target control group to which the target process currently belongs is determined; According to the control group identifier corresponding to the target control group, determine the memorycgroup to which the target process currently belongs and the blkcg_css to which the target process currently belongs; The memory cgroup to which the target process currently belongs is determined as the target controller, and the blkcg_css to which the target process currently belongs is determined as the target subsystem controller, and a corresponding relationship between the target controller and the target subsystem controller is established as the second corresponding relationship.

3. The method according to claim 2, characterized in that After determining the memorycgroup to which the target process currently belongs as the target controller, determining the blkcg_css to which the target process currently belongs as the target subsystem controller, and establishing a correspondence between the target controller and the target subsystem controller as the second correspondence, the method further includes: In response to a modification of a memory cgroup to which the target process currently belongs, determining the modified memory cgroup as the target controller; In response to the modification of blkcg_css to which the target process belongs, obtaining the modified blkcg_css as the target subsystem controller; The modified memory cgroup is determined as the target controller, and the modified blkcg_css is determined as the target subsystem controller, and the second corresponding relationship is re-established.

4. The method according to claim 1, characterized in that: Before determining the target controller corresponding to the target page memory based on the first corresponding relationship, the method includes: The target page memory is generated when the target process performs cache reading and writing, and memory accounting is performed on the target page memory to record the corresponding relationship between the target process and the target controller as the first corresponding relationship.

5. The method according to claim 1, characterized in that The determining, based on the second corresponding relationship, a target subsystem corresponding to the target controller includes: Based on the second corresponding relationship, determining a target subsystem controller corresponding to the target controller; The target subsystem corresponding to the target controller is determined according to the corresponding relationship between the target subsystem controller and the target subsystem.

6. The method according to any one of claims 1 to 5, characterized in that: When writing back the target page memory generated by the target process in the first version of the control group Cgroup v1, before determining the target controller corresponding to the target page memory based on the first corresponding relationship, the method further includes: Obtaining index node information of a target index node corresponding to the target process; In a case where the index node information indicates that the target index node has a control group write-back function enabled, determining that the target process has a cache read-write write-back restriction enabled; When writing back the target page memory generated by the target process in the first version of the control group Cgroup v1, determining the target controller corresponding to the target page memory based on the first corresponding relationship includes: If it is determined that the target process has enabled write-back restriction for cache reading and writing, when writing back the target page memory generated by the target process in the first version of the control group Cgroup v1, the target controller corresponding to the target page memory is determined based on the first corresponding relationship.

7. The method according to claim 6, characterized in that The obtaining the index node information of the target index node corresponding to the target process includes: Without enabling and determining the memory cgroup and the blkio cgroup of the Cgroup v1, obtaining dirty page count information of the target memory in the target index node; Acquire control group write-back information of the target memory; Obtaining write-back information of a control group corresponding to the file system in the target index node; The index node information is determined according to the dirty page count information of the target storage, the control group write-back information of the target storage, and the control group write-back information corresponding to the file system.

8. A data reading and writing speed limiting device, characterized in that: The device comprises: a target controller determination module, configured to determine, when writing back a target page memory generated by a target process in a first version of a control group Cgroup v1, a target controller corresponding to the target page memory based on a first correspondence, wherein the target page memory is generated when the target process performs cache reading and writing, the Cgroup v1 is pre-configured to enable write-back restriction of cache reading and writing, and the first correspondence is determined according to a controller memory cgroup of the control group to which the target process belongs; a target subsystem determination module, configured to determine a target subsystem corresponding to the target controller based on a second corresponding relationship, wherein the second corresponding relationship is determined based on a subsystem controller blkcg_css of a control group to which the target process belongs when generating the target page memory, and the target subsystem is a subsystem blkiocgroup corresponding to the blkcg_css; The speed limiting module is used to limit the writing speed of the data in the target page memory to the target memory based on the target subsystem.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images