Data scheduling method, apparatus, and electronic device

Abstract

The application provides a data scheduling method and device and electronic equipment. The embodiment automatically identifies a cold unit memory (referred to as a cold unit memory) in a first memory, whose access frequency per unit time is less than a first set threshold, and a hot unit memory (referred to as a hot unit memory) in a second memory, whose access frequency per unit time is greater than or equal to a second set threshold, automatically schedules data in the hot unit memory in the second memory to move to an idle unit memory in the first memory, and automatically schedules data in the cold unit memory in the first memory to move to an idle unit memory in the second memory, so that data with high access frequency is stored in the local first memory of the system, and data with low access frequency is stored in the second memory which is expanded, the advantages of local memory high-speed access and memory capacity expansion are maximized, and the data access efficiency is improved.

Description

Data scheduling method, device and electronic equipment TECHNICAL FIELD

[0001] The present application relates to data storage technology, in particular to a data scheduling method, device and electronic equipment. BACKGROUND

[0002] At present, the expansion memory technology such as the PCIe-based interconnection standard (CXL: Compute Express Link) for memory expansion provides a method for dynamically expanding the computing memory. For example, when the CPU computing in the data storage system needs larger capacity memory, the memory is expanded based on the above expansion memory technology, so that the expanded memory and the local memory of the system are combined to meet the large capacity memory requirement.

[0003] For the local components of the data storage system such as CPU, the delay of accessing the expanded memory is much larger than that of accessing the local memory. Taking the CPU accessing the expanded CXL memory as an example, the CPU needs to pass through the CXL interface chip, CXL Switch and other devices to access the CXL memory, which makes the access delay of the CXL memory usually 100-400ns larger than that of the local memory. In the high-frequency service scenario, the access delay of the above-mentioned expanded memory such as CXL memory will greatly reduce the service running efficiency.

[0004] SUMMARY

[0005] The present application provides a memory scheduling method, device and electronic equipment to improve data access efficiency by automatically scheduling data stored in the memory.

[0006] The present application provides a data scheduling method, which is applied to a data storage system, the data storage system comprising a local first memory and an expanded second memory; the first memory and the second memory are each divided into at least two unit memories; the method comprises:

[0007] obtaining a first target unit memory that needs to be moved; wherein the first target unit memory belongs to the first memory, and the access frequency of the first target unit memory in a unit time is less than a first set threshold; or the first target unit memory belongs to the second memory, and the access frequency of the first target unit memory in a unit time is greater than or equal to a second set threshold; the second set threshold is greater than the first set threshold;

[0008] obtaining an idle second target unit memory; when the first target unit memory belongs to the first memory, the second target unit memory belongs to the second memory; when the first target unit memory belongs to the second memory, the second target unit memory belongs to the first memory;

[0009] migrate data stored in the first target unit memory to the second target unit memory, and record migration information;

[0010] when receiving an access request sent from an application end, based on the recorded migration information and a first virtual address carried in the access request and assigned to the application end for identifying the first target unit memory, control the access to the first target unit memory to be converted to the access to the second target unit memory.

[0011] Embodiments of the present application provide a data scheduling device, which is applied to a data storage system, the data storage system comprising a local first memory and an extended second memory; the first memory and the second memory are both divided into at least two unit memories; the device comprises:

[0012] an obtaining unit, configured to obtain a first target unit memory; wherein the first target unit memory belongs to the first memory and the access frequency of the first target unit memory in a unit time is less than a first set threshold; or the first target unit memory belongs to the second memory and the access frequency of the first target unit memory in a unit time is greater than or equal to a second set threshold; the second set threshold is greater than the first set threshold; and

[0013] obtain an idle second target unit memory; when the first target unit memory belongs to the first memory, the second target unit memory belongs to the second memory; when the first target unit memory belongs to the second memory, the second target unit memory belongs to the first memory;

[0014] a migration unit, configured to migrate data stored in the first target unit memory to the second target unit memory, and record migration information;

[0015] an address conversion unit, configured to, when receiving an access request sent from an application end, based on the recorded migration information and a first virtual address carried in the access request and assigned to the application end for identifying the first target unit memory, control the access to the first target unit memory to be converted to the access to the second target unit memory.

[0016] Embodiments of the present application provide an electronic device, which comprises a processor and a machine readable storage medium; the machine readable storage medium stores computer instructions, when the computer instructions are executed by the processor, the steps in the above method are executed.

[0017] From the above technical solutions can be seen, the embodiment automatically identifies the unit memory in the first memory whose access frequency in a unit time is less than a first set threshold (referred to as cold unit memory), the unit memory in the second memory whose access frequency in a unit time is greater than or equal to a second set threshold (referred to as hot unit memory), automatically schedules data in the hot unit memory in the second memory to move to the idle unit memory in the first memory, and automatically schedules data in the cold unit memory in the first memory to move to the idle unit memory in the second memory, so that the data with high access heat is stored in the local first memory of the system, and the data with low access heat is stored in the second memory which is expanded, the advantages of local memory high-speed access and memory capacity expansion are maximized, the efficient access of the data with high access heat is ensured, the data access efficiency is improved, and the reasonable use of the second memory which is expanded is also ensured.

[0018] Further, based on the recorded information of data movement between different unit memories (i.e., movement information) and the first virtual address for identifying the first target unit memory carried by the access request sent by the application end, the embodiment controls the access to the first target unit memory to be converted to the access to the second target unit memory, the access is not perceived by the application end, and the data movement of the first target unit memory to the second target unit memory is also not perceived by the application end, without the need to modify the application end, so that the application end can smoothly run in the environment with or without the expanded memory. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0020] FIG. 1 is a method flowchart provided by an embodiment of the present application;

[0021] FIG. 2 is an application structure diagram provided by Embodiment 1 of the present application;

[0022] FIG. 3 is a threshold value updating flowchart provided by Embodiment 1 of the present application;

[0023] FIG. 4 is a data scheduling flowchart provided by Embodiment 1 of the present application;

[0024] FIG. 5 is a memory access schematic diagram provided by Embodiment 1 of the present application;

[0025] FIG. 6 is an application structure diagram provided by Embodiment 2 of the present application;

[0026] FIG. 7 is a data scheduling flowchart provided by Embodiment 2 of the present application;

[0027] FIG. 8 is a memory access schematic diagram provided by Embodiment 2 of the present application;

[0028] FIG. 9 is a structure diagram of an apparatus provided by an embodiment of the present application;

[0029] FIG. 10 is a structure diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] The exemplary embodiments will be described in detail herein below with reference to the drawings. The following description is with reference to the drawings, in which like numerals refer to like elements throughout the several figures. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples with respect to apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0032] In order to make the skilled in the art better understand the technical solutions provided by the embodiments of the present application, and make the above-mentioned purposes, characteristics and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the drawings.

[0033] Referring to FIG. 1, FIG. 1 is a flow chart of a method provided by an embodiment of the present application. The method is applied to a data storage system, which includes a local first memory such as a system local RAM and an extended second memory such as a CXL memory. The first memory and the second memory are both divided into at least two unit memories. The storage space of any unit memory is a unit storage space. Optionally, the unit storage space can be a page granularity such as 4KB, etc., which is not specifically limited in the present embodiment.

[0034] As shown in FIG. 1, the flow can include the following steps:

[0035] In step 101, a first target unit memory which needs to be data moved is obtained; wherein the first target unit memory belongs to the first memory, and the access frequency of the first target unit memory in a unit time is less than a first set threshold; or the first target unit memory belongs to the second memory, and the access frequency of the first target unit memory in a unit time is greater than or equal to a second set threshold; the second set threshold is greater than the first set threshold.

[0036] In the embodiment, when the access frequency in a unit of time in any unit is less than a first set threshold, it indicates that the access frequency of the unit is the first access frequency. When the access frequency of any unit is the first access frequency, it indicates that the data stored in the unit is cold data (data that does not need to be frequently accessed). Generally, the access frequency of cold data is relatively low, and the cold data is not important for business or application and needs to be stored for a long time.

[0037] In the embodiment, when the access frequency in a unit of time in any unit is greater than or equal to a second set threshold, it indicates that the access frequency of the unit is the second access frequency. When the access frequency of any unit is the second access frequency, it indicates that the data stored in the unit is hot data (data that needs to be frequently accessed). Generally, the access frequency of hot data is high, and the hot data is critical for business or application and needs to be quickly and efficiently accessed or processed.

[0038] In step 102, idle second target unit memory is obtained.

[0039] In the embodiment, the second target unit memory refers to a destination to which the data in the first target unit memory needs to be moved. When the first target unit memory belongs to the first memory, the second target unit memory belongs to the second memory; when the first target unit memory belongs to the second memory, the second target unit memory belongs to the first memory.

[0040] In step 103, the data stored in the first target unit memory is moved to the second target unit memory, and the moving information is recorded.

[0041] Optionally, in the embodiment, the moving method can be memory copying or DMA mode, and the embodiment is not specifically limited.

[0042] In step 104, when an access request sent from an application end is received, based on the recorded moving information and the first virtual address used to identify the first target unit memory allocated to the application end carried by the access request, the access to the first target unit memory is converted to the access to the second target unit memory.

[0043] In the embodiment, after any application end is started, the corresponding unit memory is dynamically allocated to the application end, and the unit memory can be in the first memory or the second memory, and the embodiment is not specifically limited. In the embodiment, the unit memory allocated to the application end is represented by a virtual address, and the virtual address of any unit memory corresponds to (also referred to as maps) the physical address of the unit memory, and is used to represent the unit memory. For example, the first target unit memory has a first virtual address, and the first virtual address has a corresponding (also referred to as mapping) relationship with the physical address (denoted as a first physical address) of the first target unit memory.

[0044] When any application end is assigned a corresponding unit memory such as the first target unit memory, it sends an access request to request access to the unit memory it is assigned, such as the first target unit memory. Based on this, the application is applied to step 104, when receiving the access request sent by any application end, it is found that the access request carries the first virtual address assigned by the application end to identify the first target unit memory, and based on the recorded migration information of the data stored in the first target unit memory to the second target unit memory, the application end's original access to the first target unit memory can be converted to access to the second target unit memory. The conversion is not perceived by the application.

[0045] So far, the process shown in FIG. 1 is completed.

[0046] As can be seen from the process shown in FIG. 1, the embodiment automatically identifies the unit memory in the first memory whose access frequency in a unit time is less than the first set threshold (referred to as cold unit memory), the unit memory in the second memory whose access frequency in a unit time is greater than or equal to the second set threshold (referred to as hot unit memory), automatically schedules the data migration in the hot unit memory in the second memory to the idle unit memory in the first memory, and automatically schedules the data migration in the cold unit memory in the first memory to the idle unit memory in the second memory, so that the data with high access frequency is stored in the local first memory of the system, and the data with low access frequency is stored in the extended second memory, maximizing the advantages of local memory high-speed access and extended memory capacity expansion, ensuring efficient access to data with high access frequency, improving data access efficiency, and also ensuring reasonable use of the extended second memory.

[0047] Further, based on the recorded migration information between different unit memories (i.e. migration information) and the first virtual address carried by the access request sent by the application end, the application end is assigned to identify the first target unit memory, the application end is controlled to convert the access to the first target unit memory to the access to the second target unit memory, which is not perceived by the application end, and the data migration of the first target unit memory to the second target unit memory is also not perceived by the application end, without modifying the application end, ensuring that the application end can run smoothly in the environment with or without extended memory.

[0048] It should be noted that, as an embodiment, the unit memory allocated to the application end is allocated based on the memory information of the memory allowed to be used by the application end. For example, personalized memory allocation can be performed for a specific application end: for example, if the application end runs an application with low memory response requirement, at least one unit memory of the application end can be allocated from the extended memory with slow read-write speed, i.e., the second memory, so as to leave the local memory with fast response speed to the application end running a critical application; or, the proportion of the first memory and the second memory used by a specific application running on the application end can be allocated, so as to realize that some critical applications use the first memory and some non-critical applications use the second memory based on actual demand. Of course, in this embodiment, the proportion of the first memory and the second memory used by each application can also be a default value, for example, the proportion of the first memory is 50% and the proportion of the second memory is 50%, which is not specifically limited in this embodiment.

[0049] Based on this, after the application running on any application end is started, at least one unit memory can be allocated to the application end based on the memory allocation information of the application.

[0050]

[0051] The flow shown in FIG. 1 is described below through two specific embodiments:

[0052] Embodiment 1:

[0053] Embodiment 1 is applied to a scenario of implementing the flow shown in FIG. 1 through a software address translator. The software implementation mode has no modification requirement for the current hardware, and the modification is limited to the implementation on the software side, for example, a software address translator is deployed between the CPU and the operating system (OS) in the above-mentioned data storage system, or the software address translator is integrated on the OS in the data storage system. FIG. 2 takes the integration of the software address translator on the OS as an example for illustration.

[0054] In this embodiment, the software address translator or other components in the data storage system can count the access frequency of each unit memory based on a heat statistics period T. For example, the access frequency of data in each unit memory in the recent unit time (which can be set) is counted, and if the access frequency is greater than or equal to a second set threshold (denoted as K), it is determined that the unit memory is a hot unit memory (i.e., the first target unit memory described above); if the access frequency is less than a first set threshold (denoted as L), it is determined that the unit memory is a cold unit memory (i.e., the first target unit memory described above). If the access frequency is between K and L, it is determined that the unit memory is a warm unit memory.

[0055] In this embodiment, T, K, L and the unit time can be dynamically and adaptively modified. For example, T and the unit time can be modified through a command line. K and L can be modified according to the actual use of the memory.

[0056] The following is an example of how to adaptively modify K and L:

[0057] Referring to FIG. 3, which is a flowchart of a threshold updating process according to an embodiment of the present application, the process can include the following steps:

[0058] At step 301, the usage of the local first memory is monitored. If the usage is greater than or equal to a first usage threshold, step 302 is performed. If the usage is less than the first usage threshold, step 304 is performed.

[0059] The first usage threshold can be set according to actual needs, such as 80%, and the present embodiment is not specifically limited.

[0060] At step 302, if the usage of the second memory is less than a second usage threshold, step 303 is performed.

[0061] The second usage threshold is lower than the first usage threshold. For example, the second usage threshold can be 20%, and the present embodiment is not specifically limited.

[0062] At step 303, K and L are increased.

[0063] Step 303 is performed on the premise that the usage of the local first memory is higher than the first usage threshold and the usage of the second memory is less than the second usage threshold. This premise means that the first memory is almost fully used and the usage of the second memory is very low. Under this premise, the present embodiment can increase K and L to ensure that as much data as possible is stored in the second memory.

[0064] Alternatively, K and L can be increased synchronously, such as by being increased by a set value or by a corresponding multiple, or can be increased asynchronously, such as by increasing K by m and L by n. m and n are respectively the step sizes of the adaptive adjustment of the first set threshold K and the second set threshold L, which are built in the system and can be adjusted by configuration after the system is running.

[0065] Of course, if the usage of the second memory is greater than or equal to a third usage threshold at step 302, K and L can be maintained as they are, as an embodiment. Alternatively, the third usage threshold is greater than the second usage threshold, such as the third usage threshold being the same as the first usage threshold.

[0066] At step 304, if the usage of the second memory is greater than or equal to a third usage threshold, step 305 is performed.

[0067] At step 305, K and L are decreased.

[0068] The step 305 is executed under the premise that the usage of the local first memory is lower than the first usage threshold and the usage of the second memory is greater than or equal to the third usage threshold. The premise means that the usage of the first memory is relatively low and the extended second memory is close to being fully used. Under this premise, the embodiment can reduce the K and L to ensure that as much data as possible is stored in the first memory.

[0069] Alternatively, the K and L can be reduced synchronously, such as being reduced by the same setting value or being increased by a corresponding multiple, or can be reduced asynchronously, such as K being reduced by m and L being reduced by n. The m and n are respectively the step length of the adaptive adjustment of the first setting threshold K and the second setting threshold L, which are built-in in the system and can be adjusted through configuration after the system is running.

[0070] So far, the flow shown in FIG. 3 is completed.

[0071] The adaptive adjustment of the first setting threshold K and the second setting threshold L is realized through the flow shown in FIG. 3.

[0072] Alternatively, as an embodiment, the first setting threshold K and the second setting threshold L can be used to judge all unit memories to identify whether each unit memory is a cold unit memory, a hot unit memory or a warm unit memory.

[0073] As another embodiment, the embodiment can also assign the corresponding first setting threshold K and the second setting threshold L to the unit memory assigned to at least one specific application (corresponding to the specific application) to identify whether the unit memory is a cold unit memory, a hot unit memory or a warm unit memory based on the first setting threshold K and the second setting threshold L assigned to the unit memory (corresponding to the specific application). This is equivalent to realizing that only the unit memory assigned to the specific application should be managed. For example, if an application running on an application end has a low requirement for memory delay, the first setting threshold K and the second setting threshold L corresponding to the unit memory assigned to the application (or the application end) can be large. According to the above description, the unit memory is always a cold unit memory, and the data in the unit memory can always be placed in the second memory.

[0074] In the embodiment, the software address converter can schedule the data in the memory based on a memory scheduling period. The memory scheduling period can be set according to actual needs, and the embodiment does not specifically limit it.

[0075] Referring to FIG. 4, FIG. 4 is a data scheduling flowchart provided by Embodiment 1 of the present application. In each memory scheduling period, as shown in FIG. 4, the flow can include the following steps:

[0076] Step 401, for the current unit memory, if the current unit memory is the hot unit memory, step 402 is performed, if the current unit memory is the cold unit memory, step 408 is performed, and if the current unit memory is the warm unit memory, step 414 is performed.

[0077] The current unit memory here can be one unit memory currently traversed in sequence, or one unit memory specified, and the embodiment is not limited specifically.

[0078] Step 402, if the current unit memory is in the first memory, step 403 is performed, and if the current unit memory is in the second memory, step 404 is performed.

[0079] Step 403, the data in the current unit memory is maintained unchanged.

[0080] Step 404, the current unit memory is locked to suspend reading and / or writing of the current unit memory.

[0081] At this time, step 404 is performed, and the current unit memory is the first target unit memory.

[0082] Step 405, a memory application is initiated to the OS to apply for allocating the corresponding destination unit memory to the first memory as the current unit memory.

[0083] The destination unit memory has a corresponding virtual address (denoted as VA') and a physical address (denoted as PA'). In the embodiment, the mapping relationship between VA' and PA' can be recorded through a page table. The destination unit memory here is the second target unit memory.

[0084] Step 406, the data stored in the current unit memory is moved to the destination unit memory, and moving information is recorded.

[0085] Optionally, the moving information can be, for example, the mapping relationship between the virtual address VA of the current unit memory and the virtual address VA' of the destination unit memory. The moving information can be recorded to an address translation table.

[0086] Step 407, the current unit memory is released, and the lock on the current unit memory is released.

[0087] After the above operation, the access of the application to the virtual address VA is replaced by the physical memory access to PA', and the application end is not aware of it. Examples will be described below, and details are not described here.

[0088] Step 408, if the current unit memory is in the second memory, step 409 is performed, and if the current unit memory is in the first memory, step 410 is performed.

[0089] Step 409, maintaining the data in the current unit memory unchanged.

[0090] Step 410, locking the current unit memory to suspend the reading and / or writing of the current unit memory.

[0091] At this time, the execution of step 410 is performed, and the current unit memory is the first target unit memory described above.

[0092] Step 411, initiating a memory application to the OS to apply for a corresponding target unit memory for the second memory as the current unit memory.

[0093] The target unit memory has a corresponding virtual address VA' and a physical address PA'. In the embodiment, the mapping relationship between VA' and PA' can be recorded through a page table. The target unit memory here is the second target unit memory described above.

[0094] Step 412, moving the data stored in the current unit memory to the target unit memory described above, and recording the moving information.

[0095] Optionally, the moving information can be, for example, the mapping relationship between the virtual address VA of the current unit memory and the virtual address VA' of the target unit memory. The moving information can be recorded to an address translation table.

[0096] Step 413, releasing the current unit memory and unlocking the current unit memory.

[0097] After the above operations, the access of the application to the virtual address VA is replaced by the physical memory access to PA', and the application end is not aware of it. Examples will be described below, and will not be described here.

[0098] Step 414, maintaining the data in the current unit memory unchanged.

[0099] It should be noted that step 414 is executed on the premise that the current unit memory is the warm unit memory described above. Optionally, according to actual needs, when the current unit memory is the warm unit memory described above, it can also be processed as a hot unit memory or a cold unit memory, and the embodiment is not specifically limited.

[0100] At this point, the process shown in FIG. 4 is completed. Through the process shown in FIG. 4, the scheduling of the data in the memory is realized.

[0101] Based on the above description, by means of the software address translator and the address translation table recording the migration information, the access control of the application end can be realized. As shown in FIG. 5, when receiving the access request sent from the application end, the software address translator first finds the other virtual address (here, VA' is taken as an example) having a mapping relationship with the virtual address (here, VA is taken as an example) carried in the access request based on the address translation table. Then, the software address translator outputs the found virtual address VA' to the memory management unit (MMU) in the CPU, so that the MMU obtains the corresponding physical address PA' based on the input VA' by querying the page table, so as to realize the access to the above-mentioned unit memory based on the physical address PA', realize the replacement of the access to the virtual address VA to the physical memory access of PA', and the application end is not aware of it.

[0102] It should be noted that if the software address translator does not find the other virtual address (here, VA' is taken as an example) having a mapping relationship with the virtual address (here, VA is taken as an example) carried in the access request based on the address translation table, the VA is directly output to the MMU in the CPU. If the software address translator does not find the other virtual address (here, VA' is taken as an example) having a mapping relationship with the virtual address (here, VA is taken as an example) carried in the access request, it means that the data in the physical memory corresponding to the virtual address carried in the access request has not been migrated.

[0103] In the embodiment, the access request can also carry the information of the application, and the embodiment can also be used to set the personalized memory allocation for the specific application, such as pre-setting the physical memory (which is different from the physical memory mapped by the previously allocated virtual address of the memory) required to be accessed by the application, directly accessing the physical memory corresponding to the application based on the information of the application carried in the access request, and the like. The embodiment is not specifically limited.

[0104] The embodiment 1 of the present application is described above, and the embodiment 2 of the present application is described below:

[0105] Embodiment 2:

[0106] The embodiment 2 is applied to the scenario of realizing the flow shown in FIG. 1 by means of the hardware address translator.

[0107] In this embodiment 2, the hardware address converter is integrated in the MMU in the CPU, or is deployed between the MMU in the CPU and the system bus. Compared with the software address converter (used to implement the conversion of the virtual addresses of different unit memories) in embodiment 1, the hardware address converter in this embodiment 2 is used to implement the conversion of the physical addresses of different unit memories. FIG. 6 exemplarily shows that the hardware address converter is deployed between the MMU and the system bus.

[0108] In this embodiment 2, the hardware address converter or other components in the data storage system can count the access frequency of each unit memory based on the heat statistics period T. For example, the access frequency of data in each unit memory in the recent unit time (which can be set) is counted, and if the access frequency is greater than or equal to a second set threshold (denoted as K), it is determined that the unit memory is a hot unit memory (i.e., the first target unit memory described above); if the access frequency is less than a first set threshold (denoted as L), it is determined that the unit memory is a cold unit memory (i.e., the first target unit memory described above). If the access frequency is between K and L, it is determined that the unit memory is a warm unit memory.

[0109] In this embodiment, T, K, L and the unit time can be dynamically and adaptively modified. For example, T and the unit time can be modified through a command line. K and L can be modified according to the actual use of the memory, and details can be referred to embodiment 1, which will not be described here.

[0110] In this embodiment, the hardware address converter described above can schedule the data in the memory based on a memory scheduling period. The memory scheduling period can be set according to actual needs, and this embodiment does not specifically limit it.

[0111] Referring to FIG. 7, FIG. 7 is a data scheduling flowchart provided by the embodiment 2 of the present application. In each memory scheduling period, as shown in FIG. 7, the flowchart can include the following steps:

[0112] In step 701, for the current unit memory, if the current unit memory is the hot unit memory described above, step 702 is performed, if the current unit memory is the cold unit memory described above, step 708 is performed, and if the current unit memory is the warm unit memory described above, step 714 is performed.

[0113] The current unit memory here can be one unit memory currently traversed in order, or one specified unit memory, and this embodiment does not specifically limit it.

[0114] In step 702, if the current unit memory is in the first memory, step 703 is performed, and if the current unit memory is in the second memory, step 704 is performed.

[0115] In step 703, the data in the current unit memory is maintained unchanged.

[0116] Step 704, lock the current unit memory to suspend reading and / or writing of the current unit memory.

[0117] At this time, the execution is performed to step 704, and the current unit memory is the first target unit memory described above.

[0118] Step 705, obtain a target unit memory which is idle in the first memory.

[0119] The target unit memory has a corresponding virtual address VA' and physical address PA'. In the embodiment, the mapping relationship between VA' and PA' can be recorded by a page table. The target unit memory here is the second target unit memory described above.

[0120] Step 706, move the data stored in the current unit memory to the target unit memory described above, and record the moving information.

[0121] Optionally, the moving information can be, for example, the mapping relationship between the physical address PA of the current unit memory and the physical address PA' of the target unit memory. The moving information can be recorded in an address translation table.

[0122] Step 707, release the current unit memory and unlock the current unit memory.

[0123] After the above operation, the access of the application to the physical address PA is replaced by the physical memory access to PA', and the application end is not aware of it. Examples will be described below, and details are not described here.

[0124] Step 708, if the current unit memory is in the second memory, execute step 709, and if the current unit memory is in the first memory, execute step 710.

[0125] Step 709, maintain the data in the current unit memory unchanged.

[0126] Step 710, lock the current unit memory to suspend reading and / or writing of the current unit memory.

[0127] At this time, the execution is performed to step 410, and the current unit memory is the first target unit memory described above.

[0128] Step 711, obtain a target unit memory which is idle in the second memory.

[0129] The target unit memory has a corresponding virtual address VA' and physical address PA'. In the embodiment, the mapping relationship between VA' and PA' can be recorded by a page table. The target unit memory here is the second target unit memory described above.

[0130] Step 712, move the data stored in the current unit memory to the destination unit memory, and record the moving information.

[0131] Optionally, the moving information can be a mapping relationship between the physical address PA of the current unit memory and the physical address PA' of the destination unit memory. The moving information can be recorded in the address translation table.

[0132] Step 713, release the current unit memory, and unlock the current unit memory.

[0133] After the above operation, the access to the physical address PA of the application is replaced by the physical memory access to PA', and the application is not aware of it. Examples will be described below, which will not be described here.

[0134] Step 714, maintain the data in the current unit memory unchanged.

[0135] It should be noted that the step 714 is executed on the premise that the current unit memory is the warm unit memory described above. Optionally, according to actual needs, when the current unit memory is the warm unit memory described above, it can also be processed as a hot unit memory or a cold unit memory, and the embodiment is not specifically limited.

[0136] At this point, the process shown in FIG. 7 is completed. Through the process shown in FIG. 7, the scheduling of data in the memory is realized.

[0137] Based on the above description, with the help of the hardware address translator and the address translation table recording the moving information, the access control of the application end can be realized. As shown in FIG. 8, when receiving the access request sent from the application end, the MMU finds the physical address (here, the PA described above is taken as an example) having a mapping relationship with the virtual address (here, the VA described above is taken as an example) carried by the access request in the above page. Then, the MMU outputs the found physical address PA to the hardware address translator. The hardware address translator obtains the corresponding physical address PA' based on the input physical address PA and by querying the address translation table, so as to realize the access to the destination unit memory based on the physical address PA', realize the replacement of the access to the physical address PA by the physical memory access to PA', and the application is not aware of it.

[0138] It should be noted that if the hardware address translator does not find other physical addresses having a mapping relationship with the input physical address in the address translation table, the input physical address is directly output to the hardware address translator. If the hardware address translator does not find other physical addresses having a mapping relationship with the input physical address in the address translation table, it means that the data in the physical memory corresponding to the input physical address has not been moved.

[0139] The above describes the embodiment 2 of the present application.

[0140] The device provided by the embodiment of the present application is described below.

[0141] Referring to FIG. 9, FIG. 9 is a structure diagram of a device provided by an embodiment of the present application. The device is applied to a data storage system, wherein the data storage system comprises a local first memory and an extended second memory; the first memory and the second memory are both divided into at least two unit memories; the device comprises:

[0142] an obtaining unit, configured to obtain a first target unit memory which needs to be moved; wherein the first target unit memory belongs to the first memory, and the access frequency of the first target unit memory in a unit time is less than a first set threshold; or the first target unit memory belongs to the second memory, and the access frequency of the first target unit memory in a unit time is greater than or equal to a second set threshold; the second set threshold is greater than the first set threshold; and

[0143] obtain an idle second target unit memory; when the first target unit memory belongs to the first memory, the second target unit memory belongs to the second memory; when the first target unit memory belongs to the second memory, the second target unit memory belongs to the first memory;

[0144] a moving unit, configured to move the data stored in the first target unit memory to the second target unit memory, and record moving information;

[0145] an address conversion unit, configured to, when receiving an access request sent from an application end, control the access to the first target unit memory to be converted to the access to the second target unit memory based on the recorded moving information and the first virtual address of the first target unit memory carried by the access request and allocated to the application end.

[0146] Optionally, the moving information comprises the mapping relationship between the first virtual address of the first target unit memory and the second virtual address of the second target unit memory.

[0147] The control of the access to the first target unit memory to be converted to the access to the second target unit memory based on the moving information and the first virtual address of the first target unit memory carried by the access request and allocated to the application end comprises:

[0148] determining the second virtual address having the mapping relationship with the first virtual address carried by the access request based on the moving information, so as to access the second target unit memory based on the second virtual address.

[0149] Optionally, the apparatus is applied to an address translator implemented by software.

[0150] The address translator is disposed between a CPU and an operating system (OS) in the data storage system, or the address translator is integrated into the OS in the data storage system.

[0151] Optionally, the migration information comprises a mapping relationship between a first physical address of the first target unit of memory and a second physical address of the second target unit of memory.

[0152] The control of the access to the first target unit of memory to the access to the second target unit of memory based on the migration information and the first virtual address of the application end allocated for identifying the first target unit of memory carried by the access request comprises:

[0153] obtaining a first physical address; the first physical address is a physical address of the first target unit of memory corresponding to the first virtual address;

[0154] determining a second physical address having a mapping relationship with the first physical address based on the migration information, so as to access the second target unit of memory based on the second physical address.

[0155] Optionally, the apparatus is applied to an address translator implemented by hardware.

[0156] The address translator is disposed between a memory management unit (MMU) and a system bus in the CPU in the data storage system, or the address translator is integrated into the MMU; the MMU is used to implement the conversion between the virtual address and the physical address of the same unit of memory.

[0157] Optionally, the apparatus further comprises:

[0158] a locking unit, configured to lock the first target unit of memory to suspend the reading and / or writing performed on the first target unit of memory after the obtaining unit obtains the first target unit of memory and before the migration unit migrates the data stored in the first target unit of memory to the second target unit of memory; and

[0159] after the migration unit migrates the data stored in the first target unit of memory to the second target unit of memory, further releasing the lock of the first target unit of memory to restore the reading and / or writing performed on the first target unit of memory.

[0160] Optionally, the first set threshold and the second set threshold are dynamically adjusted according to the usage rates of the first memory and the second memory.

[0161] Optionally, the first setting threshold and the second setting threshold are adjusted by the following manner:

[0162] If the usage of the first memory is greater than or equal to the first usage threshold, and the usage of the second memory is less than the second usage threshold, the first setting threshold is increased, and the second setting threshold is increased; the second usage threshold is less than the first usage threshold;

[0163] If the usage of the first memory is less than the first usage threshold, and the usage of the second memory is greater than or equal to the third usage threshold, the first setting threshold is decreased, and the second setting threshold is decreased; the third usage threshold is greater than the second usage threshold.

[0164] Optionally, the unit memory allocated to the application end is allocated based on memory information of the memory allowed to be used by the application end;

[0165] The memory information at least includes: a proportion of the first memory used by the application, and / or a proportion of the second memory used by the application.

[0166] So far, the structure of the device shown in FIG. 9 is described.

[0167] The embodiment of the present application further provides a hardware structure of the device shown in FIG. 9. As shown in FIG. 10, the hardware structure can include: a processor and a machine readable storage medium;

[0168] The machine readable storage medium stores computer instructions, and the computer instructions are executed by the processor to perform the steps in the above method.

[0169] Based on the same application concept as the above method, the embodiment of the present application further provides a machine readable storage medium, and the machine readable storage medium stores a plurality of computer instructions, and the computer instructions are executed by the processor to implement the method disclosed in the above examples of the present application.

[0170] For example, the machine readable storage medium can be: RAM (Radom Access Memory, Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state disk, any type of storage disk (such as optical disk, dvd, etc.), or similar storage medium, or combination thereof.

[0171] The systems, apparatuses, modules, or units disclosed in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the computer can be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0172] For the convenience of description, the above apparatuses are described in various units according to functions in the description. Of course, the functions of the units can be implemented in one or more software and / or hardware in the implementation of the present application.

[0173] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0175] Moreover, these computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0176] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices, to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide steps for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0177] The above only describes the embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A data scheduling method characterized by comprising: The method is applied to a data storage system, the data storage system comprising a local first memory and an extended second memory; The first memory and the second memory are each divided into at least two units of memory; The method comprises: obtaining a first target unit of memory that needs to be moved; wherein the first target unit of memory belongs to the first memory, and the access frequency of the first target unit of memory in a unit of time is less than a first set threshold; or the first target unit of memory belongs to the second memory, and the access frequency of the first target unit of memory in a unit of time is greater than or equal to a second set threshold; the second set threshold is greater than the first set threshold; obtaining a second target unit of memory that is idle; when the first target unit of memory belongs to the first memory, the second target unit of memory belongs to the second memory; when the first target unit of memory belongs to the second memory, the second target unit of memory belongs to the first memory; moving the data stored in the first target unit of memory to the second target unit of memory, and recording moving information; when receiving an access request sent from an application end, based on the recorded moving information and the first virtual address allocated by the application end to identify the first target unit of memory carried by the access request, the access to the first target unit of memory is converted to the access to the second target unit of memory.

2. The method of claim 1, wherein, The moving information comprises a mapping relationship between the first virtual address of the first target unit of memory and the second virtual address of the second target unit of memory; The conversion of the access to the first target unit of memory to the access to the second target unit of memory based on the moving information and the first virtual address allocated by the application end to identify the first target unit of memory carried by the access request comprises: determining the second virtual address having a mapping relationship with the first virtual address carried by the access request based on the moving information, so as to access the second target unit of memory based on the second virtual address.

3. The method according to claim 1 or 2, characterized in that, The method is applied to an address converter implemented by software; The address converter is deployed between a CPU and an operating system (OS) in the data storage system, or the address converter is integrated into the OS in the data storage system.

4. The method of claim 1, wherein, The moving information comprises a mapping relationship between the first physical address of the first target unit of memory and the second physical address of the second target unit of memory; The conversion of the access to the first target unit of memory to the access to the second target unit of memory based on the moving information and the first virtual address allocated by the application end to identify the first target unit of memory carried by the access request comprises: obtaining the first physical address; the first physical address is the physical address of the first target unit of memory corresponding to the first virtual address; determining the second physical address having a mapping relationship with the first physical address based on the moving information, so as to access the second target unit of memory based on the second physical address. The method is applied to an address converter implemented by hardware; 5. The method according to claim 1 or 4, characterized in that, ​ The address converter is disposed between a memory management unit (MMU) in a CPU and a system bus in the data storage system, or the address converter is integrated into the MMU; the MMU is configured to implement conversion between a virtual address and a physical address of the same unit of memory.

6. The method of claim 1, wherein, After the first target unit of memory is obtained, and before data stored in the first target unit of memory is moved to the second target unit of memory, the method further comprises: locking the first target unit of memory to suspend reading and / or writing performed on the first target unit of memory; After the data stored in the first target unit of memory is moved to the second target unit of memory, the method further comprises: unlocking the first target unit of memory to restore reading and / or writing performed on the first target unit of memory.

7. The method of claim 1, wherein, The first set threshold and the second set threshold are dynamically adjusted according to usage rates of the first memory and the second memory.

8. The method of claim 7, wherein, The first set threshold and the second set threshold are adjusted by the following method: If the usage rate of the first memory is greater than or equal to a first usage rate threshold, and the usage rate of the second memory is less than a second usage rate threshold, the first set threshold is increased, and the second set threshold is increased; The second usage rate threshold is less than the first usage rate threshold; If the usage rate of the first memory is less than the first usage rate threshold, and the usage rate of the second memory is greater than or equal to a third usage rate threshold, the first set threshold is decreased, and the second set threshold is decreased; The third usage rate threshold is greater than the second usage rate threshold.

9. The method of claim 1, wherein, The unit of memory allocated to the application end is allocated based on memory information of memory allowed to be used by the application end; The memory information at least includes: a proportion of the first memory used by the application, and / or a proportion of the second memory used by the application.

10. A data scheduling apparatus, characterized by comprising: The device is applied to a data storage system, and the data storage system includes a local first memory and an extended second memory; The first memory and the second memory are each divided into at least two units of memory; The device comprises: An obtaining unit is configured to obtain a first target unit of memory that needs to be moved; wherein the first target unit of memory belongs to the first memory, and an access frequency of the first target unit of memory in a unit of time is less than a first set threshold; or the first target unit of memory belongs to the second memory, and an access frequency of the first target unit of memory in a unit of time is greater than or equal to a second set threshold; the second set threshold is greater than the first set threshold; and An obtaining unit is configured to obtain a first target unit of memory that needs to be moved; wherein the first target unit of memory belongs to the first memory, and an access frequency of the first target unit of memory in a unit of time is less than a first set threshold; or the first target unit of memory belongs to the second memory, and an access frequency of the first target unit of memory in a unit of time is greater than or equal to a second set threshold; the second set threshold is greater than the first set threshold; and A moving unit is configured to move data stored in the first target unit of memory to the second target unit of memory, and record moving information; An address conversion unit is configured to, when receiving an access request sent from an application end, control conversion of access to a first target unit memory to access to a second target unit memory based on recorded migration information and a first virtual address carried in the access request and assigned to the application end for identifying the first target unit memory.

11. The apparatus of claim 10, wherein, The migration information includes a mapping relationship between the first virtual address of the first target unit memory and a second virtual address of the second target unit memory. The conversion of the access to the first target unit memory to the access to the second target unit memory based on the migration information and the first virtual address carried in the access request and assigned to the application end for identifying the first target unit memory includes: determining, based on the migration information, the second virtual address having a mapping relationship with the first virtual address carried in the access request, to access the second target unit memory based on the second virtual address.

12. The apparatus of claim 10 or 11, wherein, The device is applied to an address converter implemented by software. The address converter is disposed between a CPU and an operating system (OS) in the data storage system, or the address converter is integrated into the OS in the data storage system.

13. The apparatus of claim 10, wherein, The migration information includes a mapping relationship between the first physical address of the first target unit memory and a second physical address of the second target unit memory. The conversion of the access to the first target unit memory to the access to the second target unit memory based on the migration information and the first virtual address carried in the access request and assigned to the application end for identifying the first target unit memory includes: obtaining the first physical address, which is a physical address of the first target unit memory corresponding to the first virtual address; determining, based on the migration information, the second physical address having a mapping relationship with the first physical address, to access the second target unit memory based on the second physical address. The device is applied to an address converter implemented by hardware.

14. The apparatus of claim 10 or 13, wherein, The address converter is disposed between a memory management unit (MMU) and a system bus in the CPU in the data storage system, or the address converter is integrated into the MMU; and the MMU is configured to implement conversion between a virtual address and a physical address of the same unit memory. The device further includes:

15. The apparatus of claim 10, wherein, a locking unit configured to, after the obtaining unit obtains the first target unit memory and before the migration unit migrates data stored in the first target unit memory to the second target unit memory, lock the first target unit memory to suspend reading and / or writing performed on the first target unit memory; and after the migration unit migrates the data stored in the first target unit memory to the second target unit memory, further release the lock on the first target unit memory to restore the reading and / or writing performed on the first target unit memory. The first and second set thresholds are dynamically adjusted according to the usage rates of the first and second memories.

16. The apparatus of claim 10, wherein, The first and second set thresholds are adjusted by:

17. The apparatus of claim 16, wherein, ​ if the usage rate of the first memory is greater than or equal to the first usage rate threshold and the usage rate of the second memory is less than the second usage rate threshold, then increasing the first set threshold and increasing the second set threshold; the second usage rate threshold is less than the first usage rate threshold; if the usage rate of the first memory is less than the first usage rate threshold and the usage rate of the second memory is greater than or equal to the third usage rate threshold, then decreasing the first set threshold and decreasing the second set threshold; the third usage rate threshold is greater than the second usage rate threshold.

18. The apparatus of claim 10, wherein, the unit memory allocated to the application end is based on memory information of the memory allowed to be used by the application end; the memory information at least includes: a proportion of the first memory used by the application, and / or, a proportion of the second memory used by the application.

19. An electronic device, comprising: The electronic device comprises: a processor and a machine readable storage medium; the machine readable storage medium stores computer instructions, when the computer instructions are executed by the processor, the steps in any one of the methods of claims 1 to 9 are executed.

Images