Memory processing method, electronic equipment and related device

Abstract

The invention provides a memory processing method, electronic equipment and a related device, and relates to the technical field of terminals. The method is applied to the electronic equipment, the electronic equipment comprises a first storage space and a second storage space, and multiple pieces of application data of a first application are stored in the second storage space. In the method, under the condition that a first application is switched from a background to a foreground, n times of first operation are executed, the first operation comprises the steps that a data set is written into a cache block applied for from a first storage space, the data set comprises at least one piece of application data in multiple pieces of application data, the size of the data set is smaller than or equal to the size of the cache block, and the first application is switched from the background to the foreground. N is an integer greater than or equal to 3. Wherein the first size of the cache block in the ith first operation is greater than or equal to the second size of the cache block in the (i-1) th first operation, and the value of i is 2-n. In this way, at least one piece of data is aggregated and submitted to IO based on the data set, and the bandwidth utilization rate can be increased.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of terminal, and in particular, to a memory processing method, an electronic device, and related apparatuses. BACKGROUND

[0002] The DMABUF mechanism is a sharing mechanism of a memory buffer. The DMABUF memory occupancy of a single application is usually 40MB-250MB. In the case of multi-application background keep-alive, the whole machine occupancy of DMABUF is high.

[0003] In order to reduce the memory occupancy of DMABUF, there is a solution that can swap the data stored in the memory to the UFS storage medium in sequence when the application switches to the background. When the application switches back to the foreground, the data stored in the UFS storage medium is swapped into the memory in sequence. However, the size of a large amount of application data generated by the application is relatively small, for example, 80% of the application data is less than 1MB.

[0004] Therefore, the data migration between the UFS and the memory has the problem of low bandwidth utilization. SUMMARY

[0005] Embodiments of the present application provide a memory processing method, an electronic device, and related apparatuses to improve the bandwidth utilization in the process of data migration between a non-volatile storage medium (such as UFS) and a volatile storage medium (such as memory).

[0006] In a first aspect, embodiments of the present application provide a memory processing method applied to an electronic device. The electronic device includes a first storage space and a second storage space. The second storage space stores a plurality of application data of a first application. For example, the first storage space is a volatile storage, for example, the first storage space can be a memory. The second storage space is a non-volatile storage, for example, the second storage space can be a UFS, or the second storage space can also be an eMMC, a DRAM, etc. The implementation of the second storage space is not limited in the present application.

[0007] The method of the present application includes:

[0008] In the case that the first application is switched from the background to the foreground, n times of first operations are performed. The first operation includes: writing a data set into a cache block applied from the first storage space, wherein the data set includes at least one application data of the plurality of application data of the second storage space, the size of the data set is less than or equal to the size of the cache block, and n is an integer greater than or equal to 3;

[0009] The first size of the cache block in the i-th first operation is greater than or equal to the second size of the cache block in the i-1-th first operation, and i is 2 to n.

[0010] The first operation introduced herein can also be understood as a file IO operation, so that the file IO can be submitted for a data set, that is, a plurality of application data is aggregated and submitted for a file IO. Compared with the implementation mode of submitting a file IO for each application data, the technical solution of the application can effectively improve the bandwidth utilization. In the process of executing the plurality of rounds of file IO, the size of the cache block corresponding to the file IO is not determined based on the size of the application data to be stored, but the size of the cache block is either increased successively or kept the same as the previous time. In this way, the capacity of the cache block can store the application data contained in the data set.

[0011] In a possible implementation mode, when the value of i is less than or equal to 3, the first size is greater than the second size.

[0012] In the technical solution of the application, the size of the cache block can be dynamically adjusted based on the bandwidth of the file IO. However, when the value of i is less than or equal to 3, there is not enough bandwidth for comparison, so the size of the enlarged cache block can be fixedly set, so that the cache block based on the application can store the aggregated data set.

[0013] In a possible implementation mode, when the value of i is greater than 3, the size of the cache block is determined based on the bandwidth.

[0014] In this way, when the value of i is greater than 3, there is enough bandwidth for comparison to determine the actual situation of the bandwidth, so the size of the cache block can be determined based on the bandwidth, so that it can be determined whether the size of the cache block needs to be enlarged based on the bandwidth, to avoid the situation of excessive aggregation of data.

[0015] For example, the size of the cache block corresponding to the i-th round of first operation can be determined based on the bandwidth corresponding to the x rounds of first operation before the i-th round. The rounds before the i-th round, that is, the 1st to i-1th rounds, for example, can select x rounds of first operation in the 1st to i-1th rounds, and determine the size of the cache block corresponding to the i-th round of first operation based on the bandwidth corresponding to the x rounds of first operation. Wherein x can be an integer greater than or equal to 1, and the specific value of x can be selected according to actual needs, and which x rounds are selected in the 1st to i-1th rounds can also be set according to actual needs.

[0016] And, for example, whether the first condition is satisfied can be determined based on a bandwidth corresponding to the xth first operation. The first condition can be a condition for measuring whether the bandwidth utilization of the current file IO reaches a preset requirement (or can also be understood as a preset utilization rate). The specific setting of the first condition can also be selected according to actual needs.

[0017] Exemplarily, in the case where the first condition is not satisfied, it means that the bandwidth utilization of the current file IO has not reached the preset requirement. Therefore, the size of the cache block can be enlarged based on the size of the cache block corresponding to the (i-1)th first operation, so that the process of the file IO can reach a higher bandwidth. In the case where the first condition is satisfied, it means that the bandwidth utilization of the current file IO has reached the preset requirement. Therefore, in order to avoid excessive aggregation of data, the size of the cache block corresponding to the ith first operation can be the same as the size of the cache block corresponding to the (i-1)th first operation.

[0018] Here, a specific implementation is taken as an example for introduction. In a possible implementation, in the case where the value of i is greater than 3, the first size is determined based on a first bandwidth, a second bandwidth, and a second size. The first bandwidth is a bandwidth corresponding to the (i-2)th first operation, and the second bandwidth is a bandwidth corresponding to the (i-3)th first operation.

[0019] In this implementation, the bandwidth change condition can be determined based on the bandwidths of two consecutive rounds. Then, the size of the cache block of the next round (i.e., the ith round) (i.e., the first size) can be determined based on the bandwidth change condition and the size of the cache block of the previous round (i.e., the (i-1)th round) (i.e., the second size). Therefore, it can be achieved that whether the size of the cache block is enlarged based on the current size of the cache block is dynamically determined based on the bandwidth change condition, so as to adjust the size of the cache block based on the actual bandwidth utilization condition and avoid excessive aggregation of data.

[0020] In a possible implementation, in the case where the first bandwidth and the second bandwidth satisfy a first quantity relationship, the first size is equal to the second size.

[0021] In the case where the first bandwidth and the second bandwidth do not satisfy the first quantity relationship, the first size is greater than the second size. Whether the first bandwidth and the second bandwidth satisfy the first quantity relationship can be understood as the first condition described above.

[0022] In a possible implementation, the first quantity relationship includes that a ratio of the first bandwidth to the second bandwidth is less than a first preset threshold value; and / or, the first quantity relationship includes that a difference between the first bandwidth and the second bandwidth is less than a second preset threshold value.

[0023] In this implementation, the bandwidth change condition can be determined based on the ratio or difference between the bandwidths of two adjacent times, and then it is determined whether to enlarge the size of the cache block based on the bandwidth change condition. Specifically, when the first bandwidth and the second bandwidth satisfy the first quantity relationship, it means that the bandwidth change has been small, and the bandwidth gain brought by the enlargement of the size of the cache block is relatively small at this time, and the problem of over-aggregation may occur. Therefore, when the first bandwidth and the second bandwidth satisfy the first quantity relationship, the size of the cache block is not enlarged, that is, the first size is equal to the second size. In the case where the first bandwidth and the second bandwidth do not satisfy the first quantity relationship, it means that the bandwidth gain brought by the enlargement of the size of the cache block is rapidly increasing, and therefore the enlargement of the cache block can effectively improve the bandwidth utilization in this case.

[0024] In a possible implementation, in the case where the value of i is 1, the size of the cache block in the i-th first operation is a preset size. By pre-setting the size of the cache block in the first round, the initial size of the cache block can be reasonably set.

[0025] In a possible implementation, before performing the i-th first operation, the method further includes:

[0026] performing a second operation, the second operation including: applying, based on the first size, a cache block from the first storage space.

[0027] In a possible implementation, the second operation further includes: determining a data set corresponding to the cache block.

[0028] The second operation can be understood as an operation of data preparation. It can be understood that, when performing data aggregation for file IO, the technical solution of the present application does not wait for the data amount in a unit to exceed a threshold before performing file IO, but directly selects at least one piece of application data that matches the size of the cache block from a plurality of pieces of application data to obtain a data set after determining the size of the cache block, and then directly submits file IO for the data set. This implementation does not need to wait for the data amount to reach the threshold, but can directly determine the matching data set based on the size of the cache block, and then directly submit the file IO corresponding to the data set, so that the efficiency of data exchange from one storage space to another storage space can be improved.

[0029] In a possible implementation, determining the data set corresponding to the cache block includes:

[0030] determining, based on the first size, the data set corresponding to the cache block.

[0031] In a possible implementation, determining, based on the first size, the data set corresponding to the cache block includes:

[0032] Among the remaining application data in the second storage space, a combination of at least one piece of application data whose difference between the total size and the first size is the smallest is determined to obtain the data set corresponding to the cache block. The remaining application data is application data that has not been divided into the data set, or can be understood as application data whose mapping relationship with the cache block is not confirmed.

[0033] In this way, it can be determined which data is stored in the cache block based on the size of the cache block, thereby maximizing the utilization of the cache block.

[0034] In a possible implementation, the period in which the jth first operation is performed is a first period, and the period in which the second operation is performed before the j+1th first operation is a second period, the first period overlaps the second period, and j is 1 to n-1. In an implementation, the first period at least partially overlaps the second period.

[0035] In a possible implementation, the first operation is performed based on a first thread, and the second operation is performed based on a second thread.

[0036] That is, the data preparation and file IO processing described in this application are performed asynchronously, so that the file IO processing can be performed at the same time as the data preparation, thereby improving the utilization of CPU and bandwidth.

[0037] In a possible implementation, in the case where the first size is greater than the second size, the first size and the second size satisfy a second quantity relationship.

[0038] In a possible implementation, the second quantity relationship is that the first size is T times the second size, or the difference between the first size and the second size is a first value, and T is a value greater than or equal to 1. The first value can be a fixed value set in advance to indicate a constant increase based on the first value in the process of expanding the cache block.

[0039] In this implementation, by setting the first size and the second size to satisfy the second quantity relationship, the size of the cache block can be enlarged based on a specific rule, so that the sizes of the cache blocks enlarged successively satisfy the relationship of equal ratio or equal difference, facilitating the maintenance and management of the cache blocks.

[0040] In a second aspect, an embodiment of the present application provides a memory processing method applied to an electronic device, the electronic device including a first storage space and a second storage space, and the first storage space storing a plurality of pieces of application data of a first application. For example, the first storage space can be a memory, and the second storage space can be a UFS, or the second storage space can also be an eMMC, a DRAM, or the like, and the present application does not limit the implementation of the second storage space.

[0041] The method of the present application comprises:

[0042] In the case that the first application is switched from the foreground to the background, the first operation is performed n times, the first operation comprising: writing a data set into a cache block applied from the second storage space, wherein the data set comprises at least one piece of application data in a plurality of pieces of application data, the size of the data set is less than or equal to the size of the cache block, and n is an integer greater than or equal to 3;

[0043] Wherein the first size of the cache block in the i-th first operation is greater than or equal to the second size of the cache block in the (i-1)-th first operation, and i is valued from 2 to n.

[0044] In a possible implementation, in the case that i is valued less than or equal to 3, the first size is greater than the second size.

[0045] In a possible implementation, in the case that i is valued greater than 3, the size of the cache block is determined based on bandwidth.

[0046] In a possible implementation, in the case that i is valued greater than 3, the first size is determined based on a first bandwidth, a second bandwidth, and the second size, the first bandwidth being a bandwidth corresponding to the (i-2)-th first operation, and the second bandwidth being a bandwidth corresponding to the (i-3)-th first operation.

[0047] In a possible implementation, in the case that the first bandwidth and the second bandwidth satisfy a first quantitative relationship, the first size is equal to the second size.

[0048] In the case that the first bandwidth and the second bandwidth do not satisfy the first quantitative relationship, the first size is greater than the second size.

[0049] In a possible implementation, the first quantitative relationship comprises that the ratio of the first bandwidth to the second bandwidth is less than a first preset threshold value; and / or, the first quantitative relationship comprises that the difference between the first bandwidth and the second bandwidth is less than a second preset threshold value.

[0050] In a possible implementation, in the case that i is valued 1, the size of the cache block in the i-th first operation is a preset size.

[0051] In a possible implementation, before the i-th first operation is performed, the method further comprises:

[0052] Performing a second operation, the second operation comprising: applying a cache block from the second storage space based on the first size.

[0053] In a possible implementation, the second operation further comprises: determining a data set corresponding to the cache block.

[0054] In a possible implementation, determining the data set corresponding to the cache block includes:

[0055] Determining the data set corresponding to the cache block based on the first size.

[0056] In a possible implementation, determining the data set corresponding to the cache block based on the first size includes:

[0057] Determining, from the application data remaining in the first storage space, a combination of at least one piece of application data with a difference between the total size and the first size being the smallest, to obtain the data set corresponding to the cache block.

[0058] In a possible implementation, the time period for performing the jth first operation is a first time period, and the time period for performing the second operation before the j+1th first operation is a second time period, the first time period overlaps the second time period, and j is an integer from 1 to n-1.

[0059] In a possible implementation, the first operation is performed based on a first thread, and the second operation is performed based on a second thread.

[0060] In a possible implementation, in a case where the first size is greater than the second size, the first size and the second size satisfy a second quantity relationship.

[0061] In a possible implementation, the second quantity relationship is that the first size is T times the second size, or a difference between the first size and the second size is a first value, and T is a value greater than or equal to 1. The first value can be a fixed value set in advance, to indicate that a constant increase is performed based on the first value in the process of expanding the cache block.

[0062] It can be understood that the current implementation is similar to the implementation introduced in the first aspect described above, and the difference is that the first aspect introduces a scenario in which the first application is switched from the background to the foreground, and application data stored in the second storage space (such as UFS) is switched to the first storage space (such as memory). The second aspect introduces a scenario in which the first application is switched from the background to the foreground, and application data stored in the first storage space (such as memory) is switched to the second storage space (such as UFS). The related implementations are similar to those described above, and the difference is that the writing direction of the data is different, so the related implementations can be understood with reference to the above embodiments.

[0063] In a third aspect, the present application provides a memory processing method, applied to an electronic device, the electronic device including a first storage space and a second storage space, the second storage space storing a plurality of pieces of application data of a first application;

[0064] The method includes:

[0065] In a case that the first application is switched from the background to the foreground, n times of first operations are performed, the first operation comprising: writing a data set into a cache block applied from the first storage space, wherein the data set comprises at least one piece of application data in the plurality of pieces of application data of the second storage space, a size of the data set is less than or equal to a size of the cache block, and n is an integer greater than or equal to 3;

[0066] Further, n times of second operations are performed, the second operation comprising: applying the cache block from the first storage space, and determining a data set corresponding to the cache block; wherein the n times of first operations can correspond to the n times of second operations one by one, for example, one corresponding first operation can be performed after each second operation is performed.

[0067] Wherein a time period for performing the jth first operation is a first time period, and a time period for performing the j+1th second operation is a second time period, the first time period and the second time period overlap, and j is an integer from 1 to n-1. In an implementation, the first time period and the second time period at least partially overlap.

[0068] In a possible implementation, the first operation is performed based on a first thread, and the second operation is performed based on a second thread.

[0069] In a possible implementation, a first size of the cache block in the ith first operation is greater than or equal to a second size of the cache block in the i-1th first operation, and i is an integer from 2 to n.

[0070] In a possible implementation, in a case that i is less than or equal to 3, the first size is greater than the second size.

[0071] In a possible implementation, in a case that i is greater than 3, the size of the cache block is determined based on a bandwidth.

[0072] In a possible implementation, in a case that i is greater than 3, the first size is determined based on a first bandwidth, a second bandwidth, and the second size, the first bandwidth is a bandwidth corresponding to the i-2th first operation, and the second bandwidth is a bandwidth corresponding to the i-3th first operation.

[0073] In a possible implementation, in a case that the first bandwidth and the second bandwidth satisfy a first quantity relationship, the first size is equal to the second size.

[0074] In a case that the first bandwidth and the second bandwidth do not satisfy the first quantity relationship, the first size is greater than the second size.

[0075] In a possible implementation, the first quantity relationship includes that a ratio of the first bandwidth to the second bandwidth is less than a first preset threshold; and / or, the first quantity relationship includes that a difference between the first bandwidth and the second bandwidth is less than a second preset threshold.

[0076] In a possible implementation, when the value of i is 1, the size of the cache block in the i th first operation is a preset size.

[0077] In a possible implementation, before the i th first operation is performed, the method further includes:

[0078] performing a second operation, the second operation including: based on the first size, applying a cache block from the first storage space.

[0079] In a possible implementation, the second operation further includes: determining a data set corresponding to the cache block.

[0080] In a possible implementation, the determining of the data set corresponding to the cache block includes:

[0081] based on the first size, determining the data set corresponding to the cache block.

[0082] In a possible implementation, the determining of the data set corresponding to the cache block based on the first size includes:

[0083] In the remaining application data of the second storage space, determining a combination of at least one piece of application data with a minimum difference between the total size and the first size to obtain the data set corresponding to the cache block, the remaining application data being application data that has not been divided into the data set.

[0084] In a possible implementation, when the first size is greater than the second size, the first size and the second size satisfy a second quantity relationship.

[0085] In a possible implementation, the second quantity relationship is that the first size is T times the second size, or a difference between the first size and the second size is a first value, and T is a value greater than or equal to 1.

[0086] In a possible implementation, the first storage space is a volatile memory, and the second storage space is a non-volatile memory.

[0087] In a fourth aspect, the present application provides a memory processing method, applied to an electronic device, the electronic device including a first storage space and a second storage space, the second storage space storing a plurality of pieces of application data of a first application;

[0088] The method includes:

[0089] performing a second operation, the second operation comprising: applying a cache block from the first storage space, and determining a data set corresponding to the cache block;

[0090] performing a first operation, the first operation comprising: writing the data set into the cache block applied from the first storage space, wherein the data set comprises at least one application data of a plurality of application data of the second storage space, and a size of the data set is less than or equal to a size of the cache block.

[0091] In a possible implementation, the size of the cache block applied in the second operation in this execution is the same as the size of the cache block applied in the second operation in the previous round of execution.

[0092] In a possible implementation, the determining of the data set corresponding to the cache block comprises:

[0093] determining the data set corresponding to the cache block based on the size of the cache block.

[0094] In a possible implementation, the determining of the data set corresponding to the cache block based on the size of the cache block comprises:

[0095] determining, from the remaining application data of the second storage space, a combination of at least one application data with a size difference from the size of the cache block being the smallest, to obtain the data set corresponding to the cache block, the remaining application data being the application data that has not been divided into the data set.

[0096] In a possible implementation, the size of the cache block applied in the second operation in this execution is determined based on the bandwidth.

[0097] In a possible implementation, the size of the cache block applied in the second operation in this execution is determined based on the bandwidth corresponding to x rounds of the second operation performed before the second operation in this execution, wherein the bandwidth corresponding to the second operation is the bandwidth corresponding to the data set determined by the second operation. x can be an integer greater than or equal to 1.

[0098] In a possible implementation, the size of the cache block applied in the second operation in this execution can be determined based on the size of the cache block of the second operation in the previous round of execution, a first bandwidth, and a second bandwidth, wherein the first bandwidth is the bandwidth corresponding to the third round of the second operation before the second operation in this execution, and the second bandwidth is the bandwidth corresponding to the second round of the second operation before the second operation in this execution.

[0099] In a possible implementation, in a case where the first bandwidth and the second bandwidth satisfy a first quantity relationship, the size of the cache block of the second operation in this execution is equal to the size of the cache block corresponding to the second operation in the previous round;

[0100] In a case where the first bandwidth and the second bandwidth do not satisfy the first quantity relationship, a size of the cache block of the second operation executed this time is greater than a size of the cache block corresponding to the second operation of the previous round.

[0101] In a possible implementation, the first quantity relationship includes that a ratio of the first bandwidth to the second bandwidth is less than a first preset threshold; and / or, the first quantity relationship includes that a difference between the first bandwidth and the second bandwidth is less than a second preset threshold.

[0102] In a possible implementation, in a case where the second operation executed this time is the first round, the size of the cache block is a preset size.

[0103] In a possible implementation, a time period for executing the first operation is a first time period, and a time period for executing the second operation of the next round is a second time period, and the first time period and the second time period at least partially overlap.

[0104] In a possible implementation, the first operation is executed based on a first thread, and the second operation is executed based on a second thread.

[0105] In a fifth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory is used to store code instructions, and the processor is used to run the code instructions to execute the method described in the first aspect to the fourth aspect. The memory can also be understood as the non-volatile memory described above.

[0106] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores computer programs or instructions, when the computer programs or instructions run on a computer, the computer programs or instructions make the computer execute the method described in the first aspect to the fourth aspect.

[0107] In a seventh aspect, an embodiment of the present application provides a computer program product including a computer program, when the computer program runs on a computer, the computer program makes the computer execute the method described in the first aspect to the fourth aspect.

[0108] In an eighth aspect, the present application provides a chip or a chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run computer programs or instructions to execute the method described in the first aspect to the fourth aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0109] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0110] It should be understood that the second to eighth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0111] Figure 1 A schematic diagram illustrating the implementation of shared memory provided in an embodiment of this application;

[0112] Figure 2 A schematic diagram illustrating memory usage provided in an embodiment of this application;

[0113] Figure 3 Implementation illustration of data migration provided in the embodiments of this application Figure 1 ;

[0114] Figure 4 Implementation illustration of data migration provided in the embodiments of this application Figure 2 ;

[0115] Figure 5 A schematic diagram illustrating the data exchange process provided in this application embodiment;

[0116] Figure 6 Flowchart of the memory processing method provided in the embodiments of this application Figure 1 ;

[0117] Figure 7 A schematic diagram illustrating the correspondence between IO size and bandwidth provided in an embodiment of this application;

[0118] Figure 8 A flowchart illustrating the memory processing method provided in the embodiments of this application. Figure 2 ;

[0119] Figure 9 A schematic diagram illustrating the correspondence between cache blocks and datasets provided in embodiments of this application;

[0120] Figure 10 This is a schematic diagram illustrating the implementation of determining the dataset provided in an embodiment of this application. Detailed Implementation

[0121] To facilitate understanding, the relevant terms and concepts involved in the embodiments of this application will be introduced below:

[0122] 1. Keep the application running in the background

[0123] Application background keep-alive refers to the strategy of keeping an application active when it is switched to the background (invisible to the user). This is usually to ensure that the application can continue to perform certain tasks in the background, such as push notifications, data synchronization, and location tracking.

[0124] 2. UFS

[0125] UFS (Universal Flash Storage) is a high-performance, non-volatile storage medium primarily used in mobile devices such as smartphones and tablets.

[0126] 3. I / O intensive tasks

[0127] Among them, IO-intensive tasks refer to tasks whose execution speed is mainly limited by the waiting time of input / output (I / O) operations.

[0128] 4. CPU-intensive tasks

[0129] CPU-intensive tasks are those whose execution speed is primarily limited by the CPU's computing power; these tasks typically require a large amount of computing resources.

[0130] 5. Memory

[0131] In the technical solution of this application, memory can be a separate device, such as memory in a PC device, where PC stands for Personal Computer. Alternatively, memory can be part of the processor, such as memory in a mobile phone.

[0132] 6. Electronic equipment

[0133] The electronic devices in this application embodiment can be referred to as user equipment (UE), terminal, etc. For example, electronic devices can be mobile phones, tablets, personal digital assistants (PDAs), handheld devices with wireless communication functions, computing devices, in-vehicle devices, vehicle-mounted devices, or wearable devices, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in smart homes, etc. The form of the electronic devices is not specifically limited in this application embodiment.

[0134] To better understand the technical solution of this application, the relevant technologies involved in this application are explained below.

[0135] DMABUF is a memory buffer sharing mechanism. DMABUF stands for Direct Memory Access Buffer Sharing Framework. Its core purpose is to efficiently share memory data between different hardware devices (such as GPUs, cameras, screens, etc.), avoid unnecessary data copying, and thus improve system performance and reduce latency.

[0136] DMABUF utilizes DMA (Direct Memory Access) technology, allowing hardware devices (such as GPUs, cameras, and screens) to directly read and write to memory buffers without CPU intervention in data transfer, reducing CPU overhead and enabling different hardware devices to access the same memory area through DMABUF. For example, video data captured by a camera can be directly transferred to the GPU for processing via DMABUF. Similarly, frames rendered by the GPU can be directly sent to the display controller (screen) for display via DMABUF. This reduces the overhead of data copying and memory management. The cache block allocated based on the DMABUF mechanism can be called a DMA buffer (DMA cache block), hereinafter referred to as a cache block.

[0137] The following is combined Figure 1 Further explanation of DMABUF Figure 1 This is a schematic diagram illustrating the implementation of shared memory in an embodiment of this application.

[0138] like Figure 1 As shown, for example, an application can request a buffer block from memory based on the DAMBUF mechanism, such as buffer block 101 shown in the figure. The application can also write data A into this buffer block 101. Different hardware devices, such as the GPU, camera, and screen shown in the figure, can access this buffer block 101 to enable different hardware devices to share memory data.

[0139] Figure 1 The example described uses a single application requesting one cache block. In actual implementation, a single application can request multiple cache blocks. Furthermore, multiple applications can exist within an electronic device, each requesting its own corresponding cache block. Similarly, the cache blocks requested by an application can be shared by multiple hardware devices.

[0140] A single application typically uses 40MB to 250MB of memory. Therefore, when multiple applications are kept running in the background, the overall system memory usage for DMABUF will be high, or it can be understood as the overall system memory usage based on the DMABUF mechanism. For example, in some cases, the overall system memory usage for DMABUF can reach as high as 2GB.

[0141] The following can be combined Figure 2 To understand further, Figure 2 This is a schematic diagram illustrating the memory usage implementation provided in an embodiment of this application.

[0142] It needs to be understood. Figure 2 The cache blocks shown are all cache blocks allocated based on the DMABUF mechanism, which will not be repeated below.

[0143] like Figure 2 As shown, for example, Application 1 requests cache blocks 201 and 202 from memory, writes data A of Application 1 to cache block 201, and writes data B of Application 1 to cache block 202. Application 2 requests cache blocks 203 and 204 from memory, writes data C of Application 2 to cache block 203, and writes data D of Application 2 to cache block 204. Application 3 requests cache blocks 205, 206, and 207 from memory, writes data E of Application 3 to cache block 205, and writes data F and G of Application 3 to cache block 206. Application 4 requests cache block 208 from memory and writes data H of Application 4 to cache block 208.

[0144] Based on the example described above, if application 1, application 2, application 3 and application 4 are all in a background keep-alive state, then the cache blocks requested by these applications will be continuously occupied, resulting in a high overall DMABUF usage.

[0145] To reduce DMABUF memory usage and improve memory efficiency and application keep-alive experience, this application proposes a technical solution that allows data stored in memory to be swapped to a storage medium when the application switches to the background. When the application switches back to the foreground, the data stored in the background is swapped back into memory. The storage medium can include, but is not limited to, UFS, eMMC (embedded MultiMediaCard), DRAM (Dynamic Random Access Memory), etc. The following description uses UFS as an example.

[0146] For example, it can be combined Figure 3 andFigure 4 understanding, Figure 3 An implementation of data migration provided by an embodiment of the present application Figure 1 , Figure 4 An implementation of data migration provided by an embodiment of the present application Figure 2 .

[0147] As Figure 3 shown, in the scenario where the application program (for example, application 1) is in the foreground, the application program has applied for the cache block 301, the cache block 302 and the cache block 303 shown in the figure from the memory. And the data A of the application 1 is stored in the cache block 301, the data B of the application 1 is stored in the cache block 302, and the data C of the application 1 is stored in the cache block 303.

[0148] In the scenario where the application program switches from the foreground to the background, for example, the data A, the data B and the data C stored in the memory can be foregrounded to the UFS storage medium. Correspondingly, the cache block 101 and the cache block 102 in the memory can be released, so as to avoid the case that the application 1 still occupies the DAMBUF cache block in the memory in the case of background keep-alive. In this way, the high whole-machine occupation of the DMABUF in the case of multi-application background keep-alive can be realized.

[0149] And with reference to Figure 4 , in the scenario where the application program (for example, application 1) is in the background, the data A, the data B and the data C of the application 1 are all exchanged out and stored in the UFS storage medium. For example, there are also corresponding storage blocks in the UFS to store the data A, the data B and the data C. The implementation of the storage blocks for storing the data A, the data B and the data C is not limited by the present application, which can be one block or multiple blocks. The figure does not show the storage blocks, and the specific implementation of the storage blocks can be set according to actual needs.

[0150] In the scenario where the application program switches from the background to the foreground, for example, the data A, the data B and the data C stored in the UFS can be migrated to the memory. For example, the cache block 401, the cache block 402 and the cache block 403 can be applied for again in the memory, the data A is written into the cache block 401, the data B is written into the cache block 402, and the data C is written into the cache block 403, so as to realize the exchange of the application data stored in the UFS into the memory, so that the different hardware devices can directly read and write data based on the DMABUF in the case of foreground running of the application program.

[0151] Based on the above-mentioned process, it can be understood that in the process of switching the application from the background to the foreground, the data needs to be exchanged from the UFS into the memory. Then, in order to ensure the process experience of the application program switching to the foreground, the time consumption of data exchange is naturally shorter the better. The following will be combined Figure 5An example of a possible data swap-in process is described, Figure 5 A flowchart of a data swap-in process is provided for embodiments of the application.

[0152] For a single application, there are usually multiple application data in the running process of the application. For example, in the example described above, application 1 has three application data, data A, data B and data C. In the case of switching the application to the background, the multiple application data of the application are written into the UFS storage medium, and in the case of switching the application to the foreground, the multiple application data stored in the UFS storage medium need to be swapped into the memory.

[0153] In an implementation, the multiple application data can be processed in sequence, for example. The implementation of writing data from the UFS storage medium into the memory usually involves two steps, one step is data preparation, and the other step is file IO. The data preparation can include operations such as applying for a buffer, establishing a mapping relationship (such as a mapping relationship between the virtual address and the physical address of the data), and the file IO is to write data into the applied buffer.

[0154] The process of processing the multiple application data in sequence is described in detail below, taking the three application data of data A, data B and data C of application 1 as an example.

[0155] Referring to Figure 5 In period 1, data preparation can be performed for data A, which can include the operation of applying for a buffer 401 from the memory, where the size of the buffer 401 is the same as the size of data A, that is, the size of data A is as large as the size of the buffer applied. And it can also include the operation of establishing a mapping relationship between the buffer 401 and data A, so that data A can be written into the buffer 401 subsequently. Then, in period 2, file IO can be performed for data A, that is, data A is written from UFS to buffer 401. In this way, data A is swapped from UFS to the memory.

[0156] And in period 3, data preparation can be performed for data B, which can include the operation of applying for a buffer 402 from the memory, where the size of the buffer 402 is the same as the size of data B, that is, the size of data B is as large as the size of the buffer applied. And it can also include the operation of establishing a mapping relationship between the buffer 402 and data B, so that data B can be written into the buffer 402 subsequently. Then, in period 4, file IO can be performed for data B, that is, data B is written from UFSF to buffer 402. In this way, data B is swapped from UFS to the memory.

[0157] Furthermore, during period 5, data preparation can be performed on data C. This preparation may include allocating cache block 403 from memory, where the size of cache block 403 is the same as the size of data C; that is, the size of the cache block allocated corresponds to the size of data C. It may also include establishing a mapping relationship between cache block 403 and data C, so that data C can be written to cache block 403 later. Then, during period 6, file I / O can be performed on data C, that is, writing data C from UFS to cache block 403. This completes the swapping of data C from UFS into memory.

[0158] Based on the process described above, it is possible to swap multiple pieces of application data from UFS into memory. In this implementation, data preparation and file I / O operations are performed for each piece of data. The processing of each piece of data is similar, and the process is repeated until all pieces of application data have been processed.

[0159] During the process of an application switching from the foreground to the background, multiple data entries of varying sizes are typically swapped in, such as 10 to 100 data entries. For example, the size of a single data entry can range from 10KB to 14MB. In actual implementation, the number of data entries and the size of each data entry can be determined according to the actual situation of the application, and this embodiment does not impose any restrictions on this.

[0160] However, among the numerous application data entries generated during application startup, a large portion of these entries are relatively small in size (or data volume; the data size mentioned in this article can be understood as data volume, where data size is also the data size). For example, 80% of the application data is less than 1MB. Therefore, sequentially performing data preparation and file I / O processing on multiple application data entries leads to low bandwidth utilization.

[0161] Based on the above Figure 5 Taking the scenario described above as an example, if data A is 20KB in size and time period 2 is 15ms long, then the actual bandwidth within time period 2 is 1.365MB / s. Assuming the electronic device can support a bandwidth of 500MB / s during the process of writing data from UFS to memory, the bandwidth utilization within time period 2 is very low. The same applies to transmitting data B in time period 4 and data C in time period 6; both suffer from low bandwidth utilization.

[0162] In addition, in the data import processing process described above, the data preparation and file IO implementation are serial, that is, data preparation and file IO are performed for each data in turn, and the data preparation of the next data can be started only on the basis that the file IO of the data is executed.

[0163] For example, in the data import processing example described above Figure 5 In the data import processing example described above, in time period 1, time period 3 and time period 5, the CPU is performing data preparation work, and the bandwidth is not utilized. In time period 2, time period 4 and time period 5, the bandwidth is used for file writing and reading, and the CPU is not utilized.

[0164] To solve the above technical problems, the present application provides a memory processing method, which can aggregate multiple application data for file IO, and the IO aggregation can improve the bandwidth utilization. However, excessive aggregation can enlarge the proportion of data preparation time in the processing process, for example, it can cause the file IO to be delayed. In addition, excessive aggregation can also affect the bandwidth of other IO in the system, for example, in the same time period, if the aggregated data size is too large, the bandwidth occupation is too high, which can affect the available bandwidth of the remaining IO tasks. Therefore, the technical solution of the present application proposes a dynamic IO aggregation method to realize IO aggregation according to the actual bandwidth utilization, and avoid excessive aggregation.

[0165] In addition, the technical solution of the present application can also separate the data preparation and file IO processes to avoid serial processing and improve the use efficiency of bandwidth and CPU resources.

[0166] Based on the above description, the memory processing method provided by the embodiments of the present application will be described in combination with specific embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes can not be described in some embodiments.

[0167] Figure 6 Flowchart of the memory processing method provided by the embodiments of the present application Figure 1 , Figure 7 Correspondence diagram of IO size and bandwidth provided by the embodiments of the present application Figure 8 Flowchart of the memory processing method provided by the embodiments of the present application Figure 2 , Figure 9 Correspondence diagram of cache block and data set provided by the embodiments of the present application Figure 10 Implementation diagram for determining data set provided by the embodiments of the present application.

[0168] With reference to Figure 6 The memory processing method provided in the embodiments of the present application can include the following steps.

[0169] S601, a cache block is applied for from the memory, and the size of the cache block is Mi, where i represents the execution round of data preparation.

[0170] As can be understood from the above description, in the process of switching the application data of an application program from a UFS into the memory, two processes of data preparation and file IO are usually involved. The implementation in the embodiments is similar, and involves the processes of data preparation and file IO, and the process of data preparation specifically involves the operations of applying for a cache block and establishing a mapping relationship.

[0171] In the technical solution of the present application, the process of data preparation is executed for multiple rounds, and in the embodiments, i represents the execution round of data preparation, where i is an integer greater than or equal to 1. It can be understood that the initial value of i is 1, and the value of i is increased by 1 after each round of data preparation is completed, and the next round of data preparation is performed.

[0172] In each round of data preparation, the operation of applying for a cache block from the memory is involved, and the applied cache block is used to store the switched-in application data. In the embodiments, Mi is used to represent the size of the cache block applied for in the i th round of data preparation. For example, the size Mi of the cache block to be applied for in the current round can be determined in advance before each round of data preparation, and then a cache block with the size Mi is applied for from the memory. For example, the size Mi of the cache block can be determined according to the bandwidth.

[0173] In one implementation, the size of the cache block applied for in the process of data preparation can be determined based on the bandwidth. For example, for the next round (i+1 th round) of data preparation to be executed, the size of the cache block for the i+1 th round of data preparation can be determined based on the bandwidth corresponding to x rounds of data preparation executed before the i+1 th round. The value of x can range from 1 to i, and the bandwidth corresponding to which rounds of data preparation before the i+1 th round is selected can also be set according to actual needs, and the embodiments do not limit this. For example, the bandwidth corresponding to the data preparation can be obtained after the file IO corresponding to the data preparation is executed, for example, calculated based on the IO data amount and IO duration of the file IO. Alternatively, the bandwidth corresponding to a round of data preparation can also be obtained in a predictive manner after the round of data preparation is completed, and the embodiments do not limit this.

[0174] On this basis, the following takes the bandwidth corresponding to the i-2th round of data preparation and the bandwidth corresponding to the i-1th round of data preparation as an example to introduce that the cache block size of the i+1th round of data preparation is determined, and in this implementation, the value of x is 2. It can be understood that when the value of x is other values, the implementation manner is similar, and details are not described herein.

[0175] For example, after the i th round of data preparation ends, the bandwidth change condition can be determined based on the bandwidth corresponding to the i-2th round of data preparation and the bandwidth corresponding to the i-1th round of data preparation. In the case where the bandwidth change condition does not satisfy the preset condition, the size of the cache block corresponding to the i+1th round of data preparation can be determined by enlarging the size of the cache block corresponding to the i th round of data preparation. Or, in the case where the bandwidth change condition satisfies the preset condition, the size of the cache block corresponding to the i+1th round of data preparation can be determined by continuing to use the size of the cache block corresponding to the i th round of data preparation, that is, without enlarging the size, to avoid problems caused by excessive data aggregation.

[0176] In the implementation introduced above, the value of i can be 1-n, where n is the total number of execution rounds of data preparation, and n can be an integer greater than or equal to 1. For the case where the value of i is 1, that is, the first round of data preparation, the size of the cache block of the first round of application can be pre-set, that is, the value of M1 can be pre-set. For example, the value of M1 can be 1MB or 2MB, and the size of the cache block of the first round can be set according to actual needs, which is not limited in the embodiment. For the case where the value of i is 2-n, the size of the cache block can be determined according to the size of the cache block in the i-1th round of data preparation.

[0177] It can be understood that after the first round of data preparation ends, the size of the cache block corresponding to the second round of data preparation needs to be determined. In the implementation introduced above, whether the size of the cache block needs to be enlarged is determined based on the bandwidth corresponding to the two continuous rounds of data preparation completed before the current round. However, after the first round of data preparation ends, the bandwidth corresponding to the data set of the first round of data preparation cannot be obtained at the moment because the file IO corresponding to the first round of data preparation has not been executed, and thus the bandwidth change condition cannot be determined. For example, the size of the cache block corresponding to the second round of data preparation can be obtained by enlarging the size of the cache block corresponding to the first round of data preparation. That is, when the size of the cache block corresponding to the second round of data preparation is determined, the bandwidth enlargement condition is not judged, but the size is directly enlarged.

[0178] After the data preparation of the second round is completed, the size of the cache block corresponding to the data preparation of the third round needs to be determined. In the implementation described above, whether the size of the cache block needs to be enlarged is determined based on the bandwidths corresponding to the data preparations of the two rounds completed before the current round. However, after the data preparation of the third round is completed, only the bandwidth corresponding to the data preparation of the first round can be obtained (because the file IO corresponding to the data preparation of the first round has been completed), but the bandwidth corresponding to the data preparation of the second round cannot be obtained (because the file IO corresponding to the data preparation of the second round has not been completed), and at this time, the bandwidth change cannot be determined based on the bandwidths corresponding to the file IO of the two rounds. Therefore, the size of the cache block corresponding to the data preparation of the second round can be enlarged to obtain the size of the cache block corresponding to the data preparation of the third round. That is, when the size of the cache block corresponding to the data preparation of the third round is determined, whether the bandwidth is enlarged is not determined, but the size is directly enlarged.

[0179] That is, when the value of i is equal to 2 or equal to 3, the size of the cache block can be directly enlarged based on the size of the current cache block to determine the size of the cache block of the next round, without determining the bandwidth change.

[0180] When the value of i is greater than 3, the bandwidths corresponding to the data preparations of the two rounds before the current round can be obtained, and therefore, whether the size of the cache block needs to be enlarged can be determined based on the bandwidth change.

[0181] S602, determine the mapping relationship between the data set Xi and the cache block.

[0182] In the implementation of each round of data preparation, in addition to the operation of applying for a cache block, the operation of determining the mapping relationship between the data and the cache block is also included, that is, it is necessary to determine which data is written into the cache block applied in the current round. It can be determined according to the description above that in one implementation, a cache block with a corresponding size can be applied based on the size of one application data, and the mapping relationship between the application data and the cache block is established subsequently. Therefore, in the case that the size of a single application data is small, the problem of low bandwidth utilization rate described above exists.

[0183] Therefore, in the technical solution of the present application, the mapping relationship between the data set and the cache block can be determined, and at least one application data can be included in one data set, so that multiple data can be aggregated together to perform file IO, thereby avoiding the problem of low data utilization rate caused by the separate execution of file IO by each application data.

[0184] For example, at least one piece of application data can be determined from the application data remaining in the UFS in each round of data processing, so as to determine the data set corresponding to the current round of data processing. In the embodiment, Xi represents the data set determined in the i th round of data preparation. The total size of the at least one piece of application data is less than or equal to the size Mi of the cache block.

[0185] In an implementation, for example, the application data with the minimum difference between the total size and the size Mi of the cache block can be selected from the application data remaining in the UFS to form the data set Xi. There can be multiple combinations of the application data with the minimum difference between the total size and the size Mi of the cache block, for example, the selection can be in a certain order or can be random, which is not limited in the embodiment. This implementation can maximize the use of the cache block of the application and avoid memory waste.

[0186] S603, write the data set Xi into the cache block of the application.

[0187] After the steps of S601 and S602 are completed, one round of data preparation is completed. As described above, the mapping relationship between the data set Xi and the cache block of the application can be determined in one round of data preparation. Then, the file IO for the data set Xi can be submitted to write the data set Xi into the cache block of the application.

[0188] In the technical solution of the embodiment, the data preparation and the file IO processing can be executed asynchronously. For example, the data preparation and the file IO processing can be executed asynchronously. Figure 8 It is understood that after one round of data preparation is completed, on the one hand, the file IO for the data set Xi involved in the current round of data preparation is submitted, and on the other hand, the next round of data preparation can be started synchronously.

[0189] It can be understood that the file IO operation, i.e., the operation of writing the data set into the cache block of the application, is performed after each round of data preparation. Therefore, it can be determined that the file IO operation is also performed n times. That is, in the technical solution of the application, the file processing operation, which can be referred to as a first operation, is performed n times. In the technical solution of the application, the data preparation operation, which can be referred to as a second operation, is also performed n times.

[0190] S604, determine whether the data processing is completed. If yes, end the process. If no, perform S605.

[0191] Further, after completing a round of data preparation, it can also be determined whether the swap-in of application data in the UFS is completed, i.e., whether there is still unprocessed application data in the UFS, wherein the unprocessed application data is application data for which a mapping relationship has not been established.

[0192] In one case, if it is determined that the data processing is completed, the data preparation process can be ended, because there is no remaining application data that needs to be prepared.

[0193] In another case, if it is determined that the data processing is not completed, the next round of data preparation and file IO processing needs to be performed, because the subsequent steps and the next round of data preparation are continued to be performed.

[0194] S605, determining the size of the cache block of the next round based on the bandwidth corresponding to the data set of the two consecutive rounds before the current round.

[0195] In the case where the data processing is not completed, the next round of data preparation needs to be performed. As described above, the size of the cache block of the application can be determined in advance before each round of data preparation is performed.

[0196] In one implementation, the size of the cache block of the next round (i+1) can be determined based on the bandwidth corresponding to the data set of the two consecutive rounds before the current round, i.e., the bandwidth corresponding to the data set of the round i-2 and the bandwidth corresponding to the data set of the round i-1. The reason for selecting the round i-2 and the round i-1 is that the bandwidth of the data set can be determined only after the file IO is completed. At the start time of the data preparation of the round i+1, the file IO corresponding to the data preparation of the round i is usually still in the execution stage, and the bandwidth corresponding to the data set of the round i cannot be obtained. However, at the start time of the data preparation of the round i+1, the file IO corresponding to the data preparation of the round i-2 and the file IO corresponding to the data preparation of the round i-1 have been completed, and thus the bandwidths corresponding to the two rounds of file IO can be obtained, so that the data preparation of the round i+1 can be started directly without waiting. Therefore, in order to ensure that multiple rounds of data preparation can be continuously performed, the bandwidth corresponding to the data set of the round i-2 and the bandwidth corresponding to the data set of the round i-2 are compared in the embodiment to determine the size of the cache block of the round i+1.

[0197] For example, the bandwidth of the file IO for the data set of the round i-2 can be obtained after the file IO corresponding to the data preparation of the round i-2 is completed, and the bandwidth of the file IO for the data set of the round i-1 can be obtained after the file IO corresponding to the data preparation of the round i-1 is completed, and then whether the size of the cache block is enlarged is determined based on the bandwidths of the two rounds.

[0198] For facilitating the following exemplary description, the following introduces a possible correspondence between IO size and bandwidth. Figure 7 A possible correspondence between IO size and bandwidth is introduced.

[0199] In Figure 7 , the horizontal axis represents IO size, i.e., the total size of the data set determined in the data preparation, and the vertical axis represents bandwidth. Exemplarily, when the total size of the data set is 10 MB, the corresponding bandwidth is 2.6 GB / s, and the correspondence between the total size of the data set and the bandwidth can be understood with reference to Figure 7 , which will not be described herein again.

[0200] Then, after the completion of the data preparation in each round, the file IO for the data set determined in the data preparation in this round is submitted, and after the completion of the file IO, the bandwidth corresponding to the data set in this round is determined.

[0201] Further, the following introduces the implementation process of determining the size of the cache block in the next round based on the bandwidths corresponding to the data sets in the two rounds before the current round. For facilitating the description, the bandwidth corresponding to the data set in the i-2th round is denoted as IO i-2 , and the bandwidth corresponding to the data set in the i-1th round is denoted as IO i-1 .

[0202] In an implementation manner, the ratio of IO i-1 and IO i-2 , i.e., is determined.

[0203] After that, the ratio of IO i-1 and IO i-2 is compared with a first preset threshold. The first preset threshold is a threshold for judging whether the size of the cache block needs to be enlarged. For example, the first preset threshold can be a predefined fixed value, or the first preset threshold can also be an empirical value, etc.

[0204] When the ratio is greater than or equal to the first preset threshold, it means that the bandwidth in the last round has a relatively large growth compared with the bandwidth in the round before the last round, which means that the bandwidth gain caused by the increase of the size of the cache block is still in a relatively fast growth stage, and thus it can be determined that the current bandwidth does not satisfy the preset condition. Therefore, the size of the cache block in the current round (i.e., the i th round) can be enlarged to obtain the size of the cache block in the next round.For example, in each size expansion process, the size can be multiplied by T, where T is greater than or equal to 1, and the value of T can be set according to actual needs. For example, the value of T can be a preselected fixed value, or the value of T can also be a value that makes the sizes of the plurality of cache blocks meet a certain data relationship, for example, the certain data relationship can be equal ratio or equal difference, or the certain data relationship can be set according to actual needs, as long as the value of T can make the size of the cache block in the next round larger than the size of the cache block in the previous round.

[0205] For example, the size of the cache block in the i th round is 4 MB, and the value of T is fixed at 2. Then, the size of the cache block in the i th round can be expanded to obtain the size of the cache block in the i + 1 th round, which is 8 MB.

[0206] In the size expansion process, a first value can also be added to the current size to obtain the expanded size, where the first value can be a preselected fixed value, or the first value can also be a value that makes the sizes of the plurality of cache blocks meet a certain data relationship, which is similar to the above description and will not be repeated here.

[0207] For example, if the value of T is less than the first preset threshold, it means that the bandwidth of the last round and the bandwidth of the round before the last round have a small growth rate, which means that the bandwidth gain brought by the increase of the size of the cache block is already small. That is, if the size of the cache block continues to be expanded, the bandwidth can still be increased, but the growth of the bandwidth is small. As described above, if the aggregated data size is too large, it will affect the remaining IO and the data preparation time will be too long. Therefore, in this case, it can be determined that the current bandwidth meets the preset condition. Therefore, the size of the cache block can no longer be expanded, and the size of the cache block corresponding to the current round is determined as the size of the cache block corresponding to the next round.

[0208] For example, if the size of the cache block in the i th round is 8 MB, and it is determined that the size of the cache block is no longer expanded based on the bandwidth of the i th round data set and the bandwidth of the i - 1 th round data set, then the size of the cache block in the i + 1 th round can be determined as 8 MB.

[0209] In addition, in another implementation manner, the difference between IO i-1 and IO i-2 , that is, IO i-1 -IO i-2 , and then IO i-1 and IOi-2 The difference value is compared with a second preset threshold. The second preset threshold is a threshold for determining whether the size of the cache block needs to be enlarged. For example, the second preset threshold can be a predefined fixed value, or the second preset threshold can also be an empirical value.

[0210] When the difference value is greater than or equal to the second preset threshold, it represents that the bandwidth of the last round and the bandwidth of the round before the last round have a relatively large growth rate, which represents that the bandwidth gain brought by the size increase of the cache block is still in a relatively fast growth stage. Therefore, it can be determined that the current bandwidth does not meet the preset condition. Therefore, the size of the cache block corresponding to the current round can be enlarged to obtain the size of the cache block corresponding to the next round.

[0211] Alternatively, when the ratio is less than the first preset threshold, it represents that the bandwidth of the last round and the bandwidth of the round before the last round have a relatively small growth rate, which represents that the bandwidth gain brought by the size increase of the cache block has a relatively small growth rate. Therefore, the size of the cache block can no longer be expanded, and the size of the cache block corresponding to the current round is determined as the size of the cache block corresponding to the next round.

[0212] In actual implementation, the specific implementation of determining the size of the cache block of the next round based on the bandwidths corresponding to the data sets of the two consecutive rounds before the current round can be selected according to actual needs. The present embodiment does not limit this, as long as the bandwidth change is determined based on the bandwidths corresponding to the data sets of the two consecutive rounds before the current round, and then it is determined whether to continue to enlarge the size of the cache block.

[0213] For example, the corresponding relationship introduced in Figure 7 may be referred to to introduce a specific example. In Figure 7 , the horizontal axis represents the IO size, that is, the total size of the data set determined in the data preparation. The vertical axis represents the bandwidth.

[0214] Referring to the first point on the curve shown in Figure 7 , for example, the size of the data set of the i-2 round is 1 MB, it can be determined that the corresponding bandwidth is 0.5 GB / s according to Figure 7 . For another example, referring to the second point on the curve shown in Figure 7 , for example, the size of the data set of the i-1 round is 2 MB, it can be determined that the corresponding bandwidth is 1 GB / s according to Figure 7It can be determined that the corresponding bandwidth is 0.8 GB / s. Taking the ratio implementation as an example, the ratio of the bandwidth of the two rounds is 0.8 / 0.5 = 1.6, for example, the first preset threshold is set to 1.4. It can be determined that the ratio of the bandwidth is greater than the first preset threshold, so the expansion of the cache block can be performed, for example, based on the size of the cache block of the i-th round (for example, 2 MB), assuming that the value of the magnification factor T is fixed to 2, the expansion is performed to obtain the size of the cache block of the next round (i+1-th round) (that is, 2 MB x 2 = 4 MB).

[0215] For another example, referring to the 6th point on the curve shown in Figure 7 , the size corresponding to the i-2th round of data set is 16 MB, referring to Figure 7 , it can be determined that the corresponding bandwidth is 3.2 GB / s. For another example, referring to the 7th point on the curve shown in Figure 7 , the size corresponding to the i-1th round of data set is 32 MB, referring to Figure 7 , it can be determined that the corresponding bandwidth is 4 GB / s. Taking the ratio implementation as an example, the ratio of the bandwidth of the two rounds is 4 / 3.2 = 1.17, for example, the first preset threshold is set to 1.4. It can be determined that the ratio of the bandwidth is less than the first preset threshold, so the expansion of the cache block can be stopped, that is, the size of the cache block of the i-th round (for example, 2 MB) can be determined as the size of the cache block of the next round (i+1-th round) (that is, 2 MB).

[0216] After determining the size of the cache block of the i+1th round, the steps starting from S601 described above can be continued to be performed based on the determined size of the cache block to perform the next round of data preparation and file IO. Repeat the execution of multiple rounds until the complete swap-in of all data of the application in the UFS is completed.

[0217] The above is an introduction to the process of data preparation. It can be understood from the above introduction that the operation of data preparation and the operation of file IO are one-to-one, that is, in the case of performing n rounds of data preparation, n rounds of file IO will also be performed. Further, the implementation of the size of the cache block can also be described as: the first size of the cache block in the i-th file IO operation is greater than or equal to the second size of the cache block in the i-1th file IO operation.

[0218] Among them, in the case where the value of i is equal to 2 or equal to 3 (that is, the second round of file IO or the third round of file IO is performed), the bandwidth condition does not need to be judged in the data preparation process, but the expansion of the cache block is directly performed, so the first size is greater than the second size.

[0219] and, in the case where i is greater than 3, there are two cases for the first size and the second size, the first size is greater than the second size, or the first size is equal to the second size.

[0220] For example, in the case where the first condition is satisfied, the first size is equal to the second size, that is, the expansion of the cache block is not performed in the data preparation process. In the case where the first condition is not satisfied, the first size is greater than the second size, that is, the expansion of the cache block is performed in the data preparation process.

[0221] The first condition includes that the first bandwidth and the second bandwidth satisfy a first quantity relationship, wherein the first bandwidth is the bandwidth corresponding to the i-2th first operation, and the second bandwidth is the bandwidth corresponding to the i-3th first operation. That is, under the condition that the first bandwidth corresponding to the i-2th first operation and the second bandwidth corresponding to the i-3th first operation satisfy the first quantity relationship, the size of the cache block corresponding to the i th first operation is greater than the size of the cache block corresponding to the i-1th first operation, that is, in this case, the size of the cache block is enlarged in the data preparation process.

[0222] The first bandwidth may be represented as IO i-2 , and the second bandwidth may be represented as IO i-3 .

[0223] The first quantity relationship may include that the ratio of the first bandwidth to the second bandwidth is less than a first preset threshold value; and / or the first quantity relationship includes that the difference between the first bandwidth and the second bandwidth is less than a second preset threshold value.

[0224] The first condition is used to determine whether to expand the first size to determine the implementation of the second size. The implementation process of determining the size of the cache block in the next round based on the bandwidths corresponding to the two consecutive rounds before the current round is similar to the above description, except that the round number used in the description process is different. Because the above description describes the determination of the size of the cache block in the next round (i+1th round), the two rounds before the current round are selected. Because the current description is actually for the i+1th round. The implementation described here is the relationship between the i th round cache block size and the i-1th round cache block size, so the i-3th round and the i-2th round are selected. Because the current description is actually for the i th round. The specific implementation can be understood with reference to the above description.

[0225] The above-described process will be further described in detail below with reference to a specific example, Figure 8 and Figure 9The Communist Party of China shows the process of 5 rounds of data processing, and after each round of data processing, there will be file IO processing. The following will be introduced in detail for each round of data process in combination with the drawings.

[0226] 1. First round of data processing

[0227] As shown in Figure 8 , in the first round of data processing, for example, a cache block 901 can be applied from the memory. As can be understood from the above description of the embodiment, the size of the first round of cache block can be preset, that is, the size of the applied cache block can be preset. For example, the size of the cache block is set to 1MB, so the size of the cache block 901 is, for example, 1MB. In actual implementation, the size of the cache block can be set arbitrarily according to actual needs, and the embodiment does not limit this.

[0228] Then, the data set corresponding to the cache block 901 needs to be determined. In Figure 9 , the data set corresponding to each cache block is shown, and each data set can contain at least one application data.

[0229] For example, the implementation of determining the data set can be understood in combination with Figure 10 , as shown in Figure 10 . For example, there are currently 11 application data, which are data A to data K, and the data size corresponding to each application data is also shown in the figure, which is not described here.

[0230] In this embodiment, for example, among the 11 application data, the combination of application data with the smallest size difference from the size of the cache block 901 is selected to obtain the data set corresponding to the cache block 901. For example, in the first round of data preparation, the size of the cache block 901 is 1MB, so among the multiple application data shown in Figure 10 , the data combination with the smallest size difference from 1MB is selected, and for example, data A and data E can be determined as a data set, and the total size of this data set is 1MB.

[0231] Referring to Figure 9 , the data set 1 corresponding to the cache block 901 contains data A and data E. Then the mapping relationship between the cache block 901 and the data set 1 can be established. Then the file IO for the data set 1 can be submitted to write the data set 1 into the cache block 901, that is, data a and data e are aggregated and written into the cache block 901. Compared with the implementation of submitting file IO for data A and data E separately, this implementation can improve the bandwidth utilization.

[0232] and referring to Figure 8It can be determined that in the embodiment, the data preparation and the file IO are asynchronously completed. That is, after the first round of data preparation is completed, the IO processing for the data set involved in the first round of data preparation is submitted, and while the IO processing is being performed, the next round of data preparation is performed, that is, the second round of data preparation performed in time period 2 as shown in FIG. 8. Figure 8 As shown in FIG. 8, the time period corresponding to the first round of data preparation is time period 1, the time period corresponding to the file IO after the first round of data preparation is time period 6, and the time period corresponding to the second round of data preparation is time period 2. There is an overlap between time period 6 and time period 2, which means that the file IO and the data preparation are asynchronously performed. Referring to FIG. 8, Figure 8 As shown in FIG. 8, the time period corresponding to the first round of data preparation is time period 1, the time period corresponding to the file IO after the first round of data preparation is time period 6, and the time period corresponding to the second round of data preparation is time period 2. There is an overlap between time period 6 and time period 2, which means that the file IO and the data preparation are asynchronously performed. Referring to FIG. 8, Figure 8 It can also be understood that the start time of time period 6 is after the end time of time period 1, that is, the file IO corresponding to the data preparation needs to wait until the completion of the data preparation before being performed.

[0233] After the first round of data processing is completed, it can be determined whether there is remaining data, and if so, the second round of data processing is continued.

[0234] Referring to the above description of the embodiment, it can be understood that the technical solution of the present application can determine whether to expand the size of the cache block based on the bandwidth corresponding to the data processing of the previous two rounds (the i-2th round and the i-1th round) to obtain the size of the cache block of the next round. However, after the first round of data processing is completed, only one round of data processing is performed, and the file IO corresponding to the first round of data processing needs to wait until the completion of the file IO before obtaining the bandwidth corresponding to the file IO of the first round, that is, the bandwidth of the file IO of the first round cannot be obtained at present, so the bandwidth change cannot be determined based on two rounds of data processing. Therefore, for the second round of data processing, the size of the cache block can be directly expanded without determining the bandwidth change.

[0235] For example, the size of the expanded cache block can be obtained by multiplying the size of the current cache block by T, and the size is the size determined for the next round of data preparation. For example, if T is 2, the size of the cache block corresponding to the second round of data processing is 2MB.

[0236] 2. Second round of data processing

[0237] As shown in FIG. 8, in the second round of data processing, for example, a cache block 902 of 2MB can be applied from the memory. Figure 8 After that, the data set corresponding to the cache block 902 needs to be determined. For example, the data set can be determined in combination with the data set determined in the first round of data processing.

[0238] Figure 10 As shown in FIG. 8, in the second round of data processing, for example, a cache block 902 of 2MB can be applied from the memory.

[0238] After that, the data set corresponding to the cache block 902 needs to be determined. For example, the data set can be determined in combination with the data set determined in the first round of data processing. Figure 10 As shown in FIG. 8, in the second round of data processing, for example, a cache block 902 of 2MB can be applied from the memory.Figure 10 For example, there are currently 11 pieces of application data, which are data A to data K. As described above, it can be determined that, in the first round of data processing, data A and data E have been determined as data in the data set corresponding to the cache block 901, which means that the file IOs of the two pieces of data have been submitted, and thus the two pieces of data do not participate in subsequent data preparation. Therefore, in the second round of data processing, data selection is performed on the remaining application data to determine the data set.

[0239] In this embodiment, the remaining application data and the data other than data A and data E, for example, can be selected from the 9 pieces of application data to obtain a combination of application data whose total size is closest to the size of the cache block 902. For example, in the second round of data preparation, the size of the cache block 902 is 2 MB, and thus a combination of data whose total size is closest to 2 MB can be selected from the multiple remaining application data as shown in FIG. 9. For example, data G and data K can be determined as a data set, and the total size of the data set is 1.9 MB. Figure 10

[0240] Referring to FIG. 9, the data set 2 corresponding to the cache block 902 includes data G and data K. Then, a mapping relationship between the cache block 902 and the data set 2 can be established. Then, the file IO for the data set 2 can be submitted to write the data set 2 into the cache block 902, that is, data G and data H are aggregated and written into the cache block 902. Compared with the implementation of submitting the file IO for data G and data K separately, this implementation can improve the bandwidth utilization. Figure 9

[0241] Referring to FIG. 9, the data set 2 corresponding to the cache block 902 includes data G and data K. Then, a mapping relationship between the cache block 902 and the data set 2 can be established. Then, the file IO for the data set 2 can be submitted to write the data set 2 into the cache block 902, that is, data G and data H are aggregated and written into the cache block 902. Compared with the implementation of submitting the file IO for data G and data K separately, this implementation can improve the bandwidth utilization. Figure 8 It can be determined that, in this embodiment, the data preparation and the execution of the file IO are completed asynchronously. That is, after the second round of data preparation is completed, the IO processing for the data set involved in the second round of data preparation is submitted, and the next round of data preparation is performed at the same time, that is, the third round of data preparation is performed in time period 3 as shown in FIG. 9. Figure 8 As shown in FIG. 9, the time period corresponding to the second round of data preparation is time period 2, the time period corresponding to the file IO after the second round of data preparation is time period 7, and the time period corresponding to the third round of data preparation is time period 3. Time period 7 and time period 3 overlap, which means that the file IO and the data preparation are performed asynchronously. Figure 8 As shown in FIG. 9, the time period corresponding to the second round of data preparation is time period 2, the time period corresponding to the file IO after the second round of data preparation is time period 7, and the time period corresponding to the third round of data preparation is time period 3. Time period 7 and time period 3 overlap, which means that the file IO and the data preparation are performed asynchronously. Figure 8 It can also be understood that the start time of time period 7 is after the end time of time period 2, that is, the file IO corresponding to the data preparation needs to wait until the completion of a round of data preparation.

[0242] ​​After the data processing of the second round is completed, it can be determined whether there is still remaining data, and if so, the third round of data processing described below is continued.

[0243] As can be understood from the above description of the embodiments, the technical solution of the present application can determine whether to expand the size of the cache block based on the bandwidth corresponding to the data processing of the two previous rounds (the i-2th round and the i-1th round) to obtain the size of the cache block of the next round. However, after the data processing of the second round is completed, only two rounds of data processing have been performed, and the bandwidth corresponding to the file IO of the second round needs to be executed to obtain the bandwidth corresponding to the file IO of the second round. That is, the bandwidth of the file IO of the second round cannot be obtained at present, and therefore the bandwidth change cannot be determined based on the data processing of the two rounds. Therefore, for the third round of data processing, the size of the cache block can be directly enlarged without determining the bandwidth change.

[0244] For example, the size of the enlarged cache block can be obtained by multiplying the size of the current cache block by T, and the size is the size determined for the next round of data preparation. For example, if T is 2, the size of the cache block corresponding to the third round of data processing is equal to 4MB.

[0245] 3. Third round of data processing

[0246] Referring to Figure 8 , for example, a cache block 903 with a size of 4MB can be applied from the memory.

[0247] Then, the data set corresponding to the cache block 903 needs to be determined. For example, the data set can be determined in combination with Figure 10 The implementation of determining the data set can be understood as shown in Figure 10 , for example. For example, there are currently 11 application data, which are data A to data K. Similar to the above description, in the third round of data processing, data selection will be performed in the remaining application data excluding data A, data E, data G, and data K to determine the data set. Data A, data E, data G, and data K are data that have been selected in the previous rounds of data processing.

[0248] In this embodiment, for example, a combination of application data with the smallest size difference from the size of the cache block 903 can be selected from the remaining application data excluding data A, data E, data G, and data K to obtain the data set corresponding to the cache block 903. For example, in the third round of data preparation, the size of the cache block 903 is 4MB, and therefore a combination of application data with a total size of 4MB can be selected from the remaining application data excluding data A, data E, data G, and data K. Figure 10Among the illustrated multiple pieces of remaining application data, a data combination with a data size sum that is closest to 4MB is selected, so that data B, data D and data H are determined as a data set, and the total size of the data set is 4MB.

[0249] Referring to Figure 9 The data set 3 corresponding to the cache block 903 contains data B, data D and data H. Then, a mapping relationship between the cache block 903 and the data set 3 can be established. Then, a file IO for the data set 3 can be submitted to write the data set 3 into the cache block 903, that is, data B, data D and data H are aggregated and written into the cache block 903. Compared with the implementation of separately submitting file IOs for data B, data D and data H, this implementation can improve the bandwidth utilization.

[0250] Referring to Figure 8 It can be determined that in the embodiment, the data preparation and the execution of the file IO are completed asynchronously. That is, after the completion of the third round of data preparation, an IO process for the data set involved in the third round of data preparation is submitted, and the next round of data preparation is performed at the same time as the IO process, that is Figure 8 The fourth round of data preparation performed in the time period 4 is illustrated. As Figure 8 illustrated, the time period corresponding to the third round of data preparation is the time period 3, the time period corresponding to the file IO after the third round of data preparation is the time period 8, and the time period corresponding to the fourth round of data preparation is the time period 4. The time period 8 and the time period 4 overlap, which means that the file IO and the data preparation are executed asynchronously. Referring to Figure 8 It can also be understood that the start time of the time period 8 is after the end time of the time period 3, that is, the file IO corresponding to the data preparation needs to wait for the completion of a round of data preparation.

[0251] After the completion of the third round of data processing, it can be determined whether there is remaining data, and if so, the fourth round of data processing described below is continued.

[0252] Referring to the above description of the embodiment, it can be understood that the technical solution of the present application can determine whether to expand the size of the cache block based on the bandwidth corresponding to the data processing of the previous two rounds (the i-2th round and the i-1th round) to obtain the size of the cache block of the next round. For example, based on the file IO processing (that is, the file IO for the data set 1) corresponding to the data processing of the first round, the corresponding bandwidth IO1 can be determined, and based on the file IO processing (that is, the file IO for the data set 2) corresponding to the data processing of the second round, the corresponding bandwidth IO2 can be determined. For example, if the ratio of IO2 and IO1 is greater than a predetermined threshold, it can be determined that the size of the cache block needs to be further expanded.

[0253] For example, the size of the enlarged cache block can be obtained by multiplying the size of the current cache block by T. For example, if T is 2, the size of the cache block corresponding to the fourth round of data processing is 8MB.

[0254] 4. Fourth round of data processing

[0255] As shown in Figure 8 , in the fourth round of data processing, for example, a cache block 904 with a size of 8MB can be applied for from the memory.

[0256] Then, the data set corresponding to the cache block 904 needs to be determined. For example, the implementation of determining the data set can be understood as Figure 10 shown in Figure 10 . For example, there are currently 11 application data, which are data A to data K. Similar to the above description, in the fourth round of data processing, data selection will be performed in the remaining application data except data A, data E, data G, data K, data B, data D and data H to determine the data set. Among them, data A, data E, data G, data K, data B, data D and data H are data that have been selected in the previous rounds of data processing.

[0257] In this embodiment, for example, among the remaining application data except data A, data E, data G, data K, data B, data D and data H, the combination of application data with the smallest size difference from the size of the cache block 904 is selected to obtain the data set corresponding to the cache block 904. For example, in the fourth round of data preparation, the size of the cache block 904 is 8MB, so among the multiple remaining application data as shown in Figure 10 , the data combination with the smallest size difference from 8MB is selected, for example, data F and data I can be determined as a data set, and the total size of this data set is 8MB.

[0258] Referring to Figure 9 , the data set 4 corresponding to the cache block 904 contains data F and data I. Then the mapping relationship between the cache block 904 and the data set 4 can be established. Then the file IO for the data set 4 can be submitted to write the data set 4 into the cache block 904, that is, data F and data I are aggregated and written into the cache block 904. Compared with the implementation of submitting file IO for data F and data I separately, this implementation can improve the bandwidth utilization.

[0259] and referring to Figure 8It can be determined that in the embodiment, the data preparation and the file IO are asynchronously completed. That is, after the completion of the data preparation in the fourth round, the IO processing of the data set involved in the data preparation in the fourth round is submitted, and the next round of data preparation is performed at the same time, that is, the fifth round of data preparation performed in period 5 as shown in FIG. 5. Figure 8 As shown in FIG. 5, the period corresponding to the fourth round of data preparation is period 4, the period corresponding to the file IO after the fourth round of data preparation is period 9, and the period corresponding to the fifth round of data preparation is period 5. There is an overlap between period 9 and period 5, which means that the file IO and the data preparation are asynchronously performed. Referring to FIG. 5, it can be seen that the start time of period 9 is after the end time of period 4, that is, the file IO corresponding to the fourth round of data preparation cannot be performed until the fourth round of data preparation is completed. Figure 8 Figure 8 It can also be understood that the start time of period 9 is after the end time of period 4, that is, the file IO corresponding to the fourth round of data preparation cannot be performed until the fourth round of data preparation is completed.

[0260] After the completion of the fourth round of data processing, it can be determined whether there is remaining data, and if so, the fifth round of data processing is continued.

[0261] Referring to the above description of the embodiment, it can be understood that the technical solution of the present application can determine whether to expand the size of the cache block based on the bandwidth corresponding to the data processing of the previous two rounds (the i-2th round and the i-1th round) to obtain the size of the cache block in the next round. For example, based on the file IO processing corresponding to the data processing in the second round (that is, the file IO for data set 2), the corresponding bandwidth IO2 can be determined, and based on the file IO processing corresponding to the data processing in the third round (that is, the file IO for data set 3), the corresponding bandwidth IO3 can be determined. For example, if the ratio of IO3 to IO2 is less than a predetermined threshold, it can be determined that the size of the cache block does not need to be enlarged.

[0262] For example, the size of the cache block in the fourth round of data processing can be determined as the size of the cache block in the fifth round of data processing. The size of the cache block corresponding to the fifth round of data processing is equal to 8 MB.

[0263] 5. Fifth round of data processing

[0264] As shown in FIG. 5, in the fifth round of data processing, for example, a cache block 905 with a size of 8 MB can be applied from the memory. Figure 8 After that, it is also necessary to determine the data set corresponding to the cache block 905. For example, the implementation of determining the data set can be understood as follows:

[0265] Figure 10 As shown in FIG. 5, in the fifth round of data processing, for example, a cache block 905 with a size of 8 MB can be applied from the memory. Figure 10 ​​For example, there are currently 11 pieces of application data, which are data A to data K. Similar to the above description, in the data processing of the fourth round, data selection is performed on the remaining application data except data A, data E, data G, data K, data B, data D, data H, data F and data I to determine the data set. Data A, data E, data G, data K, data B, data D, data H, data F and data I are data that have been selected in the previous rounds of data processing.

[0266] In this embodiment, for example, the combination of application data with the minimum difference between the total size and the size of the cache block 905 can be selected from the remaining application data except data A, data E, data G, data K, data B, data D, data H, data F and data I to obtain the data set corresponding to the cache block 905. For example, in the data preparation of the fifth round, the size of the cache block 905 is 8 MB, so data C and data J can be selected from the remaining application data as shown in the figure to obtain a data set with a total size of 7.1 MB. Figure 10

[0267] Referring to Figure 9 The data set 5 corresponding to the cache block 905 contains data C and data J. Then, the mapping relationship between the cache block 905 and the data set 5 can be established. Then, the file IO for the data set 5 can be submitted to write the data set 5 into the cache block 905, that is, data C and data J are aggregated and written into the cache block 905. Compared with the implementation of submitting file IO for data C and data J separately, this implementation can improve the bandwidth utilization.

[0268] After the data processing of the fifth round is completed, it can be determined whether there is remaining data. If there is no remaining data, the current process can be ended, that is, the next round of data processing does not need to be performed.

[0269] Referring to Figure 8 It can be determined that in this embodiment, the data preparation and the execution of the file IO are completed asynchronously. That is, after the data preparation of the fifth round is completed, the IO processing for the data set involved in the data preparation of the fifth round is submitted, and there is no next round of data processing in the current example. As shown in Figure 8 Figure 8 The period corresponding to the data preparation of the fifth round is period 5, and the period corresponding to the file IO after the data preparation of the fifth round is period 10, wherein the start time of period 10 is after the end time of period 5, that is, the file IO corresponding to the data preparation of this round needs to wait until the completion of the data preparation of this round.

[0270] ​It also needs to be introduced that in the data preparation phase, when the data set corresponding to the cache block is determined, the scheme adopted by the technical solution of the application is to select the data combination whose data size sum is the minimum difference with the size of the cache block, so as to obtain the data set corresponding to the cache block. However, it can be understood that in the actual implementation process, there may be multiple data combinations whose data size sum is the minimum difference with the size of the cache block.

[0271] For example, for a cache block with a size of 8MB, among the multiple application data, it is assumed that there are four data with data amounts of 3MB, 5MB, 2MB and 6MB respectively. Then, the data size difference of the combination of 3MB and 5MB data with the size of the 8MB cache block is 0, and the data size difference of the combination of 2MB and 6MB data with the size of the 8MB cache block is 0. That is, the combination of 3MB and 5MB data can be determined as the data set corresponding to the cache block, and the combination of 2MB and 6MB data can also be determined as the data set corresponding to the cache block.

[0272] In the case where there are multiple data combinations whose data size sum is the minimum difference with the size of the cache block, for example, any one of the multiple data combinations introduced here can be selected as the data set corresponding to the cache block.

[0273] In summary, the technical solution of the application can expand the size of the cache block in the multi-round data processing process to realize the aggregation of multiple application data for submitting file IO. Compared with the implementation mode of separately submitting file IO for each application data, the technical solution of the application can effectively improve the bandwidth utilization. Moreover, in the process of expanding the size of the cache block, the size of the cache block is not continuously increased, but whether the size of the cache block needs to be continuously expanded is determined based on the actual bandwidth of the file IO corresponding to the data set. In the case where the bandwidth meets the preset condition, the cache block is not expanded, but the current cache block size is used. In this way, the situation of excessive aggregation of data can be avoided, so as to ensure the rationality of the time proportion of data preparation and file IO on the basis of aggregated IO.

[0274] In addition, in the technical solution of the application, when data is aggregated for file IO, the data is not first cached in a unit, and the file IO is performed when the amount of data cached in the unit exceeds a threshold. It can be understood that this implementation of waiting for the amount of data to reach a threshold before submitting file IO requires waiting for the cumulative amount of multiple data to reach a specific threshold, which will cause a time delay during the time period of waiting for the amount of data to reach the preset threshold.

[0275] For example, after data 1 is written into the cache unit, the amount of cached data does not reach the threshold; after data 2 is written into the cache unit, the amount of cached data also does not reach the threshold; after data 3 is written into the cache unit, the amount of cached data reaches the threshold. Subsequently, the three pieces of data are submitted together for file IO. It can be understood that in this implementation, the subsequent file IO is actually executed after waiting for the writing of the three pieces of data into the cache unit, and the process of waiting for the three pieces of data will cause a delay in data writing.

[0276] The technical solution of the present application is to select at least one piece of application data that matches the size of the cache block from multiple pieces of application data after determining the size of the cache block to obtain a data set, and then directly submit file IO for the data set. This implementation does not need to wait for the data amount to reach the threshold, but can directly determine the matching data set based on the size of the cache block, and then directly submit the file IO corresponding to the data set, so this implementation can improve the efficiency of data exchange from one storage space to another.

[0277] In addition, the technical solution of the present application also proposes asynchronous execution of file IO and data preparation, which enables the execution of file IO and data preparation to be parallel, for example, after submitting file IO for one data preparation, the next data preparation can be executed synchronously, which can further improve the efficiency of data exchange from one storage space to another.

[0278] For example, before the technical solution of the present application is adopted, the IO size is mainly concentrated between 500K-14MB, the IO bandwidth is about 1.2GB / s, and the bandwidth utilization rate is 20%-50%. After the technical solution of the present application is adopted, the IO size is mainly concentrated between 32-64MB, the IO bandwidth is about 3.5GB / s, and the bandwidth utilization rate is 80%-95%. As can be seen, the technical solution of the present application can significantly improve the bandwidth utilization rate.

[0279] It should also be understood that in addition to the scenario where the application program is switched from the background to the foreground, there is a problem of low bandwidth utilization rate when data is written from the UFS to the memory. In fact, in the scenario where the application program is switched from the foreground to the background, there is also a problem of low bandwidth utilization rate when data is written from the memory to the UFS. The above describes a solution involved in the process of writing data from the UFS to the memory to improve the bandwidth utilization rate. In fact, in the process of writing data from the memory to the UFS, the above-described technical solution can also be used to improve the bandwidth utilization rate. The specific implementation in this scenario is similar to the above description, and only the direction of data writing needs to be adaptively replaced. That is, the data of the application program is originally stored in the memory, and currently needs to be exchanged from the memory to the UFS.

[0280] In addition, it can be understood from the above description that the data preparation and file IO processing described in the present application are executed asynchronously. In one implementation, for example, the data preparation operation can be executed based on a first thread, and the file IO operation can be executed based on a second thread, so that the asynchronous execution of data preparation and file IO can be achieved to improve the utilization of CPU and bandwidth.

[0281] In addition, it can be understood from the above description that the data preparation and file IO processing described in the present application are executed asynchronously. In one implementation, for example, the data preparation operation can be executed based on a first thread, and the file IO operation can be executed based on a second thread, so that the asynchronous execution of data preparation and file IO can be achieved to improve the utilization of CPU and bandwidth.

[0282] In order to avoid this situation, after determining the size of the cache block in the i-th round of data preparation based on the size of the cache block in the i-1-th round of data preparation, it can be determined whether there is application data with a data size smaller than the currently determined cache block size among the plurality of application data remaining in the UFS. If there is, the currently determined cache block size is used to perform the i-th round of data preparation to apply for a cache block, because this can ensure that there is data that can be written into the cache block.

[0283] However, if there is no application data with a data size smaller than the currently determined cache block size, it means that the data remaining in the UFS cannot be written into the cache block of this size, so the size of the cache block in the i-th round of data preparation can be adjusted, for example, the currently determined cache block size can be expanded by T times to obtain the size of the cache block in the i-th round of data preparation, and then the i-th round of cache block application is performed based on this size. Alternatively, the application data with the smallest data size among the plurality of application data remaining can be determined, and the size of this application data is determined as the size of the cache block in the i-th round of data preparation, so that at least one data can be written into the cache block.

[0284] After the size of the cache block in the data preparation of the i th round is determined based on the size of the cache block in the data preparation of the i-1 th round, the specific strategy for adjusting the size of the cache block in the data preparation of the i th round can be determined according to actual needs, as long as there is at least one piece of application data that can be written into the cache block applied for in the i th round. It should also be understood that the size of the cache block is determined before the cache block is applied, so the implementation of adjusting the size of the cache block described above will be completed before the operation of applying the cache block in the data preparation of the i th round.

[0285] On the basis of the above introduction, the memory processing method provided in the present application will be further described below. The memory processing method provided in the present application can be applied to an electronic device, wherein the electronic device has a first storage space and a second storage space, the first storage space can be a volatile memory, and the first storage space can be, for example, a memory. The second storage space can be a non-volatile memory, and the second storage space can be, for example, a UFS, and the second storage space can store a plurality of pieces of application data of a first application.

[0286] The technical solution of the present application can include the following steps:

[0287] In the case where the first application is switched from the background to the foreground, the first operation is performed n times, and the first operation includes: writing a data set into a cache block applied from the first storage space, wherein the data set includes at least one piece of application data in the plurality of pieces of application data of the second storage space, the size of the data set is less than or equal to the size of the cache block, and n is an integer greater than or equal to 3.

[0288] The first operation, i.e., the file IO operation described above, the specific implementation details of performing the first operation n times by the electronic device can be understood with reference to the processing of performing multiple rounds of file IO described in the above embodiments, which will not be described here.

[0289] In addition, in the technical solution of the present application, the first size of the cache block in the i th first operation is greater than or equal to the second size of the cache block in the i-1 th first operation, and i is 2-n. That is, the size of the cache block involved in the n times of the first operation is either increasing or the same as the previous round. The determination of the size of the cache block in the multiple rounds of processing can be understood with reference to the description of the above embodiments, which will not be described here.

[0290] In summary, the technical solution of the present application can implement aggregating at least one piece of data into a data set to submit a file IO, thereby improving the bandwidth utilization.

[0291] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0292] The electronic device provided by the embodiment of the present application comprises a processor and a memory. The memory stores computer execution instructions. The processor executes the computer execution instructions stored in the memory, so that the electronic device executes the above method.

[0293] The embodiment of the present application provides a chip. The chip comprises a processor, and the processor is used to call a computer program in a memory to execute the technical solutions in the above embodiments. The implementation principle and technical effects are similar to those of the above related embodiments, which will not be described here.

[0294] The embodiment of the present application further provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by the processor to realize the above method. The method described in the above embodiments can be realized by software, hardware, firmware or any combination thereof in whole or in part. If realized in software, the functions can be stored as one or more instructions or codes on or transmitted on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can also include any medium that can transfer computer programs from one place to another. The storage medium can be any target medium accessible by a computer.

[0295] In a possible implementation, the computer readable medium can include a random access memory (RAM), a read only memory (ROM), a compact disc read only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is suitable for storing desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer readable media.

[0296] The embodiments of the present application provide a computer program product, which comprises a computer program, and when the computer program is executed, the computer program causes the computer to execute the above method.

[0297] It should be noted that the modules or components described in the above embodiments can be one or more integrated circuits configured to implement the above method, for example, one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of a processing element scheduling program code, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke program code, such as a controller. For another example, these modules can be integrated together to implement a system-on-a-chip (SOC).

[0298] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0299] The term "a plurality of" herein refers to two or more. The term "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents that the associated objects before and after it are in an "or" relationship; in the formula, the character " / " represents that the associated objects before and after it are in a "division" relationship. In addition, it should be understood that in the description of the present application, the terms "first", "second", etc. are only used for the purpose of distinguishing the description and should not be understood as indicating or implying relative importance or indicating or implying sequence.

[0300] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of differentiation and do not limit the scope of the embodiments of the present application.

[0301] It can be understood that in the embodiments of the present application, the size of the serial number of the above processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

Claims

1. A memory processing method, characterized by, The application is applied to an electronic device, the electronic device comprising a first storage space and a second storage space, and a plurality of application data of a first application being stored in the second storage space; The method comprises: In a case where the first application is switched from the background to the foreground, performing n times of first operations, the first operation comprising: writing a data set into a cache block applied from the first storage space, wherein the data set comprises at least one piece of application data in the plurality of application data of the second storage space, the size of the data set is less than or equal to the size of the cache block, and n is an integer greater than or equal to 3; wherein a first size of the cache block in the i-th first operation is greater than or equal to a second size of the cache block in the (i-1)-th first operation, and the value of i is 2-n.

2. The method of claim 1, wherein, In a case where the value of i is less than or equal to 3, the first size is greater than the second size.

3. The method according to claim 1 or 2, characterized in that, In a case where the value of i is greater than 3, the size of the cache block is determined based on bandwidth.

4. The method according to any one of claims 1 to 3, characterized in that, In a case where the value of i is greater than 3, the first size is determined based on a first bandwidth, a second bandwidth and the second size, the first bandwidth corresponding to the (i-2)-th first operation, and the second bandwidth corresponding to the (i-3)-th first operation.

5. The method of claim 4, wherein, In a case where the first bandwidth and the second bandwidth satisfy a first quantity relationship, the first size is equal to the second size. In a case where the first bandwidth and the second bandwidth do not satisfy the first quantity relationship, the first size is greater than the second size.

6. The method of claim 5, wherein, The first quantity relationship comprises that the ratio of the first bandwidth to the second bandwidth is less than a first preset threshold; and / or, the first quantity relationship comprises that the difference between the first bandwidth and the second bandwidth is less than a second preset threshold.

7. The method according to any one of claims 1 to 6, characterized in that, In a case where the value of i is 1, the size of the cache block in the i-th first operation is a preset size.

8. The method according to any one of claims 1 to 7, characterized in that, Before performing the i-th first operation, the method further comprises: performing a second operation, the second operation comprising: applying the cache block from the first storage space based on the first size.

9. The method of claim 8, wherein, The second operation further comprises: determining a data set corresponding to the cache block.

10. The method of claim 9, wherein, The determination of the data set corresponding to the cache block comprises: determining the data set corresponding to the cache block based on the first size.

11. The method of claim 10, wherein, The determination of the data set corresponding to the cache block based on the first size comprises: determining, among the remaining application data in the second storage space, a combination of at least one piece of application data with the smallest difference between the total size and the first size to obtain the data set corresponding to the cache block, the remaining application data being application data that has not been divided into a data set.

12. The method according to any one of claims 1 to 11, characterized in that, The period for performing the j-th first operation is a first period, and the period for performing the second operation before the (j+1)-th first operation is a second period, the first period and the second period overlap, and the value of j is 1-n-1.

13. The method of claim 12, wherein, The first operation is performed based on a first thread, and the second operation is performed based on a second thread.

14. The method according to any one of claims 1 to 13, characterized in that, In a case where the first size is greater than the second size, the first size and the second size satisfy a second quantity relationship.

15. The method of claim 14, wherein, The second quantity relationship is that the first size is T times the second size, or a difference between the first size and the second size is a first value, and the T is a value greater than or equal to 1.

16. The method according to any one of claims 1 to 15, characterized in that, The first storage space is a volatile memory, and the second storage space is a non-volatile memory.

17. An electronic device, comprising: The electronic device includes one or more processors and a memory. The memory is coupled to the one or more processors, and the memory is configured to store computer program codes including computer instructions, and the one or more processors are configured to invoke the computer instructions to cause the electronic device to perform the method in any one of claims 1-16.

18. A chip system, characterized by The chip system is applied to an electronic device, and the chip system includes one or more processors configured to invoke computer instructions to cause the electronic device to perform the method in any one of claims 1-16.

19. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions configured to cause an electronic device to perform the method in any one of claims 1-16 when the computer instructions run on the electronic device.

20. A computer program product, characterised in that, The computer program product includes computer program codes configured to cause an electronic device to perform the method in any one of claims 1-16 when the computer program codes run on the electronic device.

Images