A data exchange method and apparatus

Abstract

The application relates to a data exchange method and device, relates to the technical field of storage, and solves the problem that the memory exchange mechanism of an operating system kernel cannot meet the diversified needs of different applications. The data exchange method is applied to a host, the host is installed with an operating system and a target application, the operating system comprises a user state space, the user state space is provided with a user state interface of the target application, and the user state interface corresponds to a storage device. The specific scheme comprises the following steps: the host acquires an exchange request of the target application, and according to the exchange request, the user state interface is called to execute an operation on the storage device. The exchange request is a swap-in request or a swap-out request, the swap-out request is used for requesting to write data of the target application from the memory of the host into the storage device, and the swap-in request is used for requesting to write data in the storage device into the memory.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data exchange method and apparatus. Background Technology

[0002] A host computer cannot function properly without an operating system, such as Linux. The operating system kernel supports memory swapping, which involves swapping data from the host's memory to storage devices or vice versa. Besides the operating system, a host also hosts various applications. With the diversification of applications, their memory swapping needs are becoming increasingly diverse. For example, different applications may require different storage devices during memory swapping. However, the storage devices used by the operating system kernel for memory swapping are typically general-purpose, fixed devices, which cannot meet the diverse storage needs of various applications. Summary of the Invention

[0003] This application provides a data exchange method and apparatus, which solves the problem that the memory exchange mechanism of the operating system kernel cannot meet the diverse needs of different applications.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, this application provides a data exchange method applied to a host computer. The host computer has an operating system and a target application installed. The operating system includes a user-mode space, which is configured with a user-mode interface for the target application. Each user-mode interface corresponds to a storage device. Specifically, the method includes: the host computer obtaining an exchange request from the target application and, based on the exchange request, calling the user-mode interface to perform operations on the storage device. The exchange request can be either a swap-in request or a swap-out request. A swap-out request requests the writing of data from the host computer's memory to the storage device. A swap-in request requests the writing of data from the storage device to memory.

[0006] In this way, by setting up a user-mode interface for the application in user space and calling that interface to swap data out or in, user-mode memory swapping is achieved. Therefore, different user-mode interfaces can be set for different applications, each corresponding to a different storage device. This provides users with a customized user-mode memory swapping framework, meeting the diverse storage device needs of different applications and enabling users to design their own desired memory swapping mechanisms.

[0007] In one possible implementation of this application, when the swap request is a swap-out request, the method of "invoking a user-mode interface to perform operations on the storage device according to the swap request" may include: In response to the swap-out request, the host determines the first data of the target application in memory, unmaps the first virtual address and the first physical address in the pre-stored page table, allocates a second virtual address for the first data, and establishes a mapping between the second virtual address and the first physical address. Then, the host invokes the user-mode interface to write the first data indicated by the first physical address corresponding to the second virtual address from memory to the storage device, and establishes a mapping between the first virtual address and the first swap entry in the page table. The first swap entry is used to indicate the location of the first data in the storage device. The first virtual address is the virtual address of the first data in virtual memory, and the first physical address is the address of the first data in memory.

[0008] In this way, the swapping process of the first data based on the second virtual address is realized, ensuring the correctness and reliability of the swapping process in user space.

[0009] In another possible implementation of this application, the data exchange method provided by this application may further include: the host unmapping the mapping relationship between the second virtual address and the first physical address, and deleting the first data in memory.

[0010] By freeing up storage space for the first data in memory, memory usage can be saved and memory utilization rate can be improved.

[0011] In another possible implementation of this application, the data exchange method provided by this application may further include: the host sets the status information of the first virtual address to first information, the first information being used to indicate that the exchange type of the first data corresponding to the first virtual address is a user-mode exchange type.

[0012] In another possible implementation of this application, when the exchange request is a swap-in request, the method of "obtaining the exchange request of the target application" may include: the host obtaining the virtual address of the second data to be accessed, and generating a swap-in request when the status information of the virtual address of the second data is the first information, the second data is the data of the target application, and the second data is not included in memory. The first information indicates that the exchange type of the second data is a user-mode exchange type.

[0013] In another possible implementation of this application, the method of "invoking a user-mode interface to perform operations on the storage device according to a swap request" may include: the host responding to a swap request, obtaining the second swap entry corresponding to the virtual address of the second data according to a pre-stored page table, invoking a user-mode interface, writing the second data indicated by the second swap entry from the storage device into memory, and then establishing a mapping relationship between the virtual address and the physical address of the second data in the page table. The physical address of the second data is the address of the second data in memory.

[0014] In this way, the process of swapping in the second data based on page fault reports is realized, ensuring the correctness and reliability of the swapping process in user space.

[0015] In another possible implementation of this application, before the host obtains the exchange request from the target application, the data exchange method provided in this application may further include: after the target application starts, the host establishes a swap-out thread and a swap-in thread. The swap-out thread is used to perform the swap-out operation of the first data, and the swap-in thread is used to perform the swap-in operation of the second data.

[0016] In another possible implementation of this application, the method of "establishing swap-out threads and swap-in threads" described above may include: the host establishing swap-out threads and swap-in threads if the target application is included in a pre-stored whitelist. The whitelist includes at least one application that supports memory swapping.

[0017] By pre-setting a whitelist, memory swapping can be supported only for certain specific applications, thus saving memory overhead and improving host performance.

[0018] Secondly, this application provides a data exchange apparatus. The data exchange apparatus includes modules for performing a data exchange method according to the first aspect or any possible implementation thereof.

[0019] Thirdly, this application provides a host computer including a memory and a processor. The memory and the processor are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the host computer performs a data exchange method as described in the first aspect and any possible implementation thereof.

[0020] Fourthly, this application provides a chip system applied to a host computer. The chip system includes one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via lines; the interface circuits are used to receive signals from the host computer's memory and send signals to the processors, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the host computer performs a data exchange method as described in the first aspect and any possible implementation thereof.

[0021] Fifthly, this application provides a computer-readable storage medium including computer instructions that, when executed on a host computer, cause the host computer to perform a data exchange method as described in the first aspect and any possible implementation thereof.

[0022] In a sixth aspect, this application provides a computer program product comprising computer instructions that, when executed on a host computer, cause the host computer to perform a data exchange method as described in the first aspect and any possible implementation thereof.

[0023] For a detailed description of aspects two through six and their various implementations in this application, please refer to the detailed description in aspect one and its various implementations; and for a detailed description of the beneficial effects of aspects two through six and their various implementations, please refer to the beneficial effect analysis in aspect one and its various implementations, which will not be repeated here.

[0024] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0025] Figure 1 A schematic diagram of a data exchange system provided in an embodiment of this application;

[0026] Figure 2 One of the schematic diagrams of a host provided in an embodiment of this application;

[0027] Figure 3 This is a second schematic diagram of the structure of a host provided in an embodiment of this application;

[0028] Figure 4a This is a schematic diagram illustrating the data swapping out in user space provided in an embodiment of this application.

[0029] Figure 4b This is a schematic diagram illustrating the scenario of exchanging the first data provided in an embodiment of this application.

[0030] Figure 5 A schematic diagram illustrating the data swapping in user space as provided in an embodiment of this application;

[0031] Figure 6 A flowchart illustrating the data exchange method provided in an embodiment of this application;

[0032] Figure 7 This is one of the structural schematic diagrams of a data exchange device provided in the embodiments of this application;

[0033] Figure 8 This is a second schematic diagram of the structure of the data exchange device provided in the embodiments of this application. Detailed Implementation

[0034] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0036] With the development of technology, the amount of data generated in our lives and work is increasing. Cold data refers to data that is accessed infrequently but still needs to be retained for a long time. To reduce the storage cost of cold data, it can be stored on storage devices through memory swapping mechanisms.

[0037] Currently, the operating system kernel performs memory swapping as follows: when the host's memory is insufficient, the kernel writes infrequently accessed data (cold data) from memory to a storage device and then deletes that cold data from memory. When data needs to be accessed from memory and written to a storage device, the kernel can write that data from the storage device to memory. However, because the storage devices used by the operating system kernel for memory swapping are typically general-purpose fixed devices, this cannot meet the diverse storage needs of various applications.

[0038] To address the issue that the memory swapping mechanism of the operating system kernel cannot meet the diverse needs of different applications, this application provides a data swapping method applied to a host. The host has an operating system and a target application installed. The operating system includes a user-mode space, which is configured with a user-mode interface for the target application. Each user-mode interface corresponds to a storage device. After receiving a swapping request from the target application, the host can invoke the user-mode interface to perform operations on the storage device based on the swapping request. When the swapping request is an outgoing request, it requests to write data of the target application from the host's memory to the storage device; when the swapping request is an incoming request, it requests to write data from the storage device to memory. Thus, by setting up a user-mode interface for the application in the user-mode space and invoking this interface to swap out or in the application's data, user-mode memory swapping is achieved. Therefore, different user-mode interfaces can be set for different applications, and these different user-mode interfaces correspond to different storage devices, satisfying the diverse storage device needs of different applications.

[0039] To facilitate understanding by those skilled in the art, the terminology used in the embodiments of this application will be briefly introduced below.

[0040] 1. Operating System

[0041] An operating system is an essential system software for a host computer; it is the soul of the entire host. Common operating systems include Windows and Linux.

[0042] 2. User space and kernel space

[0043] Logically, an operating system can include user space and kernel space. User space is the activity space for applications, used to store application code and data. Kernel space is the activity space for the operating system, used to store the operating system or its kernel's code and data. Typically, applications or processes in user space have limited access to the host's hardware resources, only able to access a subset. The operating system or kernel in kernel space has access to all of the host's hardware resources. Furthermore, applications in user space cannot directly access the host's hardware resources; they can access them through operating system calls, library functions, and scripts.

[0044] The data exchange method provided in this application is applicable to data exchange systems. Figure 1 One structure of this data exchange system is shown. For example... Figure 1As shown, the data exchange system may include a host 11 and a storage device 12. The host 11 and the storage device 12 establish a connection using wired or wireless communication.

[0045] Host 11 is used to obtain the exchange request of the target application installed on host 11, and according to the exchange request, call the user-mode interface of the target application set in the user-mode space to perform operations on the storage device 12 corresponding to the user-mode interface. The operating system installed on host 11 includes the user-mode space.

[0046] It is understood that when multiple target applications are installed on host 11, a user-mode interface for each target application can be set in the user-mode space, meaning one user-mode interface corresponds to one target application. The user-mode interfaces for different applications can be the same or different. Furthermore, one user-mode interface corresponds to one storage device. The storage devices corresponding to the same user-mode interface can be the same or different, and the storage devices corresponding to different user-mode interfaces can be the same or different. In this embodiment, the user-mode interface corresponding to each application, and the storage device corresponding to each user-mode interface, can be pre-configured in the host, and are not limited here.

[0047] In some embodiments, the host 11 can be a terminal device, a server, a cloud computing platform, etc. The terminal device can be a mobile phone, tablet computer, laptop computer, PDA, computer, etc. Figure 1 The example shown uses a host as the server.

[0048] In some embodiments, the storage device 12 may be the memory of a host other than the host 11, or it may be a storage medium. The type of storage medium may be non-volatile flash memory (NAND flash), storage class memory (SCM), etc.

[0049] It should be noted that, in this embodiment, the operating system installed on host 11 may include not only user space but also kernel space. This kernel space can be configured with a kernel interface, which corresponds to a storage device used by the operating system kernel for memory swapping. This storage device is typically a NAND flash, SCM, or similar device.

[0050] in addition, Figure 1 The data exchange system illustrated is for illustrative purposes only and is not intended to limit the technical solutions of the embodiments of this application. Those skilled in the art should understand that in specific implementations, the data exchange system may include other nodes, and the number of nodes can be determined according to specific needs without limitation.

[0051] Figure 2 One structure of the aforementioned host is shown. For example... Figure 2 As shown, the host may include a processor 201 and a memory 202, used to implement the data exchange method provided in the embodiments of this application.

[0052] The processor 201 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0053] For example, processor 201 is used to process requests from outside the host, such as other hosts, and also to process requests generated within the host. In this embodiment, processor 201 can be used to process exchange requests obtained by the host, which are swap-out requests or swap-in requests.

[0054] Memory 202 refers to the internal memory that directly exchanges data with the processor 201. It can read and write data at any time at high speed, serving as temporary data storage for the operating system or other running programs. Memory 202 includes at least two types of memory, such as random access memory (RAM) or read-only memory (ROM). For example, RAM can be dynamic random access memory (DRAM) or SCM. DRAM is a semiconductor memory, and like most random access memory (RAM), it is a type of volatile memory. SCM is a composite storage technology that combines the characteristics of traditional storage devices and RAM. SCM can provide faster read and write speeds than hard drives, but its access speed is slower than DRAM, and its cost is lower than DRAM. It should be noted that DRAM and SCM are only illustrative examples in this embodiment; memory 202 may also include other random access memories, such as static random access memory (SRAM). For read-only memory, examples include programmable read-only memory (PROM) and erasable programmable read-only memory (EPROM), but these are not limited to. Additionally, memory 202 can also be a dual in-line memory module (DIMM), i.e., a module composed of DRAM, or a solid-state disk (SSD). Memory 202 can exist independently and be coupled to processor 201. Memory 202 can also be integrated with processor 201.

[0055] In practical applications, a host can be configured with multiple memory modules, which can be of the same type or different types. This application does not limit the quantity or type of memory in the host. Figure 2 The example shown uses only one memory module 202. Furthermore, memory module 202 can be configured to have a power-saving function. The power-saving function means that when the system experiences a power outage and is then powered on again, the data stored in memory module 202 will not be lost. Memory with a power-saving function is called non-volatile memory.

[0056] Memory 202 stores the program code and data of applications installed on the host, as well as the code and data of the operating system installed on the host. Applications are programs that run on top of the operating system.

[0057] It should be noted that, Figure 2 The structure shown does not constitute a limitation on this host, except... Figure 2 In addition to the components shown, the host computer may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. For example, the host computer may also include input / output devices such as a sound card, optical drive, hard drive, monitor, keyboard, mouse, speakers, graphics tablet, microphone, and camera.

[0058] Combination Figure 1 and Figure 2 , Figure 3 This illustrates one structure of the host computer. For example... Figure 3 As shown, the host computer has an operating system and at least one target application installed. The operating system may include user-mode space and kernel-mode space. The user-mode space includes the program code and data of each application within the at least one application, while the kernel-mode space includes the program code and data of the operating system kernel. Furthermore, the user-mode space is configured with a user-mode interface for each application within the at least one application. In a specific implementation, the program code of each application in the user-mode space may include a driver or protocol stack corresponding to the application's user-mode interface. By running the driver or protocol stack, the application's user-mode interface can be invoked.

[0059] In the host machine, the user-mode space may include a preloading module, a swapping-out module, and a swapping-in module. The kernel-mode space may include a page table management module and a page fault reporting module. These modules work together to complete the user-mode memory swapping mechanism. This application uses a target application as an example to describe the user-mode memory swapping method in detail. The memory swapping methods for other target applications are similar and will not be described in detail here.

[0060] The page table management module manages the host's page tables. A page table is a data storage structure used to store the mapping between virtual addresses (VA) and physical addresses (PA) of data. The virtual address is the logical address of the data in virtual memory, and the physical address is the actual address of the data in memory. In one implementation, the data stored in the page table can be implemented using a hash table.

[0061] The preloading module is used to establish swap-out and swap-in threads and initialize relevant resources after the target application starts. The swap-out thread performs the swap-out operation for the first data, and the swap-in thread performs the swap-in operation for the second data. Relevant resources refer to the various resources required for swapping out the first data or swapping in the second data.

[0062] For example, the relevant resources may include page tables. Optionally, the relevant resources may also include swap cache. Swap cache is a portion of storage space allocated from the host's memory, used to temporarily store the first data when it is swapped from memory to a storage device. Once the swap cache is full, the data in the swap cache is then stored on the storage device.

[0063] Optionally, a preload module is used to establish swap-out and swap-in threads and initialize related resources after the target application starts by loading the program code of the target application in user space.

[0064] Optionally, since not every application supports user-space memory swapping, the preloading module can determine whether the target application is included in a pre-stored whitelist before creating the swap-out and swap-in threads. This whitelist includes at least one application that supports memory swapping. If the target application is found to be in the whitelist, the preloading module creates the swap-out and swap-in threads; otherwise, it performs no operation. Applications included in the whitelist can be memory-intensive, such as games and video applications. Applications with low memory usage, such as calendars and calculators, are typically not included in the whitelist because their data volume is small and memory swapping is unnecessary. Thus, by pre-setting a whitelist, only supporting memory swapping for specific applications can save memory overhead and improve host performance.

[0065] After the preloading module establishes the swap-out thread and the swap-in thread, the swap-out module can wait to receive swap-out requests, and the swap-in module can wait to receive swap-in requests. The following section describes the situation where the swap-out module receives a swap-out request.

[0066] Figure 4a The process of swapping out the first data is shown, as follows: Figure 4a As shown, the swap-out module is used to obtain the swap-out request from the target application and call the swap-out thread to swap out the first set of data. This swap-out request is used to request that the data of the target application be written from the host's memory to the storage device corresponding to the user-space interface of the target application.

[0067] It is understood that the swap-out module can generate a swap-out request for the target application in response to a user's swap-out operation. Alternatively, the swap-out module can also generate a swap-out request for the target application when it determines that the memory for running the target application is insufficient. This embodiment of the application does not limit the triggering method of the swap-out request.

[0068] For example, suppose a host has 13GB of memory and is currently running two applications, consuming 10GB of memory. The user then launches another application. If this second application requires 5GB of memory, the host will generate a swap-out request for that application when it determines that there is insufficient memory available.

[0069] The swap-out module is also used to respond to a swap-out request from the target application by determining, in memory, the first data of the target application to be swapped out. This first data can be cold data of the target application, that is, data that is accessed infrequently but needs to be stored for a long time. For example, when the target application is a game application, the first data can be the historical data of that game application.

[0070] The swap-out module, after identifying the first data of the target application, also determines whether that first data is allowed to be swapped out. If it is determined that the first data is not allowed to be swapped out, the swap-out module terminates the swap-out process. If it is determined that the first data is allowed to be swapped out, the swap-out module can send a message to the page table management module by calling a customized interface. This message is used by the swap-out module to notify the page table management module that the first data is data to be swapped out. The customized interface refers to the interface for communication between user space and kernel space.

[0071] After receiving a message from the swap-out module, the page table management module can unmap the first virtual address and first physical address of the first data in the pre-stored page table, allocate a second virtual address for the first data, and establish a mapping between the second virtual address and the first physical address. Finally, the page table management module is also used to return the second virtual address of the first data to the swap-out module through a customized interface. Here, the first virtual address and the second virtual address are two different virtual addresses of the first data in virtual memory, while the first physical address is the actual address of the first data in memory.

[0072] Optionally, in this embodiment, the page table management module is further configured to, after receiving a message from the swap-out module in user space, set the status information of the first virtual address to first information, which indicates that the swapping type of the first data corresponding to the first virtual address is user-mode swapping. Alternatively, the page table management module is further configured to, after receiving a message from the swap-out module in kernel space, set the status information of the first virtual address to second information, which indicates that the swapping type of the first data corresponding to the first virtual address is kernel-mode swapping.

[0073] The swap-out module also receives the second virtual address of the first data sent by the page table management module. It then calls the user-space interface of the target application through the running driver or protocol stack. Based on the mapping relationship between the second virtual address and the first physical address in the page table, it writes the first data indicated by the first physical address corresponding to the second virtual address from memory to the storage device corresponding to the user-space interface. The data written to the storage device is the third data. After successfully writing the third data, the storage device can return the first swap entry to the swap-out module through the user-space interface. The first swap entry indicates the location of the third data in the storage device.

[0074] Optionally, in the embodiments of this application, such as Figure 4b As shown, the swap-out module can call the user-mode interface of the target application to first write the first data from memory to the swap memory. The data written to the swap memory is the swap data. After the swap memory is full, the swap data can be written from the swap memory to the storage device corresponding to the user-mode interface. At this time, the data written to the storage device is the third data.

[0075] It should be noted that in this embodiment, the first data, the third data, and the exchanged data are the same data stored in memory, a storage device, and a swap space, respectively. Only the storage location of the data changes; the content of the data is the same. Furthermore, after the first data is written from memory to the storage device, when the first data needs to be accessed in memory, the third data can be written from the storage device to memory. In this case, the virtual address of the data written to memory is the same as the virtual address of the first data.

[0076] The switching module is also used to send the first switching entry to the page table management module through a customized interface after receiving the first switching entry.

[0077] The page table management module is also used to establish a mapping relationship between the first virtual address and the first switching entry in the page table after receiving the first switching entry. The page table management module is also used to unmap the second virtual address and the first physical address in the page table and delete the first data from memory. This completes the swapping out of the first data. By releasing the storage space of the first data in memory, memory usage can be saved and memory utilization improved.

[0078] Optionally, in this embodiment, the page table management module is further configured to, after establishing the mapping relationship between the first virtual address and the first switching entry, set the status information of the first switching entry to third information, wherein the third information is used to indicate that the third data indicated by the first switching entry is data swapped out from memory to the storage device.

[0079] Thus, through Figure 4a The process of swapping out the first data based on the second virtual address was implemented, ensuring the correctness and reliability of the swapping out process in user space.

[0080] After explaining how the swap-out module receives a swap-out request, we will now explain how the swap-in module receives a swap-in request.

[0081] Figure 5 The process of swapping in the second data is shown. For example... Figure 5 As shown, the swap-in module is used to obtain the swap-in request from the target application and call the swap-in thread to perform the swap-in of the second data. This swap-in request is used to request that data in the storage device corresponding to the user-space interface of the target application be written into memory.

[0082] Understandably, when a user needs to view the second data of a target application, they can perform the viewing operation on the host. The host's processor responds to the user's viewing operation by generating a data access request and sending it to the page table management module. This data access request can include data access information, such as the virtual address of the second data. After receiving the data access request, the page table management module can query the virtual address of the second data in the page table. If the data location information corresponding to the virtual address of the second data in the page table is a physical address, it indicates that the second data is stored in memory, and the second data can be retrieved from memory based on this physical address. If the status information of the data location information corresponding to the virtual address of the second data in the page table is a third type of information, it indicates that the data location information stored in the page table is a second swap entry point, and the second data is not stored in memory. The page table management module can send an indication message to the page fault reporting module to trigger a page fault reporting. After receiving the indication message from the page table management module, the page fault reporting module can determine whether the swapping operation is a kernel-mode swapping or a user-mode swapping. Specifically, the page fault reporting module can determine whether the status information of the virtual address of the second data in the page table is the first or second information. If the status information of the virtual address of the second data is the first information, it indicates that the swapping type of the second data is user-mode swapping. In this case, the page fault reporting module can generate a swap-in request for the target application if it is determined that the second data is data of the target application. Then, it sends the swap-in request to the swap-in module in user-mode space through the user fault handling function (userfaultfd) or the signal handling function (signal). This swap-in request can include the virtual address of the second data. In this way, the swap-in module in user-mode space can obtain the swap-in request of the target application. If the status information of the virtual address of the second data is the second information, it indicates that the swapping type of the second data is kernel-mode swapping. In this case, the page fault reporting module can send a swap-in request to the swap-in module in kernel-mode space so that the swap-in module can swap in the second data.

[0083] It should be noted that, in this embodiment, the second data refers to data swapped out from memory to the storage device, and the userfaultfd or signal can be registered during the swapping out process of the second data. Specifically, it can be registered after the swapping out module determines that the second data is allowed to be swapped out. The swapping out process of the second data is the same as the swapping out process of the first data in the above embodiment, and will not be described again here.

[0084] The swap-in module is also used to respond to the swap-in request of the target application, call the customized interface, obtain the second swap entry corresponding to the virtual address of the second data in the page table through the page table management module, and call the user-mode interface of the target application to write the second data indicated by the second swap entry from the storage device corresponding to the user-mode interface into memory.

[0085] In the specific implementation, before writing the second data from the storage device to memory, the swap-in module can first allocate a third virtual address and a third physical address. After allocating the addresses, the swap-in module can copy the second data from the storage device to the location corresponding to the third physical address and establish a mapping relationship between the third virtual address and the third physical address. At this time, the data stored in the third physical address is the fourth data. Then, the swap-in module can instruct the page table management module to allocate a new physical address for the second data, copy the fourth data indicated by the third physical address corresponding to the third virtual address to the location corresponding to the new physical address, establish a mapping relationship between the virtual address and the physical address of the second data in the page table, release the mapping relationship between the third virtual address and the third physical address, and delete the fourth data stored at the third physical address. The data stored in the physical address of the second data is now the fifth data. This completes the swap-in of the second data, allowing data to be read and written in memory.

[0086] Optionally, in this embodiment, the swap-in module can call the user-space interface of the target application to write the second data from the storage device corresponding to the user-space interface into a buffer. The data written into the buffer is called buffered data. After allocating a third virtual address and a third physical address for the second data, the buffered data is written from the buffer to the location corresponding to the third physical address. The data written to the third physical address is called the fourth data. Finally, after allocating a physical address for the second data, the fourth data is copied from the third physical address to the location corresponding to the physical address of the second data. The data written to the physical address of the second data is called the fifth data.

[0087] It's understandable that the aforementioned buffer is also a portion of memory allocated for temporarily storing data when the second data from the storage device is swapped into memory. After a physical address is allocated for the second data in memory, the data from the buffer is written to that allocated physical address, thus avoiding slow host data exchange speeds due to slow read / write speeds of the storage device.

[0088] It should be noted that in this embodiment, the second data, fourth data, fifth data, and buffer data are the same data stored in the storage device, the third physical address in memory, the physical address of the second data in memory, and the buffer, respectively. Only the storage location of the data changes; the content of the data is the same. Furthermore, if the second data is data written from memory to the storage device, then the virtual address of the initial data and the virtual address of the fifth data are the same virtual address.

[0089] Thus, through Figure 5 The process of swapping in the second data based on page fault reports was implemented, ensuring the correctness and reliability of the swapping process in user space.

[0090] The following is combined Figures 1-5 The data exchange method provided in the embodiments of this application will be described in detail.

[0091] It is understood that the same step or step with the same function or technical feature in the embodiments of this application can be referenced and learned from each other in different embodiments.

[0092] It is understood that in the embodiments of this application, the host may execute some or all of the steps in the embodiments of this application. These steps are merely examples, and the embodiments of this application may also execute other steps or variations thereof. Furthermore, the steps may be executed in different orders as presented in the embodiments of this application, and it is not necessary to execute all the steps in the embodiments of this application.

[0093] In the embodiments of this application, the specific structure of the execution subject of the data exchange method is not particularly limited, as long as it can implement the method provided in the embodiments of this application. For example, the execution subject of the data exchange method provided in the embodiments of this application can be a host, or a component applied in the host, such as a chip; the embodiments of this application do not limit this. The following embodiments are described using a host as an example to illustrate the execution subject of the data exchange method.

[0094] like Figure 6 As shown, the data exchange method provided in this application embodiment may include the following steps 601-613. Step 601 is a preparatory operation performed by the preloading module in user space before memory exchange. Steps 602-608 are the process of swapping first data from memory to a storage device. Steps 609-613 are the process of swapping second data from the storage device into memory.

[0095] 601. After the target application starts, if the target application is included in the pre-stored whitelist, the host establishes a swap-out thread and a swap-in thread.

[0096] The whitelist includes at least one application that supports memory swapping. The swap-out thread performs the swap-out operation for the first piece of data, and the swap-in thread performs the swap-in operation for the second piece of data.

[0097] It should be noted that the specific description of step 601 can be found in the description of the preloading module establishing the swap-out thread and the swap-in thread in another embodiment, and will not be repeated here.

[0098] 602. The host obtains the swap-out request of the target application.

[0099] 603. In response to the swap-out request, the host determines the first data of the target application in memory.

[0100] 604. The host removes the mapping relationship between the first virtual address and the first physical address in the pre-stored page table and sets the status information of the first virtual address to the first information.

[0101] Wherein, the first virtual address is the virtual address of the first data in virtual memory, and the first physical address is the address of the first data in memory.

[0102] 605. The host allocates a second virtual address to the first data and establishes a mapping relationship between the second virtual address and the first physical address.

[0103] 606. The host calls the user-mode interface to write the first data indicated by the first physical address corresponding to the second virtual address from memory to the storage device.

[0104] 607. The host establishes a mapping relationship between the first virtual address and the first switching entry in the page table.

[0105] The first exchange entry is used to indicate the location of the first data in the storage device.

[0106] 608. The host removes the mapping relationship between the second virtual address and the first physical address, and deletes the first data in memory.

[0107] It should be noted that the specific descriptions of steps 602-608 can be found in the description of the swapping module and page table management module swapping the first data from memory to the storage device in another embodiment, and will not be repeated here.

[0108] 609. The host obtains the virtual address of the second data to be accessed.

[0109] 610. If the host's status information for the virtual address of the second data is the first information, the second data is the data of the target application, and the second data is not included in memory, the host generates a swap-in request for the target application.

[0110] 611. In response to the swap-in request, the host obtains the second switching entry corresponding to the virtual address of the second data according to the pre-stored page table.

[0111] 612. The host calls the user-mode interface to write the second data indicated by the second switching entry from the storage device into memory.

[0112] 613. The host establishes a mapping relationship between the virtual address of the second data and the physical address of the second data in the page table.

[0113] The physical address of the second data is the address of the second data in memory.

[0114] It should be noted that the specific descriptions of steps 609-613 can be found in the descriptions of the swapping module, page table management module, and page fault reporting module swapping the second data from the storage device into memory in another embodiment, and will not be repeated here.

[0115] This application provides a data exchange method applied to a host. The host has an operating system and a target application installed. The operating system includes a user-mode space, which is configured with a user-mode interface for the target application. Each user-mode interface corresponds to a storage device. After receiving an exchange request from the target application, the host can invoke the user-mode interface to perform operations on the storage device according to the exchange request. When the exchange request is an outgoing request, it requests to write data of the target application from the host's memory to the storage device; when the exchange request is an incoming request, it requests to write data from the storage device to memory. Thus, by setting up a user-mode interface for the application in the user-mode space and invoking this interface to perform data swapping out or incoming, user-mode memory exchange is achieved. Therefore, different user-mode interfaces can be set for different applications, and these different user-mode interfaces correspond to different storage devices, providing users with a customized user-mode memory exchange framework. This meets the diverse storage device needs of different applications, allowing users to design their desired memory exchange mechanism.

[0116] Furthermore, since user-mode read / write processes are faster than kernel-mode read / write processes, user-mode memory swapping performance is higher than kernel-mode memory swapping performance. To achieve even higher performance, current storage device drivers or protocol stacks have been moved to user mode, making it impossible for kernel-mode memory swapping mechanisms to use storage devices moved to user mode. This application implements a user-mode memory swapping mechanism that can use storage devices moved to user mode. Tests of the user-mode memory swapping mechanism in a relational database management system (MySQL) and a transaction processing performance council (TPCC) scenario show that the swap-out or swap-in latency is in the microsecond range, achieving user-mode memory swapping without the user's awareness, thus improving the user experience.

[0117] For example, suppose in a MySQL and TPCC scenario, the host swaps out 28% of the data in memory. Table 2 shows the relevant performance parameters of the user-space memory swapping mechanism. As shown in Table 1, when the host swaps out data, the performance loss is only 2%, while the performance loss of the kernel-space memory swapping mechanism is greater than 2%.

[0118] Table 1

[0119]

[0120] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] like Figure 7 The diagram shown is a structural schematic of a data exchange device 70 provided in an embodiment of this application. The data exchange device 70 can be a host, a CPU within a host, a control module within a host, or a client within a host. The data exchange device 70 is used to execute... Figure 6 The data exchange method is shown. The data exchange apparatus 70 may include an acquisition unit 71 and a processing unit 72.

[0122] The acquisition unit 71 is used to acquire the swap request from the target application. The swap request can be an in request or an out request. An out request is used to request that data of the target application be written from the host's memory to the storage device, while an in request is used to request that data in the storage device be written to memory. For example, combined with... Figure 6 The acquisition unit 71 can be used to execute steps 602 and 610. The processing unit 72 is used to call the user-mode interface to perform operations on the storage device according to the exchange request acquired by the acquisition unit 71.

[0123] Optionally, the swap request is a swap-out request. Processing unit 72, specifically, responds to the swap-out request by determining the first data of the target application in memory, releasing the mapping relationship between the first virtual address and the first physical address in the pre-stored page table, allocating a second virtual address for the first data, and establishing a mapping relationship between the second virtual address and the first physical address. Then, it calls the user-mode interface to write the first data indicated by the first physical address corresponding to the second virtual address from memory to the storage device, and establishes a mapping relationship between the first virtual address and the first swap entry in the page table. The first swap entry is used to indicate the location of the first data in the storage device. The first virtual address is the virtual address of the first data in virtual memory, and the first physical address is the address of the first data in memory.

[0124] Optional, such as Figure 8 As shown, the data exchange device 70 further includes a release unit 73 and a deletion unit 74. The release unit 73 is used to release the mapping relationship between the second virtual address and the first physical address. For example, combined with... Figure 6 The release unit 73 can be used to perform step 608 to release the mapping relationship between the second virtual address and the first physical address. The deletion unit 74 is used to delete the first data in memory. For example, combined with Figure 6 The deletion unit 74 can be used to perform the deletion of the first data in memory in step 608.

[0125] Optional, such as Figure 8 As shown, the data exchange device 70 further includes a setting unit 75. The setting unit 75 is used to set the status information of the first virtual address to first information, which indicates that the exchange type of the first data corresponding to the first virtual address is a user-mode exchange type.

[0126] Optionally, when the exchange request is a swap-in request, the acquisition unit 71 is specifically used to: acquire the virtual address of the second data to be accessed; when the status information of the virtual address of the second data is the first information, the second data is the data of the target application, and the second data is not included in the memory, generate a swap-in request, wherein the first information is used to indicate that the exchange type of the second data is a user-mode exchange type.

[0127] Optionally, the processing unit 72 is specifically used for: responding to a swap-in request, obtaining the second swap entry corresponding to the virtual address of the second data according to the pre-stored page table; calling the user-mode interface to write the second data indicated by the second swap entry from the storage device into memory; and establishing a mapping relationship between the virtual address of the second data and the physical address of the second data in the page table, wherein the physical address of the second data is the address of the second data in memory.

[0128] Optional, such as Figure 8 As shown, the data exchange apparatus 70 also includes an establishment unit 76. The establishment unit 76 is used to establish a swap-out thread and a swap-in thread after the target application starts. The swap-out thread is used to perform the swap-out operation of the first data, and the swap-in thread is used to perform the swap-in operation of the second data. For example, combined with... Figure 6 Unit 76 can be used to execute step 601.

[0129] Optionally, unit 76 is specifically used to establish swap-out and swap-in threads if the target application is included in a pre-stored whitelist, wherein the whitelist includes at least one application that supports memory swapping.

[0130] Of course, the data exchange device 70 provided in this application embodiment includes, but is not limited to, the modules described above.

[0131] In actual implementation, the acquisition unit 71, processing unit 72, release unit 73, deletion unit 74, setting unit 75, and creation unit 76 can be... Figure 2 The processor 201 shown calls program code in memory 202 to implement this. For details of its execution, please refer to [reference needed]. Figure 6 The description of the data exchange method shown is omitted here.

[0132] Another embodiment of this application provides a computer-readable storage medium storing computer instructions that, when executed on a host computer, cause the host computer to perform the various steps of the method flow shown in the above method embodiment.

[0133] Another embodiment of this application provides a chip system applied to a host computer. The chip system includes one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via lines. The interface circuits are used to receive signals from the host computer's memory and send signals to the processors, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the host computer performs the various steps executed by the host computer in the method flow shown in the above method embodiments.

[0134] In another embodiment of this application, a computer program product is also provided, which includes computer instructions that, when executed on a host, cause the host to perform the various steps executed by the host in the method flow shown in the above method embodiment.

[0135] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0136] The above description is merely a specific embodiment of this application. Any variations or substitutions conceived by those skilled in the art based on the specific embodiments provided in this application should be covered within the protection scope of this application.

Claims

1. A data exchange method applied to a host, characterized in that, The host is installed with an operating system and a target application, the operating system includes a user space, the user space is provided with a user interface of the target application, the user interface corresponds to a storage device, and the user interface includes: After the target application is started, an exchange-out thread and an exchange-in thread are established, the exchange-out thread is used to perform an exchange-out operation of first data, and the exchange-in thread is used to perform an exchange-in operation of second data; An exchange request of the target application is obtained, the exchange request is an exchange-in request or an exchange-out request, the exchange-out request is generated in a case that a memory space of the memory of the host for running the target application is insufficient, the exchange-in request is generated in a case that data location information corresponding to a virtual address of viewed data in a query operation request indicates the storage device, the exchange-out request is used to request to write data of the target application from the memory of the host to the storage device, and the exchange-in request is used to request to write data in the storage device to the memory; According to the exchange request, the user interface is called to perform an operation on the storage device.

2. The data exchange method of claim 1, wherein, The exchange request is the exchange-out request, and according to the exchange request, the user interface is called to perform an operation on the storage device, including: In response to the exchange-out request, first data of the target application is determined in the memory; A mapping relationship between a first virtual address and a first physical address in a pre-stored page table is released, the first virtual address is a virtual address of the first data in a virtual memory, and the first physical address is an address of the first data in the memory; A second virtual address is allocated for the first data, and a mapping relationship between the second virtual address and the first physical address is established; The user interface is called to write the first data indicated by the first physical address corresponding to the second virtual address from the memory to the storage device; A mapping relationship between the first virtual address and a first exchange entry is established in the page table, and the first exchange entry is used to indicate a position of the first data in the storage device.

3. The data exchange method of claim 2, wherein, The data exchange method further includes: The mapping relationship between the second virtual address and the first physical address is released, and the first data in the memory is deleted.

4. The data exchange method according to claim 2 or 3, characterized in that, The data exchange method further includes: State information of the first virtual address is set as first information, and the first information is used to indicate that an exchange type of the first data corresponding to the first virtual address is a user state exchange type.

5. The data exchange method of claim 1, wherein, The exchange request is the exchange-in request, and the exchange request of the target application is obtained, including: A virtual address of second data to be accessed is obtained; In a case that state information of the virtual address of the second data is first information, the second data is data of the target application, and the memory does not include the second data, the exchange-in request is generated, and the first information is used to indicate that an exchange type of the second data is a user state exchange type.

6. The data exchange method of claim 5, wherein, According to the exchange request, the user interface is called to perform an operation on the storage device, including: In response to the swap-in request, a second swap entry corresponding to a virtual address of the second data is obtained according to a pre-stored page table; The user-space interface is called to write the second data indicated by the second swap entry from the storage device to the memory; A mapping relationship between the virtual address of the second data and a physical address of the second data is established in the page table, and the physical address of the second data is an address of the second data in the memory.

7. The data exchange method according to any one of claims 1 to 6, characterized in that, The swap-out thread and the swap-in thread are established, including: In a case where the target application is included in a pre-stored white list, the swap-out thread and the swap-in thread are established, and the white list includes at least one application supporting memory swapping.

8. A data exchange device located at a host, characterized by The host is installed with an operating system and a target application, the operating system includes a user-space, the user-space is provided with a user-space interface of the target application, the user-space interface corresponds to a storage device, and the data exchange device includes: An establishing unit is configured to establish a swap-out thread and a swap-in thread after the target application is started, the swap-out thread is used to perform a swap-out operation of first data, and the swap-in thread is used to perform a swap-in operation of second data; An obtaining unit is configured to obtain a swapping request of the target application, the swapping request is a swap-in request or a swap-out request, the swap-out request is generated in a case where a memory space of the memory of the host for running the target application is insufficient, and the swap-in request is generated in a case where data location information corresponding to a virtual address of viewed data indicated by a query operation request indicates the storage device, the swap-out request is used to request writing data of the target application from the memory of the host to the storage device, and the swap-in request is used to request writing data in the storage device to the memory; A processing unit is configured to call the user-space interface to perform an operation on the storage device according to the swapping request obtained by the obtaining unit.

9. The data exchange apparatus according to claim 8, wherein The swapping request is the swap-out request, and the processing unit is specifically configured to: In response to the swap-out request, first data of the target application is determined in the memory; A mapping relationship between a first virtual address and a first physical address in a pre-stored page table is released, the first virtual address is a virtual address of the first data in a virtual memory, and the first physical address is an address of the first data in the memory; A second virtual address is allocated for the first data, and a mapping relationship between the second virtual address and the first physical address is established; The user-space interface is called to write the first data indicated by the first physical address corresponding to the second virtual address from the memory to the storage device; A mapping relationship between the first virtual address and a first swap entry is established in the page table, and the first swap entry is used to indicate a position of the first data in the storage device.

10. The data exchange apparatus according to claim 9, wherein The data exchange device further includes a releasing unit and a deleting unit. The releasing unit is configured to release the mapping relationship between the second virtual address and the first physical address. The deleting unit is configured to delete the first data in the memory.

11. The data exchange device according to claim 9 or 10, characterized in that The data exchange device further includes a setting unit. The setting unit is configured to set state information of the first virtual address as first information, the first information being used to indicate that an exchange type of the first data corresponding to the first virtual address is a user-mode exchange type.

12. The data exchange apparatus according to claim 8, wherein The exchange request is the swap-in request, and the obtaining unit is specifically configured to: obtain a virtual address of second data to be accessed; in a case where state information of the virtual address of the second data is first information, the second data is data of the target application, and the memory does not include the second data, generate the swap-in request, the first information being used to indicate that an exchange type of the second data is a user-mode exchange type.

13. The data exchange apparatus according to claim 12, wherein The processing unit is specifically configured to: in response to the swap-in request, obtain a second exchange entry corresponding to the virtual address of the second data according to a pre-stored page table; invoke the user-mode interface to write the second data indicated by the second exchange entry from the storage device to the memory; establish a mapping relationship between the virtual address of the second data and a physical address of the second data in the page table, the physical address of the second data being an address of the second data in the memory.

14. The data exchange device of any one of claims 8-13, wherein, The establishing unit is specifically configured to: in a case where the target application is included in a pre-stored whitelist, establish the swap-out thread and the swap-in thread, the whitelist including at least one application supporting memory exchange.

15. A data exchange device, characterized by The data exchange apparatus includes a memory and a processor; the memory and the processor are coupled; the memory is configured to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, the data exchange apparatus performs the data exchange method according to any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, The computer instructions, when executed on a data exchange apparatus, cause the data exchange apparatus to perform the data exchange method according to any one of claims 1-7.

Images