Data processing method, server and cluster

Abstract

The invention provides a data processing method, servers and a cluster, which can be applied to the technical field of memory management, and comprises the following steps: in the cluster, each server scans a first physical page allocated to the server, and compares the content of the first physical page with the content of a physical page of a first set in a memory pool when the state information of the server is a first state; the state information of each physical page is stored in the memory pool. If a first target physical page in the first set is the same as the content of the first physical page, it shows that the memory pool has repeated physical pages of the first physical page, mapping of the first physical page is transferred to the first target physical page, and the first physical page is released. According to the method, the scanning work of the page is executed on all the servers respectively, so that the problem of how to scan physical pages on other servers in a cluster is solved. In addition, by additionally defining state information for the physical pages, only the physical pages meeting a certain state are subjected to the step of memory deduplication, so that the deduplication time and the computing power cost are saved.

Description

Technical Field

[0001] The present application relates to the field of memory management technology, and in particular to a data processing method, server, and cluster. Background Art

[0002] With the massive growth of data, especially unstructured data such as images, digitized voice and video, the demand for memory in applications has increased significantly. In terms of hardware, the industry is also constantly introducing new interconnection technologies, which can connect multiple devices and realize the sharing of hardware resources. In August 2022, the open interconnection protocol (Compute Express Link, CXL) 3.0 protocol standard was released. The new standard supports cache consistency, which allows multiple physical machines or accelerators to access the same memory and ensure data correctness.

[0003] With the support of new interconnection technologies, it is possible to collect the memory within a cluster to form a memory pool and uniformly allocate memory to all devices in the cluster in a server cluster form, so that each device will have more available memory.

[0004] However, memory often contains a large amount of duplicate data. Memory deduplication can significantly improve memory utilization and reduce memory costs. Content-Based Page Sharing (CBPS) is widely used in the industry to implement memory deduplication. On a single host, the memory deduplication algorithm used in the Linux operating system is called Kernel Same Page Merging (KSM). KSM is not applicable to a cluster because the memory in the cluster's memory pool belongs to different hosts and is managed by their respective operating systems. Deduplication requires consideration of inter-host coordination and data access correctness. Summary of the Invention

[0005] The present application provides a data processing method, server, and cluster for solving the memory deduplication problem of the cluster. Specifically, for a multi-server cluster, during the page scanning process, each server only scans its own physical pages and compares them with the physical pages of the first set in the memory pool (built in real time during the scanning process of each server), solving the problem of how to scan the physical pages on other servers in a multi-server cluster. In addition, by additionally defining status information for the physical pages, only physical pages that meet a certain status (i.e., the first status) are subject to the memory deduplication step, saving deduplication time and computing power costs.

[0006] Based on this, the embodiments of the present application provide the following technical solutions:

[0007] In the first aspect, the present application first provides a method for data processing, which is applied to a cluster, and the cluster includes n servers, n≥2, and each server in the cluster acts as an execution subject to execute the embodiment method of the present application. The method specifically includes: first, each server in the cluster will scan the physical page assigned to it. The server that is currently performing page scanning is called the first server, and the scanned physical page assigned to the first server is called the first physical page. When the first server scans each of its own first physical pages in turn, it will also determine the current status information of each first physical page in turn. When the status information of the first physical page is the first state, the first server will compare the content of the first physical page with the physical pages in the first set in the memory pool. Among them, one physical page corresponds to one status information, and the status information of each physical page is stored in the memory pool, which can be specifically stored as memory pool metadata in the memory pool metadata area. The memory pool is a logical collection of the memory of each server in the cluster. If a physical page with the same content as the first physical page exists in the first set, this physical page is called the first target physical page. The first server then transfers the mapping of the first physical page to the first target physical page. This way, the first target physical page not only has its own mapping but also the mapping of the first physical page. This first target physical page is called a shared page. Since the mapping of the first physical page has been transferred to the first target physical page, the first server releases the physical page, returning it to a free page.

[0008] In the above-mentioned embodiment of the present application, for a cluster of multiple servers, during the page scanning process, each server only scans its own physical pages and compares them with the physical pages of the first set in the memory pool, solving the problem of how to scan the physical pages on other servers in a cluster of multiple servers. In addition, by additionally defining status information for the physical pages, only physical pages that meet a certain state (i.e., the first state) are subjected to the memory deduplication step, saving deduplication time and computing power costs. The embodiment of the present application can reduce the memory duplication rate in the cluster and increase the available memory.

[0009] In a possible implementation of the first aspect, in addition to transferring the mapping of the first physical page to the first target physical page, a reverse mapping of the first target physical page may be updated based on the mapping of the first physical page.

[0010] In the above embodiment of the present application, it is described how to record reverse mapping when multiple servers share a physical page, thereby solving the problem of recording reverse mapping after memory deduplication.

[0011] In a possible implementation of the first aspect, when the status information of the first physical page is the first state, an implementation method for comparing the content of the first physical page with the physical pages in the memory pool belonging to the first set may be: first, when the first server determines that the status information of the first physical page is the first state, it further determines whether the first physical page is marked with a target identifier by other servers in the cluster (which may be called the second server), and the target identifier may also be called a deduplication identifier. If so, the first server will further compare the content of the first physical page with the physical pages in the memory pool belonging to the first set.

[0012] In the above-mentioned embodiment of the present application, the first physical page being in the first state is only a major prerequisite for the subsequent content comparison. In order to reduce the number of pages and times of comparison, the first physical page is further screened, and then the content of the screened first physical page is compared with the physical pages of the first set to simplify the deduplication step and save time for content comparison.

[0013] In one possible implementation of the first aspect, when the state information of a first physical page is in the first state, before comparing the content of the first physical page with the physical pages in the first set within the memory pool, it is also necessary to determine whether the first physical page has just been updated from the second state to the first state. That is, first, it is determined that the first physical page is in the second state and the content of the first physical page has not changed within a preset time period (i.e., the content is stable within the preset time period). Then, the state information of the first physical page is updated from the second state to the first state. At this time, the first physical page satisfies the following conditions: ① the state information is in the first state; ② the first state has just been converted from the second state.

[0014] In the above-mentioned embodiment of the present application, it is defined that the first state of the first physical page is just converted from the second state, thereby reducing the number of pages and the number of times of comparison, thereby achieving the purpose of simplifying the deduplication steps.

[0015] In a possible implementation of the first aspect, when the status information of the first physical page is in the first state, before comparing the content of the first physical page with the physical pages belonging to the first set in the memory pool, the method may also include: determining that the first physical page is in the first state and the content of the first physical page has changed. At this time, the first server will update the status information of the first physical page from the first state to the second state, and move the first physical page out of the second set in the memory pool (private pages are initially in the second set).

[0016] In the above-mentioned embodiment of the present application, the processing method for the situation in which the first physical page exists in the first state but the content has changed during the screening process is specifically described, which is comprehensive and feasible.

[0017] In a possible implementation of the first aspect, after comparing the content of the first physical page with the physical pages in the first set in the memory pool, if the content of the first physical page is different from the content of the physical pages in the first set, the first physical page is further compared with the physical pages in the second set in the memory pool. When there is a physical page with the same content as the first physical page in the second set, in this case, the physical page with the same content as the first physical page in the second set can be called the second target physical page. The first server adds the first physical page to the first set (the physical pages in the first set are shared pages). Since there is a physical page with the same content as the first physical page, the first physical page can be used as a shared page, and the shared page needs to be added to the first set. In addition, the first server also needs to update the status information of the first physical page from the first state to the third state.

[0018] In the above embodiment of the present application, it is specifically described that if no duplicate page is found in the first set, the search continues in the second set; if a duplicate page is found in the second set, the first physical page is placed in the first set and updated to the third state. This is feasible.

[0019] In a possible implementation manner of the first aspect, when the content of the first physical page is identical to the content of the second target physical page, the mapping of the first physical page may be changed to read-only.

[0020] In the above embodiment of the present application, when the content of the first physical page is the same as the content of the second target physical page, the first server can further change the mapping of the first physical page to read-only to avoid data errors when multiple servers access the first physical page at the same time.

[0021] In a possible implementation manner of the first aspect, when the content of the first physical page is identical to the content of the second target physical page, the second target physical page may be further marked with a target identifier.

[0022] In the above-mentioned embodiment of the present application, the method of adding target identifiers ensures that during the page deduplication process, each server only processes its own physical pages. For non-shared pages of other servers, only the physical pages are marked, and the marked physical pages are delayed until the other server performs page deduplication. This cross-server delayed deduplication mechanism solves the problem that the current server cannot operate the physical pages of other servers.

[0023] In a possible implementation of the first aspect, after comparing the contents of the first physical page with the physical pages in the memory pool belonging to the second set, if the content of the first physical page is different from the content of the physical pages in the second set, the first physical page is moved into the second set.

[0024] In the above implementation of the present application, if no duplicate page is found in the second set, the first physical page is added to the second set for the next content comparison of each server in the cluster, which is feasible.

[0025] In a possible implementation of the first aspect, the method for scanning the first physical page assigned to the first server may be: periodic scanning, for example, the page scanning step may be performed by a periodically awakened thread; the method for the first server to scan the page may also be conditional triggered scanning, when a preset condition is met, the thread is triggered to perform the page scanning step, and this application does not specifically limit this.

[0026] In the above-mentioned embodiments of the present application, several typical page scanning methods are specifically described, which are flexible.

[0027] In one possible implementation of the first aspect, the memory pool further includes a memory pool metadata area. The memory pool metadata area stores memory pool metadata. The memory pool metadata is used to record status information (e.g., first status, second status, etc.) of physical pages of each server in the cluster, as well as the correspondence between physical pages and servers (i.e., which server uses the physical page). One piece of memory pool metadata corresponds to one physical page.

[0028] In the above-mentioned embodiment of the present application, a memory pool metadata is created for each physical page through a new secondary management structure, and the page metadata of the physical page on each server can be associated with this metadata, thereby solving the problem of physical page management when multiple servers use the same physical page.

[0029] A second aspect of the present application provides a server having the functionality to implement the method of the first aspect or any possible implementation of the first aspect. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functionality described above.

[0030] A third aspect of the present application provides a cluster, which includes n servers, n≥2, and the servers may be the servers described in the second aspect above.

[0031] The fourth aspect of the present application provides a server, which may include a memory, a processor, and a bus system, wherein the memory is used to store computer programs (also referred to as programs, computer-readable instructions), and the processor is used to call the program stored in the memory to execute the method of the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0032] In a fifth aspect, the present application provides a computer-readable storage medium having instructions stored therein. When the instructions are executed on a computer, the computer can execute the method of the first aspect or any possible implementation of the first aspect.

[0033] In a sixth aspect, the present application provides a computer program or a computer program product containing instructions. When the computer program or computer program product runs on a computer, it enables the computer to execute the method of the above-mentioned first aspect or any possible implementation of the first aspect.

[0034] In the seventh aspect of the present application, a chip is provided, which includes at least one processor and at least one interface circuit, the interface circuit is coupled to the processor, the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, and the at least one processor is used to run a computer program or instruction, which has the function of implementing the method of the first aspect or any possible implementation of the first aspect. The function can be implemented by hardware, software, or a combination of hardware and software, and the hardware or software includes one or more modules corresponding to the above functions. In addition, the interface circuit is used to communicate with other modules outside the chip.

[0035] In some implementations of the present application, some of the one or more processors may implement some steps of the above method through dedicated hardware. For example, processing involving a neural network model may be implemented by a dedicated neural network processor or a graphics processor.

[0036] The method provided in the embodiment of the present application can be implemented by one chip or by multiple chips working together. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of the system architecture provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of an implementation of the method of the embodiment of the present application in server cluster hardware provided in the embodiment of the present application;

[0039] Figure 3 A flowchart of a data processing method provided in an embodiment of the present application;

[0040] Figure 4 Another flowchart of the data processing method provided in an embodiment of the present application;

[0041] Figure 5 An algorithm flow chart provided for an embodiment of the present application;

[0042] Figure 6 A component structure diagram of an example provided in an embodiment of the present application;

[0043] Figure 7 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0044] Figure 8 A schematic diagram of the structure of a cluster provided in an embodiment of the present application;

[0045] Figure 9 Another structural diagram of the server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] This application provides a data processing method, server, and cluster for solving the cluster's memory deduplication problem. This is achieved through a proposed decentralized page scanning mechanism, whereby the page scanning work is distributed among all servers for execution, and then compared with the physical pages of the first set in the memory pool. This solves the problem of how to scan physical pages on other servers in a multi-server cluster. In addition, by additionally defining status information for physical pages, only physical pages that meet certain statuses are subject to memory deduplication, saving deduplication time and computing power costs.

[0047] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0048] In order to better understand the solutions of the embodiments of the present application, the following first introduces the relevant terms and concepts that may be involved in the embodiments of the present application. It should be understood that the interpretation of the relevant terms and concepts may be limited by the specific circumstances of the embodiments of the present application, but it does not mean that the present application is limited to only such specific circumstances. The specific circumstances of different embodiments may also vary, which is not specifically limited here.

[0049] (1)host

[0050] The host refers to a server computer, which can be referred to as the server for short.

[0051] (2) Cluster

[0052] All hosts connected by the interconnect bus form a cluster.

[0053] (3) Remote Memory

[0054] In a cluster, the memory on each host is local memory to itself, and the memory of other hosts connected through the interconnection bus is called remote memory.

[0055] (4) Physical page

[0056] In the Linux kernel, all available physical memory is divided into contiguous parts of the same size, called blocks or page frames, which can also be called physical pages, pages, or pages.

[0057] (5) Mapping

[0058] In the Linux kernel, when allocating physical pages to processes, a mapping between the virtual address of the user address space and the physical page is established. The process does not directly access the physical address of the physical page, but accesses the physical page through the virtual address. At this time, the hardware memory management unit (MMU) is responsible for the mapping conversion.

[0059] (6) Reverse mapping

[0060] Finding all virtual addresses that map a physical page through it is called reverse mapping. The Linux operating system maintains reverse mapping information for each physical page to quickly find all processes that map the physical page.

[0061] (7) Page metadata

[0062] Consider the Linux kernel's struct page structure, which represents the metadata of a physical page and records the state of the physical page on a host. On each host, page metadata corresponds one-to-one with the physical page. If the physical page is shared (i.e., exists on at least two hosts), each host will have page metadata, containing information specific to that host.

[0063] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0064] First, the system architecture and overall process of the method of the embodiment of the present application are described. Figure 1 , Figure 1This is a schematic diagram of a system architecture provided in an embodiment of the present application. The system architecture includes hardware 101, a memory pool 102, and an operating system 103, which are described below:

[0065] (1) Hardware 101

[0066] Using new interconnect bus technologies, such as CXL, multiple hosts or their memory devices are connected to form a cluster consisting of multiple devices. Each host can access the memory on any device in the cluster through the interconnect bus. All memory in the cluster is logically pooled to form a memory pool, which can provide memory resources to any computer in the cluster.

[0067] (2) Memory Pool 102

[0068] All memories within the cluster are logically abstracted and centralized to form a memory pool, which supports providing memory resources to any host within the cluster. The memory pool is responsible for managing all physical memories within the cluster. In the embodiment of the present application, it can be divided into a memory pool metadata area 1021 and an available memory area 1022, wherein the memory of the memory pool metadata area 1021 is used to record the allocation and usage of all memories. Specifically, the memory pool metadata area 1021 includes multiple memory pool metadata, and the memory pool metadata is used to record the status information of the physical pages of each server in the cluster (such as the first state, second state, third state, etc. described below) and the correspondence between the physical page and the server (i.e., the information on which host uses the physical page). One memory pool metadata corresponds to one physical page. The memory of the available memory area 1022 is used to be allocated to each host in the cluster for use.

[0069] It should be noted that, in the following embodiments, the physical pages in the first set and the second set constructed in real time belong to the memory of the available memory area 1022, and the relevant status information of the first set and the second set can be recorded in the memory pool metadata area 1021. In the embodiment of the present application, the first set can also be called a shared set, and the physical pages belonging to the shared set are shared pages (i.e., multiple servers in the cluster are allowed to use them), and the second set can also be called a private set, and the physical pages belonging to the private set are private pages (i.e., only one server in the cluster is allowed to use them).

[0070] (3) Operating System 103

[0071] That is, the operating system deployed on each host, mainly referring to the memory subsystem here, which is responsible for the management and control of memory resources. It can include: memory pool management module 1031, memory management module 1032 and memory deduplication module 1033, among which the memory pool management module 1031 is responsible for managing the memory pool, including initialization of the memory pool, expansion and contraction of the memory pool, update of memory pool metadata, allocation of memory from the memory pool, etc.; the memory management module 1032 is responsible for allocating memory to processes on each host, and it needs to first apply for memory from the memory pool management module 1031 before allocating it to the process; the memory deduplication module 1033 is responsible for finding physical pages with duplicate content in the memory pool and performing page deduplication tasks.

[0072] The method of the embodiment of the present application is used to deduplicate memory, thereby saving memory. It can be widely used in business scenarios in most server fields, including cloud computing, big data, AI large models and other scenarios. Specifically, this application does not limit the actual application scenarios.

[0073] It should be noted that, in the embodiments of this application, Figure 1 The system architecture is for illustration only and does not limit the deployment of each unit / module.

[0074] The product implementation form of this application is a program code included in the operating system software and deployed on the server hardware. During operation, the program code of this application runs in the operating system of each server in the cluster. Figure 2 An implementation form of the present application in a cluster is given, in which the memory pool management and memory deduplication of the software part and the memory of the hardware part are newly added modules based on the existing operating system software. As an example, within the memory management, the present device may include: a memory allocation module, a page metadata management module, a reverse mapping module, etc.; in the memory pool management, the present device may include: a memory pool metadata management module; in the memory deduplication, the present device may include: a page scanning module, a page state conversion module, a page deduplication module, etc. In the memory, memory space is allocated to each physical page to store the corresponding memory pool metadata, and the rest is available memory for allocation to each server.

[0075] It should be noted that, in the embodiments of this application, Figure 2 The division of units or modules into software and hardware parts is for illustration only. In some implementations, they can be divided into more fine-grained modules / sub-modules to achieve functional decoupling; in other embodiments, fewer modules can be used to reduce the interaction between modules, which is not limited in this application.

[0076] Based on the above system architecture, the data processing method provided by the embodiment of the present application is introduced below. The method is applied to a cluster, which includes n servers, n≥2, and each server in the cluster acts as an execution subject to execute the method of the embodiment of the present application. For details, please refer to Figure 3 , Figure 3 A flow chart of a data processing method provided in an embodiment of the present application specifically includes the following steps:

[0077] 301. A first server scans a first physical page allocated to the first server.

[0078] First, each server in the cluster scans the physical page assigned to it. The server currently scanning the page is called the first server, and the physical page to be scanned assigned to the first server is called the first physical page.

[0079] It should be noted that, in some embodiments of the present application, the first server may scan the page in a periodic manner. For example, the page scanning step may be performed by a periodically awakened thread. The first server may scan the page in a conditionally triggered manner. When a preset condition is met, the thread is triggered to perform the page scanning step. This application does not specifically limit this.

[0080] It should also be noted that in some embodiments of the present application, for ease of management, the memory pool can be divided into an available memory area and a memory pool metadata area. The physical pages allocated to each server are all from the available memory area of ​​the memory pool. This means that the physical page scanned by the first server will not be the physical page of the memory pool metadata area. The memory pool metadata area of ​​the memory pool stores memory pool metadata. The memory pool metadata is used to record the status information of the physical pages of each server in the cluster (such as the first state, the second state, etc.) and the correspondence between the physical page and the server (that is, which server uses the physical page). One memory pool metadata corresponds to one physical page. This cluster metadata management structure creates a metadata for each physical page, through which the page metadata of the physical page on each server can be associated. Since the page metadata records the reverse mapping of the physical page, when multiple servers share the same physical page, the metadata can be associated with the corresponding page metadata, and the reverse mapping of the page metadata can be updated accordingly. This can solve the problem of physical page management when multiple servers use the same physical page.

[0081] It should be noted that since the first server scans the first physical page assigned to it, the content of the first physical page is not blank but is used. In the embodiment of the present application, a physical page that has not been used (i.e., has blank content) is called a free page, and a physical page in the available memory area of ​​the memory pool is a free page. Once a free page is assigned to a server, it is no longer a free page.

[0082] 302. When the status information of the first physical page is in the first state, the first server compares the content of the first physical page with the physical pages in the memory pool that belong to the first set, and stores the status information in the memory pool.

[0083] As the first server sequentially scans each of its first physical pages, it simultaneously determines the current state information for each first physical page. If the state information for a first physical page is the first state, the first server compares the content of that first physical page with the physical pages in the first set within the memory pool. Each physical page corresponds to a piece of state information, and the state information for each physical page is stored in the memory pool, specifically as memory pool metadata in the memory pool metadata area. The memory pool is a logical collection of the memory of each server in the cluster.

[0084] It should be noted that, in some embodiments of the present application, the first state may mean that the physical page is only allowed to be used by one server in the cluster, and the content of the physical page does not change within a preset time period (that is, the content is stable within the preset time period). The physical page in the first state may also be called a private page.

[0085] It should also be noted that, in the embodiment of the present application, the first physical page being in the first state is only a major prerequisite for subsequent content comparison. In other embodiments of the present application, in order to reduce the number of pages and the number of times compared, the first physical page can be further filtered, and then the content of the filtered first physical page can be compared with the physical pages of the first set. The further filtering process includes but is not limited to:

[0086] (1) The state information of the first physical page is in the first state, and the first physical page is marked with a target identifier by other servers in the cluster.

[0087] In an embodiment of the present application, when the first server determines that the status information of the first physical page is the first state, it further determines whether the first physical page is marked with a target identifier by other servers in the cluster (which may be called the second server). The target identifier may also be called a deduplicated identifier. If so, the first server will further compare the content of the first physical page with the physical pages in the memory pool belonging to the first set.

[0088] (2) The state information of the first physical page is just updated from the second state to the first state.

[0089] In the embodiment of the present application, when the first server determines that the state information of the first physical page is in the first state, it also needs to determine whether the first physical page has just been updated from the second state to the first state. In other words, if the first physical page is initially in the second state, and the content of the first physical page has not changed within a preset time period (i.e., the content is stable within the preset time period), the state information of the first physical page is updated from the second state to the first state. At this time, the first physical page satisfies the following conditions: ① The state information is in the first state; ② The first state has just been converted from the second state.

[0090] It should be noted that, in some embodiments of the present application, the second state may mean that the physical page is only allowed to be used by one server in the cluster, and the content of the physical page has changed within a preset time period (that is, the content is unstable within the preset time period). The physical page in the second state may also be called an unstable page.

[0091] It should also be noted that during the above screening process, there is another situation: the first physical page is in the first state, but its content has changed. In this case, the first physical page can also be called a changed private page. In this case, the first server will update the state information of the first physical page from the first state to the second state, that is, convert the changed private page to an unstable page, and remove the first physical page from the second set in the memory pool (private pages are initially in the second set).

[0092] It should be noted that the relevant information of the first set and the second set described in the embodiment of the present application can be recorded in the memory pool metadata area of ​​the memory pool, and the physical pages included in the first set and the second set are present in the available memory area of ​​the memory pool. The physical pages in the first set and the second set change in real time, which is determined by the physical pages being added or removed from the corresponding set in real time by each server during the execution of the method of the embodiment of the present application.

[0093] 303. When the content of the first physical page is identical to the content of the first target physical page, the first server transfers the mapping of the first physical page to the first target physical page and releases the first physical page. The first target physical page is a physical page in the first set.

[0094] If a physical page with the same content as the first physical page exists in the first set, this physical page is called the first target physical page. The first server then transfers the mapping of the first physical page to the first target physical page. This way, the first target physical page not only has its own mapping but also the mapping of the first physical page. This first target physical page is called a shared page. Since the mapping of the first physical page has been transferred to the first target physical page, the first server releases the physical page, returning it to a free page.

[0095] It should be noted that in some embodiments of the present application, in addition to transferring the mapping of the first physical page to the first target physical page, the reverse mapping of the first target physical page can also be updated based on the mapping of the first physical page, thereby solving the problem of recording the reverse mapping after memory deduplication.

[0096] It should also be noted that, in some embodiments of the present application, if the first server does not find a physical page in the first set with the same content as the first physical page, that is, the content of the first physical page is different from the content of the physical pages in the first set, in this case, the first server further compares the content of the first physical page with the physical pages in the second set in the memory pool, and may perform different steps based on different comparison results:

[0097] (1) When there is a physical page in the second set with the same content as the first physical page.

[0098] In this case, the physical page in the second set that has the same content as the first physical page can be called the second target physical page. The first server adds the first physical page to the first set because the physical pages in the first set are shared pages. Since there is a physical page with the same content as the first physical page, the first physical page can be used as a shared page and needs to be added to the first set. In addition, the first server also needs to update the status information of the first physical page from the first state to the third state.

[0099] It should be noted that, in some embodiments of the present application, the third state may mean that the physical page is allowed to be used by multiple servers in the cluster, and the content of the physical page does not change within a preset time period (that is, the content is stable within the preset time period). The physical page in the third state is a shared page.

[0100] It should also be noted that, in some other embodiments of the present application, when the content of the first physical page is the same as the content of the second target physical page, the first server may further change the mapping of the first physical page to read-only to avoid data errors when multiple servers access the first physical page at the same time.

[0101] It should also be noted that in some other embodiments of the present application, when the content of the first physical page is the same as the content of the second target physical page, the first server can further mark the second target physical page with a target identifier (i.e., a deduplication identifier). This method ensures that during the page deduplication process, each server only processes its own physical pages. For non-shared pages of other servers, only the physical pages are marked. The marked physical pages are delayed until the other server performs page deduplication. This cross-server delayed deduplication mechanism solves the problem that the current server cannot operate the physical pages of other servers.

[0102] (2) When there is no physical page in the second set with the same content as the first physical page.

[0103] In this case, the first server does not find a physical page with the same content as the first physical page in the second set, that is, the content of the first physical page is different from the content of the physical pages in the second set, indicating that the first physical page is not a shared page. Then the first physical page is added to the second set for the next content comparison of each server in the cluster.

[0104] It should be noted that in the above embodiments of the present application, each server is described to perform page scanning and page deduplication. In actual applications, before implementing the method of the embodiment of the present application, it is necessary to initialize the memory pool. If new memory is added, the memory pool needs to be updated, and the memory pool metadata and page metadata information of the memory need to be updated when allocating memory to each server.

[0105] The following describes the entire process of initializing the memory pool, processing new memory, scanning pages, and deduplicating memory, combining the above data processing methods. For details, please refer to Figure 4 , Figure 4 Another flowchart of the data processing method provided in an embodiment of the present application is provided. For ease of explanation, a private page with a target identifier or a private page that has just been converted from an unstable page is used as a comparison object for memory deduplication. As an example, the method may include the following steps:

[0106] 401. Initialize the memory pool and add memory to the memory pool.

[0107] This step is performed when each server in the cluster is turned on. In this step, if the server that is turned on is the first server in the cluster, the memory pool needs to be initialized, otherwise it does not need to be initialized. Afterwards, the memory pool is expanded (for example, the memory pool can be expanded by the memory pool management module), and memory pool metadata is created for the memory newly added to the memory pool. Then the memory is added to the memory pool. The memory pool metadata records the global status information of the physical page, for example, it can include a first state (i.e., private page), a second state (i.e., unstable page), a third state (i.e., shared page), and a fourth state (i.e., free page). When the newly added memory is initially added, the memory in the memory pool metadata area is all unstable pages, and the memory in the available memory area is all free pages. The memory subsequently allocated to each server for use is all memory in the available memory area.

[0108] 402. Update the memory pool metadata and page metadata of the memory when allocating memory.

[0109] This step is performed when the operating system on a server allocates memory to a process. In this step, one or more free pages x are obtained from the memory pool (e.g., performed by a memory allocation module), and then the memory pool metadata of these free pages x are updated (e.g., performed by a memory pool metadata management module). The one or more free pages x are converted into a corresponding number of unstable pages y (i.e., all allocated free pages are converted, and whether they are free pages or unstable pages can be identified by additionally recorded marks, and the conversion is also achieved by changing the marks). Page metadata is created on the current server (one page corresponds to one page metadata). The page metadata records the information of the physical page on the current server, including the mapping and reverse mapping of the physical page on the current server.

[0110] 403. Periodically scan the physical pages allocated to the current server on each server and search for duplicate physical pages in the memory pool.

[0111] This step is performed by a thread on each server in the cluster. If periodic scanning is performed, the thread can be a periodically awakened thread. For ease of explanation, the following explanation uses periodic scanning as an example and will not be repeated.

[0112] In this step, a periodic scan (e.g., performed by a page scanning module) is performed on the non-shared pages (i.e., unstable pages or private pages, which may be one or more) currently used by the current server to determine whether the content of the physical page has changed since the last scan. For unstable pages that have not changed and private pages that have changed, step 404 is executed. Then, for private pages with target identifiers (marked by other servers), step 405 is executed.

[0113] 404. Convert the scanned physical pages with stable content into private pages, and convert the scanned physical pages with unstable content into unstable pages.

[0114] This step is executed by a thread on each server in the cluster. This can be the thread in step 403 or an additional thread, without limitation. If periodic scanning is performed, this thread can be a periodically awakened thread. For ease of explanation, the following description uses periodic scanning as an example and will not be further elaborated.

[0115] In this step, private pages with changed content are converted to unstable pages, and unstable pages that have not changed within a preset period of time are converted to private pages (for example, this can be performed by the page state conversion module), and step 405 is executed for the newly converted private pages.

[0116] 405. For the scanned duplicate physical pages, the mappings of both physical pages are transferred to one of the physical pages, and the other physical page is released.

[0117] This step is executed by a thread on each server in the cluster. This can be the thread in step 403 or an additional thread, without limitation. If periodic scanning is performed, this thread can be a periodically awakened thread. For ease of explanation, the following description uses periodic scanning as an example and will not be further elaborated.

[0118] In this step, the physical page that has been executed to this step (i.e., the private page with the target identifier in step 403 and the private page that has just been converted in step 404) is compared with the shared pages on the first set (e.g., the shared tree) and the private pages on the second set (e.g., the private tree) in the cluster, and a physical page with the same content is searched for. If a shared page with the same content is found on the shared tree, all mappings of the physical page on the current server are transferred to the shared page, the reverse mapping of the shared page is updated, and the physical page is released (e.g., it can be executed by the page deduplication module); if a shared page with the same content is not found on the shared tree, the search continues on the private tree. When a private page with the same content is found on the private tree, the physical page is converted to a shared page and placed on the shared tree. All its mappings on the current server are modified to read-only, and the corresponding private page is marked as the target identifier (e.g., it can be executed by the page deduplication module); when a page with the same content is not found on either tree, the physical page is inserted into the private tree.

[0119] It should be noted that, in the embodiment of the present application, steps 403 to 405 constitute the core algorithm of the present application, which can be specifically described as follows: Figure 5 As shown, Figure 5This is an algorithm flow chart provided by the embodiment of the present application. The detailed process of the process is shown in steps 403 to 405, which will not be described here. The algorithm can be called RKSM algorithm, and the thread that executes the algorithm can be named rksmd thread. The complete process of rksmd thread is as follows Figure 5 shown.

[0120] To facilitate further understanding of the above embodiment method, the following takes a specific example as an example to illustrate the above embodiment method. First, the component structure of the example is explained. For details, please refer to Figure 6 , Figure 6 The component structure diagram of an example provided in the embodiment of this application is a memory deduplication technology implemented in the Linux kernel in a cluster built on the CXL interconnect bus. This embodiment of the application can reduce memory duplication and increase available memory.

[0121] As an example, in the Linux kernel, the modules involved in the embodiments of the present application may include a memory management module, a memory pool management module, and a page deduplication module. Among them, the memory management module may include a memory allocation module, a page metadata management module, and a reverse mapping module; the memory pool management module may include a memory pool metadata management module; the memory deduplication module may include a page scanning module, a page state conversion module, and a page deduplication module. In server hardware, the components involved in the embodiments of the present application may include local memory, a CXL bus, and memory on other hosts connected via the CXL bus. In order to achieve decentralized page deduplication on multi-server shared memory, the embodiments of the present application implement the following specific steps:

[0122] (1) Initialize the memory pool and add memory to the memory pool.

[0123] This step is performed when the Linux system on each server in the cluster boots. Before booting, some local memory is isolated to prevent it from being used during system startup. During boot, the server allocates memory pool metadata for this isolated memory (e.g., this can be performed by the memory pool management module). This isolated memory is then added to the memory pool and the memory pool information is updated. After this, all servers in the cluster can apply for use of this memory.

[0124] (2) When allocating memory, the memory pool metadata and page metadata information of the memory are updated.

[0125] This step is executed when a page fault occurs in a process on each server in the cluster and memory allocation occurs (page fault is just an example and is not the only condition that triggers memory allocation). Specifically, the memory pool is first locked (e.g., it can be executed by the memory pool management module) to prevent other servers from accessing the memory pool at the same time and causing concurrency errors. Then, one or more free pages are taken from the memory pool (the specific number of free pages is determined at the time of allocation), and the one or more free pages are converted into unstable pages and then the memory pool is unlocked. Page management metadata is allocated to each of the one or more physical pages, and then they are allocated to the process with the page fault (e.g., it can be executed by the memory allocation module), and mapping and reverse mapping are established.

[0126] (3) Periodically scan the physical pages allocated to the current server on each server and search for duplicate physical pages in the memory pool.

[0127] This step is performed by the rksmd thread in the Linux kernel on each server in the cluster. In this step, each non-shared page (i.e., private page + unstable page) assigned to the current server is periodically scanned, and it is determined whether the content of each non-shared page has changed compared to the previous round of scanning. As an example, the hash value of the page content can be calculated to determine whether the content has changed. Specifically, first, the hash value of the page content is calculated and compared with the hash value calculated when the physical page was last scanned. By comparing the hash values ​​of the two rounds of scanning, it is determined whether it is a "changed private page" or an "unchanged unstable page". If the hash value of the private page changes or the hash value of the unstable page remains unchanged, it is a case where the status information needs to be corrected, and step (four) is executed. For private pages with target identifiers, step (five) is executed.

[0128] (4) Convert the scanned physical pages with stable contents into private pages, and convert the scanned physical pages with unstable contents into unstable pages.

[0129] This step is performed by the rksmd thread in the Linux kernel on each server in the cluster. When the physical page is a private page and the hash value of the private page changes, the state of the page is changed from private page to unstable page, and the page is removed from the private tree; when the physical page is an unstable page and the hash value of the unstable page remains stable within a preset time, the physical page should be a private page or a shared page, but a physical page with the same content has not yet been found, so the state of the page is changed from unstable page to private page. At this time, the private page has just been converted from an unstable page, and step (5) is performed on it.

[0130] (5) For the duplicate physical pages scanned, the mapping of the two pages is transferred to one of the pages, and the other page is released.

[0131] This step is performed by the rksmd thread in the Linux kernel on each server in the cluster. The private page with the target identifier in the above step (3) and the private page just converted from the unstable page in step (4) can both be called the physical page to be deduplicated. For the physical page to be deduplicated, first find a shared page with the same content on the shared tree. If found, all mappings and reverse mappings of this physical page are transferred to the shared page, and then release this physical page. If no shared page with the same content is found on the shared tree, continue to find a private page with the same content on the private tree. If found, convert this physical page into a shared page and insert it into the shared tree. Add the target identifier to the found private page. When the server to which it belongs performs a scan and scans the page, deduplication is performed (the mapping must be transferred to the shared page of the shared tree before deletion, because each server can only process its own physical page). If no page with the same content is found on the two trees, insert this physical page into the private tree.

[0132] It should be noted that the embodiments of the present application can be applied not only to clusters built based on the CXL interconnect bus, but also to clusters of other interconnect buses. As long as any server in the cluster can access the memory of other servers in the cluster, the method of the embodiments of the present application can be used.

[0133] On the basis of the above embodiments, in order to better implement the above solutions of the embodiments of the present application, the following also provides related devices for implementing the above solutions. Figure 7 , Figure 7 A schematic diagram of a server provided in an embodiment of the present application, the server 700 may specifically include: a scanning module 701, a content comparison module 702 and a release module 703, wherein the scanning module 701 is used to scan the first physical page assigned to the server, the server belongs to a cluster, the cluster includes n servers, n≥2; the content comparison module 702 is used to compare the content of the first physical page with the physical pages belonging to the first set in the memory pool when the status information of the first physical page is the first state, the memory pool is a collection of memories of each server in the cluster, the status information is stored in the memory pool, and one physical page corresponds to one status information; the release module 703 is used to transfer the mapping of the first physical page to the first target physical page and release the first physical page when the content of the first physical page is the same as the content of the first target physical page, the first target physical page is a physical page in the first set.

[0134] In a possible design, the release module 703 is further configured to: after transferring the mapping of the first physical page to the first target physical page, update the reverse mapping of the first target physical page based on the mapping of the first physical page.

[0135] In one possible design, the content comparison module 702 is specifically used to: when the status information of the first physical page is the first state, determine whether the first physical page is marked with a target identifier by other servers in the cluster; if so, compare the content of the first physical page with the physical pages in the memory pool belonging to the first set.

[0136] In one possible design, the content comparison module 702 is further used to: before comparing the content of the first physical page with the physical pages belonging to the first set in the memory pool, determine that the first physical page is in the second state and the content of the first physical page has not changed within a preset time period; and update the state information of the first physical page from the second state to the first state.

[0137] In one possible design, the content comparison module 702 is further used to: before comparing the content of the first physical page with the physical pages belonging to the first set in the memory pool, determine that the first physical page is in a first state and the content of the first physical page has changed; update the state information of the first physical page from the first state to the second state, and move the first physical page out of the second set in the memory pool.

[0138] In one possible design, the content comparison module 702 is further used to: when the content of the first physical page is different from the content of the physical pages in the first set, compare the content of the first physical page with the physical pages in the second set in the memory pool; when the content of the first physical page is the same as the content of the second target physical page, add the first physical page to the first set, and update the status information of the first physical page from the first state to the third state, and the second target physical page is a physical page in the second set.

[0139] In a possible design, the content comparison module 702 is further configured to: when the content of the first physical page is identical to the content of the second target physical page, change the mapping of the first physical page to read-only.

[0140] In a possible design, the content comparison module 702 is further configured to: when the content of the first physical page is identical to the content of the second target physical page, mark the second target physical page with a target identifier.

[0141] In a possible design, the content comparison module 702 is further configured to: when the content of the first physical page is different from the content of the physical pages in the second set, move the first physical page into the second set.

[0142] In a possible design, the scanning module 701 is specifically used to periodically scan the first physical page allocated to the server.

[0143] In one possible design, the memory pool also includes: a memory pool metadata area; the memory pool metadata area includes multiple memory pool metadata, which is used to record the status information of the physical pages of each server and the correspondence between the physical pages and the servers. One memory pool metadata corresponds to one physical page.

[0144] It should be noted that the information interaction, execution process, etc. between the modules / units in the server 700 are based on the same concept as the above-mentioned method embodiments in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0145] The present application also provides a cluster, see Figure 8 , Figure 8 A schematic diagram of a cluster provided in an embodiment of the present application, the cluster may include n Figure 7 The server described in the corresponding embodiment is used to implement Figure 7 The functions of the server 700 in the corresponding embodiment.

[0146] It should be noted that the information interaction, execution process, etc. between the servers in cluster 800 are based on the same concept as the above-mentioned method embodiments in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0147] Next, another server provided by the embodiment of the present application is introduced. Figure 9 , Figure 9 A schematic diagram of the structure of a server provided in an embodiment of the present application, wherein the server 900 may be deployed with Figure 7 The server 700 described in the corresponding embodiment is used to implement Figure 7In the corresponding embodiment, the functions of the server 700, specifically, the server 900 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 922 and memories 932, and one or more storage media 930 (e.g., one or more mass storage devices) storing application programs 942 or data 944. The memories 932 and storage media 930 may be temporary storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown), each of which may include a series of instruction operations on the computer device 900. Furthermore, the central processing unit 922 may be configured to communicate with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the computer device 900.

[0148] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems 941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0149] In the embodiment of the present application, the central processing unit 922 is used to execute Figure 3 、 Figure 4 or Figure 5 The steps performed by the first server in the corresponding embodiment. For example, the central processor 922 can be used to: scan the first physical page assigned to the first server, and when the status information of the first physical page is in the first state, compare the content of the first physical page with the physical pages belonging to the first set in the memory pool, where the memory pool is a collection of memories of each server in the cluster, the status information is stored in the memory pool, and one physical page corresponds to one status information; when the content of the first physical page is the same as the content of the first target physical page, transfer the mapping of the first physical page to the first target physical page, and release the first physical page, where the first target physical page is a physical page in the first set.

[0150] It should be noted that the specific manner in which the CPU 922 performs the above steps is the same as that in the present application. Figure 3 、 Figure 4 or Figure 5 The corresponding method embodiments are based on the same concept, and the technical effects they bring are also the same as the above-mentioned embodiments of this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0151] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0152] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by means of dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0153] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0154] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A data processing method, characterized in that: Applied to a cluster, the cluster includes n servers, n ≥ 2, the method includes: Scanning a first physical page assigned to a first server, the first server belonging to the cluster; When the state information of the first physical page is in a first state, compare the first physical page with physical pages in a first set in a memory pool, where the memory pool is a set of memories of servers in the cluster, and the state information is stored in the memory pool, with one physical page corresponding to one piece of state information; When the content of the first physical page is identical to the content of a first target physical page, the mapping of the first physical page is transferred to the first target physical page, and the first physical page is released. The first target physical page is a physical page in the first set.

2. The method according to claim 1, characterized in that After transferring the mapping of the first physical page to the first target physical page, the method further includes: Based on the mapping of the first physical page, a reverse mapping of the first target physical page is updated.

3. The method according to any one of claims 1 to 2, characterized in that When the state information of the first physical page is in the first state, comparing the first physical page with the physical pages in the memory pool that belong to the first set includes: When the state information of the first physical page is in a first state, determining whether the first physical page is marked with a target identifier by a second server, the second server belonging to the cluster; If so, the first physical page is compared with the physical pages in the memory pool that belong to the first set.

4. The method according to any one of claims 1 to 3, characterized in that When the state information of the first physical page is in the first state, before comparing the content of the first physical page with the physical pages in the memory pool that belong to the first set, the method further includes: Determining that the first physical page is in a second state and that content of the first physical page has not changed within a preset time period; The state information of the first physical page is updated from the second state to the first state.

5. The method according to any one of claims 1 to 3, characterized in that When the state information of the first physical page is in the first state, before comparing the content of the first physical page with the physical pages in the memory pool that belong to the first set, the method further includes: determining that the first physical page is in a first state and content of the first physical page has changed; The state information of the first physical page is updated from the first state to a second state, and the first physical page is removed from the second set in the memory pool.

6. The method according to any one of claims 1 to 5, characterized in that After comparing the first physical page with the physical pages in the memory pool that belong to the first set, the method further includes: When the content of the first physical page is different from the content of the physical pages in the first set, comparing the content of the first physical page with the physical pages in the second set in the memory pool; When the content of the first physical page is identical to that of the second target physical page, the first physical page is added to the first set, and the state information of the first physical page is updated from the first state to the third state. The second target physical page is a physical page in the second set.

7. The method according to claim 6, characterized in that The method further comprises: When the content of the first physical page is identical to the content of the second target physical page, the mapping of the first physical page is changed to read-only.

8. The method according to any one of claims 6 to 7, characterized in that The method further comprises: When the content of the first physical page is identical to the content of the second target physical page, a target identifier is marked on the second target physical page.

9. The method according to any one of claims 6 to 8, characterized in that After comparing the first physical page with the physical pages in the memory pool that belong to the second set, the method further includes: If the content of the first physical page is different from the content of the physical pages in the second set, the first physical page is moved into the second set.

10. The method according to any one of claims 1 to 9, characterized in that Scanning the first physical page assigned to the first server includes: The first physical page allocated to the first server is scanned periodically.

11. The method according to any one of claims 1 to 10, characterized in that The memory pool also includes: Memory pool metadata area; The memory pool metadata area includes a plurality of memory pool metadata, which is used to record the status information of the physical pages of each server and the corresponding relationship between the physical pages and the servers. One memory pool metadata corresponds to one physical page.

12. A server, characterized in that: include: A scanning module, configured to scan a first physical page allocated to the server, wherein the server belongs to a cluster, and the cluster includes n servers, where n≥2; a content comparison module, configured to, when the state information of the first physical page is in a first state, compare the content of the first physical page with physical pages in a first set in a memory pool, the memory pool being a collection of memories of each server in the cluster, the state information being stored in the memory pool, one physical page corresponding to one piece of state information; The release module is used to transfer the mapping of the first physical page to the first target physical page and release the first physical page when the content of the first physical page is the same as the content of the first target physical page. The first target physical page is a physical page in the first set.

13. The server according to claim 12, wherein: The release module is further used for: After the mapping of the first physical page is transferred to the first target physical page, a reverse mapping of the first target physical page is updated based on the mapping of the first physical page.

14. The server according to any one of claims 12 to 13, characterized in that: The content comparison module is specifically used to: When the state information of the first physical page is in the first state, determining whether the first physical page is marked with a target identifier by other servers in the cluster; If so, the first physical page is compared with the physical pages in the memory pool that belong to the first set.

15. The server according to any one of claims 12 to 14, characterized in that: The content comparison module is further used to: Before comparing the content of the first physical page with the physical pages in the first set in the memory pool, determining that the first physical page is in the second state and that the content of the first physical page has not changed within a preset time period; The state information of the first physical page is updated from the second state to the first state.

16. The server according to any one of claims 12 to 14, characterized in that: The content comparison module is further used to: Before comparing the content of the first physical page with the physical pages in the first set in the memory pool, determining that the first physical page is in a first state and that the content of the first physical page has changed; The state information of the first physical page is updated from the first state to a second state, and the first physical page is removed from the second set in the memory pool.

17. The server according to any one of claims 12 to 16, characterized in that: The content comparison module is further used to: When the content of the first physical page is different from the content of the physical pages in the first set, comparing the content of the first physical page with the physical pages in the second set in the memory pool; When the content of the first physical page is identical to that of the second target physical page, the first physical page is added to the first set, and the state information of the first physical page is updated from the first state to the third state. The second target physical page is a physical page in the second set.

18. The server according to claim 17, wherein: The content comparison module is further used to: When the content of the first physical page is identical to the content of the second target physical page, the mapping of the first physical page is changed to read-only.

19. The server according to any one of claims 17-18, characterized in that The content comparison module is further used to: When the content of the first physical page is identical to the content of the second target physical page, a target identifier is marked on the second target physical page.

20. The server according to any one of claims 17 to 19, characterized in that: The content comparison module is further used to: If the content of the first physical page is different from the content of the physical pages in the second set, the first physical page is moved into the second set.

21. The server according to any one of claims 12 to 20, characterized in that: The scanning module is specifically used for: The first physical page allocated to the server is periodically scanned.

22. The server according to any one of claims 12 to 21, characterized in that: The memory pool also includes: Memory pool metadata area; The memory pool metadata area includes a plurality of memory pool metadata, which is used to record the status information of the physical pages of each server and the corresponding relationship between the physical pages and the servers. One memory pool metadata corresponds to one physical page.

23. A cluster, characterized in that: The cluster includes n servers, where n≥2, and the servers are the servers described in any one of claims 12-22.

24. A server, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store computer-readable instructions, and the processor is used to read the computer-readable instructions and implement the method according to any one of claims 1 to 11.

25. A computer storage medium, characterized in that Computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

26. A computer program product, characterized in that The computer program product includes computer-readable instructions, and when the computer-readable instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

27. A chip comprising a processor and a data interface, characterized in that: The processor reads instructions stored in the memory through the data interface and executes the method according to any one of claims 1 to 11.

Images