Memory sharing method and apparatus

Abstract

A memory sharing method. The method comprises: a first process acquires index data of an object to be accessed and a plurality of first metadata nodes, wherein each first metadata node comprises a plurality of addresses, each address in a first metadata node at a higher level is used for indexing to one first metadata node at an adjacent lower level, the index data comprises a plurality of sub-indexes, each sub-index corresponds to a first metadata node at one level, and each address is obtained by performing address offsetting on a start address of the first metadata node by a corresponding sub-index; and on the basis of the index data and the plurality of first metadata nodes, the first process indexes, level by level, from the first metadata node at the highest level among the plurality of first metadata nodes to a target metadata node among the plurality of first metadata nodes at the lowest level. The present application can achieve cross-address space sharing.

Description

A memory sharing method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202510489967.9, filed on April 17, 2025, entitled "A Memory Sharing Method and Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of communications, and more particularly to a memory sharing method and apparatus. Background Technology

[0003] In modern computer systems, different processes need to communicate or share data. Traditional inter-process object sharing techniques typically use shared memory. With the evolution of computer architecture towards multi-core processors and distributed systems, multiple processes often need to access the same shared memory, and the shared memory allocated to processes is also dynamically changing.

[0004] Especially when facing applications that require high real-time performance, such as database systems, cloud computing platforms, and distributed computing frameworks, the shared memory mechanisms in existing technologies lack flexible space management capabilities. When expanding or shrinking shared memory, they cannot dynamically expand or shrink memory space according to the actual situation, resulting in resource waste. Summary of the Invention

[0005] This application provides a memory sharing method that enables cross-address space sharing. Furthermore, when expanding and shrinking the shared memory space, it can achieve fine-grained, small-scale, and multiple expansions and shrinkages, resulting in strong scalability.

[0006] In a first aspect, this application provides a memory sharing method, the method comprising: a first process acquiring index data of an object to be accessed and a plurality of first metadata nodes; wherein each of the first metadata nodes contains a plurality of addresses, each address in a higher-level first metadata node being used to index to an adjacent next-level first metadata node; the index data including a plurality of sub-indexes, each sub-index corresponding to a level of first metadata node, each sub-index being used as an address offset from the starting address contained in the corresponding level of first metadata node to determine an address in the corresponding level of first metadata node; the first process, based on the index data and the plurality of first metadata nodes, indexing level by level from the highest-level first metadata node in the plurality of first metadata nodes to a target metadata node in the lowest-level plurality of first metadata nodes; the first process accessing data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0007] Index data can be used by different processes when indexing shared memory addresses. Since different processes have different address spaces, cross-address space sharing is required. Therefore, pointers cannot be used between upper and lower level metadata, because a pointer in a process's address space describes the virtual address of a physical address in its virtual space. The virtual address of the same physical memory is different in different address spaces, so this value is meaningless to other processes. In this embodiment, a sub-index from index data that is universally applicable across address spaces is used as an offset value, serving as a "pointer" in shared memory, thereby achieving cross-address space sharing.

[0008] In this embodiment, multiple first metadata nodes adopt a tree structure, using "index" instead of "pointer" to achieve cross-address space sharing. In addition, when expanding and shrinking the shared memory space, fine-grained expansion and shrinking in small amounts and multiple times can be achieved, resulting in strong scalability.

[0009] In one possible implementation, the physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

[0010] In one possible implementation, the index data further includes fast addressing information indicating the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the method further includes:

[0011] The first process accesses the data of the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0012] In one possible implementation, the method further includes: when the first process needs new shared memory, it sends a memory allocation request to the third process; the third process allocates newly added index data and newly added first metadata node to multiple processes, including the first process; the newly added index data and the newly added first metadata node are used to index the newly allocated shared memory, wherein the multiple processes all have permission to access the newly allocated shared memory.

[0013] In one possible implementation, when the first process needs new shared memory, it sends a memory allocation request to the third process, including: when the first process determines in the free list that there are insufficient object slots in an allocable state, it sends a memory allocation request to the third process; wherein, the object slot is the smallest allocation granularity of shared memory; and the free list contains the allocable state of each object slot.

[0014] In one possible implementation, the first process accesses data on the object to be accessed based on the address contained in the target metadata node, including: the first process performing a read operation on the object to be accessed based on the address contained in the target metadata node; and reading a first version number identifier included in the space of the object to be accessed; the method further includes: after the read operation, reading a second version number identifier included in the space of the object to be accessed; and determining whether the read operation was successful based on whether the first version number identifier and the second version number identifier are the same.

[0015] Secondly, this application provides a memory sharing device, the device comprising:

[0016] The first process acquisition module is used to acquire the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, and each sub-index is used as the starting address contained in the first metadata node at the corresponding level for address offset, so as to determine an address in the first metadata node at the corresponding level.

[0017] The indexing module of the first process is used to index from the highest-level first metadata node among the multiple first metadata nodes to the target metadata node among the lowest-level multiple first metadata nodes, based on the index data and the multiple first metadata nodes.

[0018] The access module of the first process is used to access the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0019] In one possible implementation, the physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

[0020] In one possible implementation, the index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the access module of the first process is further configured to perform data access on the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0021] In one possible implementation,

[0022] The transceiver module of the first process is used to send a memory allocation request to the third process when new shared memory is needed;

[0023] The device also includes:

[0024] The allocation module of the third process is used to allocate newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all multiple processes have permission to access the newly allocated shared memory.

[0025] In one possible implementation, the transceiver module of the first process is used for:

[0026] When there are insufficient object slots in the free list to be allocated, a memory allocation request is sent to a third process; where the object slot is the smallest allocation granularity of shared memory; the free list contains the allocation status of each object slot.

[0027] In one possible implementation, the access module of the first process is used for:

[0028] The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed;

[0029] After the read operation, the second version number identifier included in the space of the object to be accessed is read;

[0030] The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

[0031] Thirdly, this application provides a data processing device, including: a processor, a memory, and a bus, wherein: the processor and the memory are connected via the bus;

[0032] This memory is used to store computer programs or instructions;

[0033] The processor is configured to call or execute programs or instructions stored in the memory to implement the steps described in the first aspect and any possible implementation of the first aspect.

[0034] Fourthly, this application provides a computer storage medium including computer instructions that, when executed on an electronic device or server, perform the steps described in the first aspect and any possible implementation thereof.

[0035] Fifthly, this application provides a computer program product that, when run on an electronic device or server, performs the steps described in the first aspect and any possible implementation thereof.

[0036] Sixthly, this application provides a chip system including a processor for supporting devices in implementing the functions involved in the foregoing aspects, such as transmitting or processing data or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for memory-sharing devices. The chip system may be composed of chips or may include chips and other discrete devices.

[0037] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0038] Figure 1 is a schematic diagram of the application system framework according to an embodiment of this application;

[0039] Figure 2 is a flowchart illustrating a memory sharing method provided in an embodiment of this application;

[0040] Figures 3 to 6b are schematic diagrams of the application system framework of the embodiments of this application;

[0041] Figure 7 is a schematic diagram of a memory sharing device provided in an embodiment of this application;

[0042] Figure 8 is a schematic diagram of an architecture provided in an embodiment of this application;

[0043] Figure 9 is a schematic diagram of an architecture provided in an embodiment of this application. Detailed Implementation

[0044] The embodiments of this application are described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. As those skilled in the art will understand, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0045] The memory sharing method provided in this application is applied to a computer system, which can be a server, terminal device, virtual machine (VM), or container. The architecture of this computer system can be understood with reference to Figure 1. Figure 1 is a schematic diagram of the computer system architecture.

[0046] A computing device, also known as a computer system, can logically be divided into a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as processing units, memory, and memory control units, the functions and structure of which will be described in detail later. The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software. Furthermore, in this application embodiment, the computer system can be a handheld device such as a smartphone, or a terminal device such as a personal computer. This application does not particularly limit the type of system, as long as it can read and run the program code containing the memory sharing method of this application embodiment to collect sample processes according to the memory sharing method of this application embodiment. The executing entity of the memory sharing method in this application embodiment can be a computer system, or a functional module within a computer system capable of calling and executing programs.

[0047] In this application, a program or program code refers to a set of ordered instructions (or code) used to implement a relatively independent function. A process is a single execution of a program and its data on a computer device. Programs typically employ modular design, which breaks down the program's functionality into multiple smaller functional modules. A program contains at least one function, which is a code segment that implements a functional module. Therefore, a function is the basic unit of program functional modularization and can also be considered a subroutine.

[0048] Figure 1 is a schematic diagram of the architecture of a computing device 100 provided in an embodiment of this application. The computing device shown in Figure 1 is used to execute a memory sharing method. The computing device 100 may include at least one processor 110 and memory 120.

[0049] Optionally, the computing device 100 may further include a system bus, wherein the processor 110 and the memory 120 are respectively connected to the system bus. The processor 110 can access the memory 120 through the system bus; for example, the processor 110 can perform data reading and writing or code execution in the memory 120 through the system bus.

[0050] The processor 110 primarily interprets the instructions (or code) of a computer program and processes data within the computer software. These instructions and data can be stored in memory 120 or cache 116.

[0051] In this embodiment, the processor 110 may be an integrated circuit chip with signal processing capabilities. By way of example and not limitation, the processor 110 may be a general-purpose processor, a system-on-a-chip (SOC), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any combination of the above devices. The general-purpose processor may be a microprocessor, etc. Each processor 110 includes a memory control unit 114 and at least one processing unit 112. The processing unit 112, also referred to as a core, processing core, or central processing unit (CPU), is the most important component of the processor. The processing unit 112 may be manufactured from single-crystal silicon using a specific manufacturing process. All calculations, command reception, command storage, and data processing are performed by the core. Processing units can independently execute program instructions, utilizing parallel computing capabilities to accelerate program execution. Various processing units have fixed logical structures; for example, a processing unit includes execution units, instruction-level units, and bus interfaces.

[0052] The memory control unit 114 is used to control the data interaction between the memory 120 and the processing unit 112. Specifically, the memory control unit 114 can receive memory access requests from the processing unit 112 and control access to the memory based on the memory access requests. By way of example and not limitation, in the embodiments of this application, the memory control unit may be a device such as a memory management unit (MMU).

[0053] In this embodiment, each memory control unit 114 can address the memory 120 via the system bus. Furthermore, an arbitrator (not shown) can be configured on the system bus to handle and coordinate contention for access by multiple processing units 112.

[0054] In this embodiment, the processing unit 112 and the memory control unit 114 can be connected via internal chip lines, such as address lines, to enable communication between the processing unit 112 and the memory control unit 114.

[0055] Optionally, each processor 110 may also include a cache 116, where the cache is a buffer for data exchange (called a cache). When the processing unit 112 needs to read data, it first looks for the required data in the cache. If the data is found, it is executed directly; otherwise, it looks for the data in memory. Since the cache operates much faster than memory, the purpose of the cache is to help the processing unit 112 run faster.

[0056] Memory 120 provides running space for processes in computing device 100. For example, memory 120 can store computer programs (specifically, program code) used to create processes. Memory, also known as internal memory, is used to temporarily store computational data from processor 110, as well as data exchanged with external storage devices such as hard disks. As long as the computer is running, processor 110 will load the data that needs to be processed into memory for computation, and after the computation is completed, processing unit 112 will then send the result back out.

[0057] By way of example and not limitation, in this embodiment, memory 120 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory 120 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0058] It should be understood that the structure of the computing device 100 listed above is only an illustrative example and this application is not limited thereto. The computing device 100 in the embodiments of this application may include various hardware in the computer system in the prior art. For example, the computing device 100 may also include other storage besides memory 120, such as disk storage.

[0059] To facilitate understanding of the embodiments of this application, some basic concepts involved in this application will be briefly explained.

[0060] 1. Virtual Address: A virtual address is a specific address in the address space that the operating system can recognize or generate. Its size range is determined by the bitness of the operating system running on the processor. For example, if the operating system running on the processor is 32-bit, the virtual address is also 32-bit, and its virtual address range is 0-0xFFFFFFFF (4GB); if the operating system running on the processor is 64-bit, the virtual address is also 64-bit, and its address space is 0-0xFFFFFFFFFFFFFFFF (16EB).

[0061] Virtual addresses can be divided into multiple virtual address spaces according to actual needs, such as user-mode address space and kernel-mode address space. User-mode address space can be accessed by user-mode programs (such as reading, writing, opening, closing, or drawing) and kernel-mode programs (such as process management, memory management, file management, or device management), while kernel-mode address space can only be accessed by kernel-mode programs at runtime.

[0062] 2. Physical address: This can be a specific address within the address space of a hardware storage device such as internal memory.

[0063] 3. Virtual Page: Also known as a page. The MMU can use a paging mechanism to manage the virtual address space in units of virtual pages. Each page can include a virtual address space of a preset size. A virtual address space can include more than one virtual address, and one virtual address space can correspond to a set of page tables. This set of page tables is used to determine the physical address corresponding to the virtual address in the virtual address space. The set of page tables can include a first-level page table, or it can include a first-level page table and at least one second-level page table. The second-level page table can include second-level page tables, third-level page tables, or even lower-level page tables.

[0064] 4. Physical Pages: In the Linux kernel, the primary function of the MMU is to map "virtual addresses" to real "physical addresses." Specifically, the MMU uses physical pages as the basic unit of memory management. Different architectures support different physical page sizes. For example, in a 32-bit architecture, the supported physical page size is 4 kilobytes (KB), in a 64-bit architecture, it is 8 KB, and in a 128-bit architecture, it is 16 KB, and so on.

[0065] Physical page partitioning usually follows certain requirements. A 4KB page is typically called a regular page, while pages larger than 4KB are typically called "large pages." For example, a 2-megabyte (MB) page and a 1-gigabyte (GB) page are both called large pages. Of course, large pages can have other sizes, but they are usually multiples of 4KB.

[0066] 5. User mode: refers to the non-privileged state, which is the execution environment of a normal process. It cannot perform hardware-defined privileged operations, such as directly manipulating page table registers.

[0067] 6. Kernel mode: refers to privileged mode, which is the execution state of the OS kernel and services, and can perform privileged operations.

[0068] 7. User Space: Refers to the VA range that processes running in user space can access; it is part of the Visual Studio.

[0069] 8. Kernel Space: Refers to the VA range that can only be accessed by processes in kernel mode, and is part of VS.

[0070] 9. Inter-process communication: refers to the means of exchanging data between processes, including signals, message queues, pipes, shared memory, etc.

[0071] In addition, the following points are provided to facilitate understanding of the embodiments of this application.

[0072] First, in this application, "at least one" refers to one or more, and "more than one" refers to two or more. Furthermore, in the embodiments of this application, "first," "second," and various numerical designations (e.g., "#1," "#2," etc.) are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The sequence numbers of the processes below do not imply an order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. It should be understood that the objects described in this way can be interchanged where appropriate to describe solutions other than those in the embodiments of this application. In addition, in the embodiments of this application, the term "201," etc., is merely an identifier for descriptive convenience and does not limit the order of execution steps.

[0073] Second, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0074] Third, the term "storage" in the embodiments of this application can refer to storage in one or more memories. These memories can be separate installations or integrated into an encoder, decoder, processor, or communication device. Alternatively, some memories can be separately installed, while others are integrated into the decoder, processor, or communication device. The type of memory can be any form of storage medium, and this application does not limit this.

[0075] Fourth, the term "comprising" (also referred to as "includes", "including", "comprises" and / or "comprising") used in the embodiments of this application, when used in this specification, specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0076] Fifth, the word "if" in the embodiments of this application can be interpreted as meaning "when" or "upon" or "in response to determination" or "in response to detection". Similarly, depending on the context, the phrase "if it is determined..." or "if [the stated condition or event] is detected" can be interpreted as meaning "when it is determined..." or "in response to determination..." or "when [the stated condition or event] is detected" or "in response to detection of [the stated condition or event]".

[0077] Sixth, the terminology used in the description of the various examples in the embodiments of this application is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and in the appended claims, the numerical forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0078] Seventh, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0079] In modern computer systems, different processes need to communicate or share data. Traditional inter-process object sharing techniques typically use shared memory. With the evolution of computer architecture towards multi-core processors and distributed systems, multiple processes often need to access the same shared memory, and the shared memory allocated to processes is also dynamically changing.

[0080] Especially when facing applications that require high real-time performance, such as database systems, cloud computing platforms, and distributed computing frameworks, the shared memory mechanisms in existing technologies lack flexible space management capabilities. When expanding or shrinking shared memory, they cannot dynamically expand or shrink memory space according to the actual situation, resulting in resource waste.

[0081] The memory sharing method provided in this application will be described in detail below with reference to the accompanying drawings.

[0082] It should be understood that the memory sharing method provided in the embodiments of this application can be applied to computer systems, such as the system shown in Figure 1.

[0083] It should also be understood that the embodiments shown below do not particularly limit the specific structure of the execution subject of the method provided in the embodiments of this application. As long as the method provided in the embodiments of this application can be implemented by running a program that records the code of the method provided in the embodiments of this application. For example, the execution subject of the method provided in the embodiments of this application can be a device, or a functional module in the device that can call and execute a program.

[0084] Figure 2 is a schematic flowchart of a memory sharing method provided in this application. The memory sharing method includes the following steps:

[0085] 201. The first process obtains the index data of the object to be accessed and multiple first metadata nodes; wherein, each of the first metadata nodes contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, and each sub-index is used as the starting address contained in the first metadata node at the corresponding level for address offset, so as to determine an address in the first metadata node at the corresponding level;

[0086] 202. The first process indexes from the highest level first metadata node to the target metadata node in the lowest level of the multiple first metadata nodes, based on the index data and the multiple first metadata nodes.

[0087] In this embodiment, the first process can be the owner (or holder) of a shared object (that is, a block of shared memory, which refers to the same memory area that multiple processes can access through shared memory), and has the authority to allocate, release, read and write the shared object.

[0088] In this embodiment, the second process can be a renter of the shared object, with read-only permissions for the shared object.

[0089] In this application embodiment, the third process can be a memory management process in the kernel, which can be called a mapper. A mapper is a process that has the authority to map physical memory to other processes. Specifically, it has the authority to allocate and release memory in the address space of other processes.

[0090] It should be understood that in the embodiments of this application, the specific roles of the first process, the second process and the third process can correspond one-to-one with specific shared objects, and the roles of different processes can be different when processing different shared objects.

[0091] In this embodiment, the shared object of the first process (and the second process) may be allocated by the third process. Specifically, the third process may allocate information used to index the address of the shared object to the first process (and the second process). In this embodiment, the third process may allocate multiple first metadata nodes and index data as information to index the address of the shared object to the first process.

[0092] The following section will first introduce the process of applying for the identity of a renter for a shared object in the first process:

[0093] In one possible implementation, referring to Figure 4, the first process can register the user identity of a specific shared object (e.g., holder or lessee). Specifically, the mapper can first determine the permissions of each shared object (e.g., determine the first process (the holder in Figure 4 has the permissions to allocate, release, and read / write, while the lessee has the permission to read)). The first process can notify the third process to apply to register as the holder of a certain shared object. After the third process authenticates the first process, it allocates metadata for it in the address space of the third process and maps it to the address spaces of those who have access to the shared memory (e.g., holder and lessee) in the form of shared memory. The specific process can be shown in Figure 4.

[0094] In one possible implementation, when the holder initially allocates a shared object, it needs to initiate a memory allocation request to the mapper. The mapper allocates corresponding metadata pages and data pages (i.e., shared memory) for itself and all other user processes, and establishes a sharing relationship. Referring to Figure 3, to achieve flexible scaling of the shared object allocator, this embodiment can use a tree-like data structure (also called a radix tree, such as multiple first metadata nodes in this embodiment) to implement the indexing of shared memory. Based on this structure, the allocation and reclamation of shared memory can also be realized. Therefore, the metadata includes the intermediate nodes and data nodes of the radix tree, and the physical page where the metadata is located is shared by the mapper, holder, and lessee. The page where the data is located is shared by the holder and lessee.

[0095] Specifically, the third process can allocate index data and multiple first metadata nodes to the first process. The third process can also allocate index data (the same as that allocated by the first process) and multiple second metadata nodes to the second process. The first process can index the shared memory address based on the index data and multiple first metadata nodes, and the second process can index the shared memory address based on the index data and multiple second metadata nodes. In other words, when the third process allocates memory, it can allocate the same index data to different processes.

[0096] The multiple first metadata nodes can be hierarchical, with each level including at least one first metadata node. For example, the multiple first metadata nodes can be a tree structure (e.g., a radix tree), where each root node can index multiple leaf nodes. For instance, the highest-level first metadata node is the root node, and based on the highest-level first metadata node, multiple leaf nodes (i.e., multiple first metadata nodes of multiple adjacent next-level nodes) can be indexed. Similarly, each first metadata node of the next level (compared to the highest-level first metadata node) can also be indexed to multiple first metadata nodes of multiple adjacent next-level nodes, until the lowest-level first metadata node is indexed. The lowest-level first metadata node can indicate the address of the memory space (e.g., the starting virtual address), based on which shared memory can be accessed.

[0097] For example, referring to Figure 6a, the multiple first metadata nodes in Figure 6a can be Ln Node, L1 Node, and so on up to DNode. Among them, Ln Node is the highest-level first metadata node, L1 Node is the first metadata node of the next level adjacent to the highest level, and DNode is the lowest-level first metadata node. Based on Ln Node, L1 Node can be indexed, and the indexing continues level by level until DNode. DNode can contain the address of memory space, which points to shared memory (such as the data chunk in Figure 6a). Shared memory can be divided into multiple memory units (i.e., object slots). The address of the memory space contained in DNode can point to the starting address of an object slot in shared memory.

[0098] Specifically, the indexing process described above can be performed using index data. The index data includes multiple sub-indexes, each corresponding to a level of the first metadata node. Each first metadata node contains multiple addresses, with each address in a higher-level first metadata node used to index a first metadata node in the next adjacent level. During indexing, the sub-index can serve as the address offset of the starting address of the corresponding level's first metadata node. The starting address of each first metadata node can be offset by the address of the corresponding sub-index, and the offset address is an address of the first metadata node. Based on this address, a first metadata node in the next adjacent level can be indexed.

[0099] For example, referring to Figure 6a, the index data (specifically, the slow path region in the index data) can contain multiple sub-indexes. Sub-index 1 corresponds to an Ln Node, sub-index 2 corresponds to an L1 Node, and during indexing, sub-index 1 can be used as the address offset of the starting address of the Ln Node. The offset address (belonging to the Ln Node) can point to a certain L1 Node. Sub-index 2 can be used as the address offset of the starting address of the L1 Node. The offset address (belonging to the L1 Node) can point to a certain L2 Node, and so on, until a certain Dnode is indexed. The address of the Dnode (uva in Figure 6a) can point to an object slot.

[0100] Index data can be used by different processes when indexing shared memory addresses. Since different processes have different address spaces, cross-address space sharing is required. Therefore, pointers cannot be used between upper and lower level metadata, because a pointer in a process's address space describes the virtual address of a physical address in its virtual space. The virtual address of the same physical memory is different in different address spaces, so this value is meaningless to other processes. In this embodiment, a sub-index from index data that is universally applicable across address spaces is used as an offset value, serving as a "pointer" in shared memory, thereby achieving cross-address space sharing.

[0101] In this embodiment, multiple first metadata nodes adopt a tree structure, using "index" instead of "pointer" to achieve cross-address space sharing. In addition, when expanding and shrinking the shared memory space, fine-grained expansion and shrinking can be achieved in small amounts and multiple times, with strong scalability and less memory resource waste. The capacity of the shared memory space can be expanded and shrunk in real time as needed, without the need to pre-allocate large blocks of memory, and can share composite data structures across address spaces.

[0102] In one possible implementation, when expanding memory, for example when the first process needs new memory space, the first process can send a memory allocation request to the third process. Then, the third process can determine the new memory space (memory page) and expand the structure of the radix tree (that is, multiple first metadata nodes) as needed (for example, on the basis of the original multiple first metadata nodes, add new first metadata nodes and corresponding index data, and the newly added first metadata nodes and corresponding index data can index the address of the new memory space). The third node can allocate the newly added information to each process (processes with permission to access the newly allocated shared memory).

[0103] Specifically, when the first process needs new shared memory, it sends a memory allocation request to the third process; the third process allocates newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all multiple processes have permission to access the newly allocated shared memory.

[0104] Next, we will explain under what circumstances the first process needs to request new memory space from the third process.

[0105] In one possible implementation, the first process can determine the need for new shared memory when it finds that there are insufficient allocatable object slots in the free list, where the object slots represent the smallest allocation granularity of the shared memory; the free list contains the allocatable status of each object slot.

[0106] The free list records the allocatability status of each object slot. Allocatability can be understood as whether an object can be allocated on a data page within the address space. If a data page in that address space already holds an object, it is in an unallocable state; otherwise, it is in an allocatable state. Since the shared space allocated to a process may not be contiguous, meaning the object slots are not contiguous, the free list can also contain indices of adjacent object slots to already occupied slots.

[0107] Specifically, when a holder allocates an object, it first checks if there are any available objects in the corresponding size's object slot in the free list of the local address space. If not, the mapper allocates the necessary shared metadata nodes (LnNode, Dnode) and shared data chunks to all holders and lessees, completing the growth of the radix tree. The data chunks are then split, with a small metadata area reserved before each chunk to store the metadata (length, allocation status) of all objects within that chunk. The data area of ​​the chunk is then divided into multiple object slots. These slots are implicitly attached to the free list, meaning the memory where the free object resides stores the indexes of its preceding and following objects on the free list, without requiring additional memory to maintain the free list. The overall process is illustrated in Figure 5.

[0108] After the shared memory allocator has expanded, it retrieves an available object from the corresponding slot in the free list and allocates it to the user. When there are no free objects of the available size, the larger object is split into smaller, allocatable objects. During subsequent object allocations, if there are allocatable objects in the free list, the holder automatically handles the allocation and metadata area updates, which the lessee can directly perceive through the shared memory. No additional inter-process communication is required during this process.

[0109] Simultaneously, the holder can also initiate the release of shared objects, a process that is the reverse of the object allocation process. The process can be illustrated in Figure 6b: when releasing a single object, the object is placed in the free list (free_list); when all objects in the entire data page are in the free_list, the page mapping of that data page is released from the address spaces of all holders / tenants.

[0110] When the holder releases an object, it directly adds the corresponding object to the free list, without requiring additional inter-process communication. When the mapper needs to reclaim memory, it scans each shared data block and metadata node. If there are subtrees in the radix tree that do not maintain any allocated objects, it unmaps these shared memory blocks in turn and finally reclaims the physical memory.

[0111] In one possible implementation, the index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; thus, the first process can access the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0112] Since address indexing based on sub-indexes and multiple first metadata nodes in the index data needs to be determined level by level, the time and computational costs are relatively high. Therefore, it can be called a slow path index. The advantage of slow paths is that there is no capacity limit. In this embodiment, fast path indexing can also be supported. Specifically, the index data also includes fast addressing information, which can indicate the address offset of the starting address of the first metadata node at the highest level of the address of the object to be accessed. For example, referring to Figure 4, the object index is divided into two parts: a fast path index segment and a slow path index segment. The fast path index segment stores the global offset of the fast path object relative to the root node of the first metadata node (that is, the first metadata node at the highest level). When accessing the object later, it is only necessary to add the global offset of the object to the starting address of the fast path to complete the indexing. The advantage of fast path indexing is that the time and computational costs are relatively low.

[0113] Optionally, the fast path size can be configured as needed. The relationship between the length x (in bits) of the fast path index segment and the size fsize (in bytes) of the fast path reserved mapping area is: fsize = 2 x .

[0114] The shared object allocator's object index supports both high-speed addressing (fast path) and unlimited capacity (slow path), and can be flexibly configured by the user. It can simultaneously address the diverse needs of various application scenarios regarding shared object usage, and the system has significant optimization potential. Fast and slow paths are flexibly configured as needed; the fast path object indexing speed is close to that of a single-address-space allocator, while the slow path has unlimited capacity. This allows for a balance between allocator performance and memory consumption in different scenarios.

[0115] Read and write operations are encapsulated by the shared object allocator, which provides a unified interface to the outside world. The addressing and read / write synchronization mechanisms inside the allocator are completely transparent to users. If the address spaces of holders and renters are merged in subsequent system version evolutions, there is no need to modify the code that calls the shared object read and write interface, thus achieving compatibility with system configuration.

[0116] Managed object synchronization mechanisms are supported throughout the object's lifecycle. The object operation interface supports both managed and self-managed modes. The interface remains consistent across all user process address spaces, whether merged or separate, eliminating the need for complex synchronization strategy modifications and enabling flexible configuration that allows for both separation and combination. It is user-friendly and flexible, requiring only a single address space perspective during development. No repetitive coding is needed in scenarios involving merged or separate address spaces for multiple processes / system components, further supporting flexible configuration that allows for both separation and combination.

[0117] 203. The first process accesses the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0118] In one possible implementation, the first process can perform a read operation on the object to be accessed based on the address contained in the target metadata node; and the space of the object to be accessed includes a version number identifier; the success of the read operation is determined by whether the version number identifier before the read operation and the version number identifier after the read operation are the same.

[0119] This application embodiment supports the managed object (coobj) pattern, where a version number can be configured in the memory control where each shared object (e.g., the object to be accessed) resides. When a process reads an object, it can read the object's version number before and after the read. If the version numbers match, the read is successful; otherwise, it retryes. The holder can atomically modify the version number before and after modifying the object's content.

[0120] The memory sharing apparatus provided in the embodiments of this application will now be described in detail with reference to Figures 7 to 9. It should be understood that the descriptions of the apparatus embodiments correspond to the descriptions of the method embodiments; therefore, any content not described in detail can be referred to the method embodiments above. For the sake of brevity, some content will not be repeated.

[0121] Figure 7 shows a schematic diagram of a memory sharing device 700 provided in an embodiment of this application. The memory sharing device 700 can be applied to a processing core running processes, and can be used to execute the aforementioned memory sharing method, such as the method shown in Figure 2. The memory sharing device 700 may include:

[0122] The first process acquisition module 701 is used to acquire the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, and each sub-index is used as the starting address contained in the first metadata node at the corresponding level for address offset, so as to determine an address in the first metadata node at the corresponding level.

[0123] The specific description of the acquisition module 701 can be found in the description of step 201 in the above embodiment, and the similarities will not be repeated here.

[0124] The index module 702 of the first process indexes from the highest-level first metadata node to the target metadata node among the lowest-level first metadata nodes based on the index data and the plurality of first metadata nodes.

[0125] The specific description of the index module 702 can be found in step 202 of the above embodiment, and the similarities will not be repeated here.

[0126] The access module 703 of the first process performs data access on the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0127] The specific description of the access module 703 can be found in step 203 of the above embodiment, and the similarities will not be repeated here.

[0128] In one possible implementation, the physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

[0129] In one possible implementation, the index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the access module of the first process is further configured to perform data access on the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0130] In one possible implementation,

[0131] The transceiver module of the first process is used to send a memory allocation request to the third process when new shared memory is needed;

[0132] The device also includes:

[0133] The allocation module of the third process is used to allocate newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all multiple processes have permission to access the newly allocated shared memory.

[0134] In one possible implementation, the transceiver module of the first process is used for:

[0135] When there are insufficient object slots in the free list to be allocated, a memory allocation request is sent to a third process; where the object slot is the smallest allocation granularity of shared memory; the free list contains the allocation status of each object slot.

[0136] In one possible implementation, the access module of the first process is used for:

[0137] The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed;

[0138] After the read operation, the second version number identifier included in the space of the object to be accessed is read;

[0139] The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

[0140] This application embodiment also provides a chip system 800, as shown in FIG8. The chip system 800 includes at least one processor and at least one interface circuit. As an example, when the chip system 800 includes a processor and an interface circuit, the processor may be the processor 810 shown in the solid box in FIG8 (or the processor 810 shown in the dashed box), and the interface circuit may be the interface circuit 820 shown in the solid box in FIG8 (or the interface circuit 820 shown in the dashed box).

[0141] When the chip system 800 includes two processors and two interface circuits, the two processors include processor 810 shown in the solid box and processor 810 shown in the dashed box in Figure 8, and the two interface circuits include interface circuit 820 shown in the solid box and interface circuit 820 shown in the dashed box in Figure 8. This is not limited. Processor 810 and interface circuit 820 can be interconnected via lines. For example, interface circuit 820 can be used to receive signals (e.g., instructions stored in memory). As another example, interface circuit 820 can be used to send signals to other devices (e.g., processor 810).

[0142] For example, interface circuit 820 can read instructions stored in memory and send those instructions to processor 810. When the instruction is executed by processor 810, it can cause a memory-sharing device or a device accessing memory to perform the steps in the above embodiments. Of course, the chip system 800 may also include other discrete devices, and this application embodiment does not specifically limit this.

[0143] Another embodiment of this application provides a computer-readable storage medium storing instructions that, when executed on a memory-sharing device, perform the various steps of the method flow shown in the above-described method embodiments. In some embodiments, the disclosed method can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0144] Figure 9 schematically illustrates a conceptual partial view of a computer program product provided in an embodiment of this application, the computer program product including a computer program for executing computer processes on a computing device.

[0145] In one embodiment, the computer program product is provided using a signal-bearing medium 900. This signal-bearing medium 900 may include one or more program instructions that, when executed by one or more processors, provide the functions or portions thereof described above with reference to FIG2. Thus, for example, one or more features 201-203 in FIG2 may be provided by one or more instructions associated with the signal-bearing medium 900. Furthermore, example instructions are also described in FIG9.

[0146] In some examples, the signal carrying medium 900 may include a computer-readable medium 901, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0147] In some implementations, the signal carrying medium 900 may include a computer recordable medium 902, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0148] In some implementations, the signal-bearing medium 900 may comprise a communication medium 903, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). The signal-bearing medium 900 may be transmitted by a wireless communication medium 903 (e.g., a wireless communication medium conforming to the IEEE 1502.11 standard or other transmission protocols). One or more program instructions may be, for example, computer-executable instructions or logic implementation instructions.

[0149] In some examples, such as the memory sharing method for Figure 2, it can be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 901, a computer-recordable medium 902, and / or a communication medium 903.

[0150] It should be understood that the arrangements described herein are for illustrative purposes only. Therefore, those skilled in the art will understand that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements may be omitted depending on the desired outcome. Furthermore, many of the described elements are functional entities that can be implemented as discrete or distributed components, or in any suitable combination and location with other components.

[0151] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When executed on a computer and when the computer execution instructions are executed, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0152] Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or it can contain one or more data storage devices such as servers or data centers that can be integrated with that medium. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A memory sharing method, characterized by, The method includes: The first process acquires the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, each sub-index corresponds to a first metadata node at a level, and each sub-index is used as an address offset of the starting address contained in the first metadata node at the corresponding level to determine an address in the first metadata node at the corresponding level; The first process indexes from the highest-level first metadata node to the target metadata node among the lowest-level first metadata nodes, based on the index data and the plurality of first metadata nodes. The first process accesses the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

2. The method of claim 1, wherein, The physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

3. The method according to claim 1 or 2, characterized in that, The index data further includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the method further includes: The first process accesses the data of the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: When the first process needs new shared memory, it sends a memory allocation request to the third process. The third process allocates newly added index data and newly added first metadata node to multiple processes, including the first process; the newly added index data and the newly added first metadata node are used to index the newly allocated shared memory, wherein all the multiple processes have permission to access the newly allocated shared memory.

5. The method of claim 4, wherein, When the first process needs new shared memory, it sends a memory allocation request to the third process, including: When the first process determines that there are insufficient allocatable object slots in the free list, it sends a memory allocation request to the third process; wherein, the object slot is the smallest allocation granularity of shared memory; and the free list contains the allocatable status of each object slot.

6. The method according to any one of claims 1 to 5, characterized in that, The first process accesses data from the object to be accessed based on the address contained in the target metadata node, including: The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed; the method further includes: After the read operation, the second version number identifier included in the space of the object to be accessed is read; The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

7. A memory sharing apparatus, comprising: The device includes: The first process acquisition module is used to acquire the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, each sub-index corresponds to a first metadata node at a level, and each sub-index is used as an address offset of the starting address contained in the first metadata node at the corresponding level to determine an address in the first metadata node at the corresponding level; The indexing module of the first process is used to index from the highest-level first metadata node among the multiple first metadata nodes to the target metadata node among the lowest-level multiple first metadata nodes, based on the index data and the multiple first metadata nodes. The access module of the first process is used to access the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

8. The apparatus of claim 7, wherein, The physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

9. The apparatus of claim 7 or 8, wherein, The index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the highest-level first metadata node; the access module of the first process is further configured to allow the first process to access the object to be accessed based on the starting address of the highest-level first metadata node and the fast addressing information.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The transceiver module of the first process is used to send a memory allocation request to the third process when new shared memory is needed; The device further includes: The allocation module of the third process is used to allocate newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all the multiple processes have permission to access the newly allocated shared memory.

11. The apparatus of claim 10, wherein, The transceiver module of the first process is used for: When there are insufficient object slots in the free list to be allocated, a memory allocation request is sent to a third process; wherein, the object slot is the smallest allocation granularity of shared memory; and the free list contains the allocation status of each object slot.

12. The apparatus of any one of claims 7 to 11, wherein, The access module of the first process is used for: The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed; After the read operation, the second version number identifier included in the space of the object to be accessed is read; The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

13. A computing device, comprising: Including processor and memory; The processor is configured to execute instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that, When the instruction is executed by the computing device, the computing device performs the method as described in any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 6.

16. A chip, characterized by It includes at least one processing unit and an interface circuit, the interface circuit being used to provide program instructions or data to the at least one processing unit, the at least one processing unit being used to execute the program instructions to implement the method of any one of claims 1 to 6.

Images