A memory sharing method and apparatus thereof

Abstract

A memory sharing method, the method comprising: a first process obtaining index data of an object to be accessed and a plurality of first metadata nodes; wherein each first metadata node contains a plurality of addresses, and each address in a first metadata node at a higher level is used to index to a first metadata node at a next lower level; the index data comprises a plurality of sub-indices, each sub-index corresponding to a first metadata node at a level, and each address being obtained by performing address offset on a start address of the first metadata node by a corresponding sub-index; and the first process indexes from a first metadata node at a highest level to a target metadata node at a lowest level among the plurality of first metadata nodes layer by layer according to the index data and the plurality of first metadata nodes. The application can realize cross-address space sharing.

Description

Technical Field

[0001] This application relates to the field of communications, and more particularly to a memory sharing method and apparatus thereof. Background Technology

[0002] In modern computer systems, different processes need to communicate or share data. Traditional inter-process object sharing techniques typically use shared memory. With the evolution of computer architecture towards multi-core processors and distributed systems, multiple processes often need to access the same shared memory, and the shared memory allocated to processes is also dynamically changing.

[0003] Especially when facing applications that require high real-time performance, such as database systems, cloud computing platforms, and distributed computing frameworks, the shared memory mechanisms in existing technologies lack flexible space management capabilities. When expanding or shrinking shared memory, they cannot dynamically expand or shrink memory space according to the actual situation, resulting in resource waste. Summary of the Invention

[0004] This application provides a memory sharing method that enables cross-address space sharing. Furthermore, when expanding and shrinking the shared memory space, it can achieve fine-grained, small-scale, and multiple expansions and shrinkages, resulting in strong scalability.

[0005] In a first aspect, this application provides a memory sharing method, the method comprising: a first process acquiring index data of an object to be accessed and a plurality of first metadata nodes; wherein each of the first metadata nodes contains a plurality of addresses, each address in a higher-level first metadata node being used to index to an adjacent next-level first metadata node; the index data including a plurality of sub-indexes, each sub-index corresponding to a level of first metadata node, each sub-index being used as an address offset from the starting address contained in the corresponding level of first metadata node to determine an address in the corresponding level of first metadata node; the first process, based on the index data and the plurality of first metadata nodes, indexing level by level from the highest-level first metadata node in the plurality of first metadata nodes to a target metadata node in the lowest-level plurality of first metadata nodes; the first process accessing data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0006] Index data can be used by different processes when indexing shared memory addresses. Since different processes have different address spaces, cross-address space sharing is required. Therefore, pointers cannot be used between upper and lower level metadata, because a pointer in a process's address space describes the virtual address of a physical address in its virtual space. The virtual address of the same physical memory is different in different address spaces, so this value is meaningless to other processes. In this embodiment, a sub-index from index data that is universally applicable across address spaces is used as an offset value, serving as a "pointer" in shared memory, thereby achieving cross-address space sharing.

[0007] In this embodiment, multiple first metadata nodes adopt a tree structure, using "index" instead of "pointer" to achieve cross-address space sharing. In addition, when expanding and shrinking the shared memory space, fine-grained expansion and shrinking in small amounts and multiple times can be achieved, resulting in strong scalability.

[0008] In one possible implementation, the physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

[0009] In one possible implementation, the index data further includes fast addressing information indicating the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the method further includes:

[0010] The first process accesses the data of the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0011] In one possible implementation, the method further includes: when the first process needs new shared memory, it sends a memory allocation request to the third process; the third process allocates newly added index data and newly added first metadata node to multiple processes, including the first process; the newly added index data and the newly added first metadata node are used to index the newly allocated shared memory, wherein the multiple processes all have permission to access the newly allocated shared memory.

[0012] In one possible implementation, when the first process needs new shared memory, it sends a memory allocation request to the third process, including: when the first process determines in the free list that there are insufficient object slots in an allocable state, it sends a memory allocation request to the third process; wherein, the object slot is the smallest allocation granularity of shared memory; and the free list contains the allocable state of each object slot.

[0013] In one possible implementation, the first process accesses data on the object to be accessed based on the address contained in the target metadata node, including: the first process performing a read operation on the object to be accessed based on the address contained in the target metadata node; and reading a first version number identifier included in the space of the object to be accessed; the method further includes: after the read operation, reading a second version number identifier included in the space of the object to be accessed; and determining whether the read operation was successful based on whether the first version number identifier and the second version number identifier are the same.

[0014] Secondly, this application provides a memory sharing device, the device comprising:

[0015] The first process acquisition module is used to acquire the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, and each sub-index is used as the starting address contained in the first metadata node at the corresponding level for address offset, so as to determine an address in the first metadata node at the corresponding level.

[0016] The indexing module of the first process is used to index from the highest-level first metadata node among the multiple first metadata nodes to the target metadata node among the lowest-level multiple first metadata nodes, based on the index data and the multiple first metadata nodes.

[0017] The access module of the first process is used to access the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0018] In one possible implementation, the physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

[0019] In one possible implementation, the index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the access module of the first process is further configured to perform data access on the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0020] In one possible implementation,

[0021] The transceiver module of the first process is used to send a memory allocation request to the third process when new shared memory is needed;

[0022] The device also includes:

[0023] The allocation module of the third process is used to allocate newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all multiple processes have permission to access the newly allocated shared memory.

[0024] In one possible implementation, the transceiver module of the first process is used for:

[0025] When there are insufficient object slots in the free list to be allocated, a memory allocation request is sent to a third process; where the object slot is the smallest allocation granularity of shared memory; the free list contains the allocation status of each object slot.

[0026] In one possible implementation, the access module of the first process is used for:

[0027] The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed;

[0028] After the read operation, the second version number identifier included in the space of the object to be accessed is read;

[0029] The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

[0030] Thirdly, this application provides a data processing device, including: a processor, a memory, and a bus, wherein: the processor and the memory are connected via the bus;

[0031] This memory is used to store computer programs or instructions;

[0032] The processor is configured to call or execute programs or instructions stored in the memory to implement the steps described in the first aspect and any possible implementation of the first aspect.

[0033] Fourthly, this application provides a computer storage medium including computer instructions that, when executed on an electronic device or server, perform the steps described in the first aspect and any possible implementation thereof.

[0034] Fifthly, this application provides a computer program product that, when run on an electronic device or server, performs the steps described in the first aspect and any possible implementation thereof.

[0035] Sixthly, this application provides a chip system including a processor for supporting devices in implementing the functions involved in the foregoing aspects, such as transmitting or processing data or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for memory-sharing devices. The chip system may be composed of chips or may include chips and other discrete devices.

[0036] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the application system framework in an embodiment of this application;

[0038] Figure 2 A flowchart illustrating a memory sharing method provided in an embodiment of this application;

[0039] Figures 3 to 6b This is a schematic diagram of the application system framework in an embodiment of this application;

[0040] Figure 7 A schematic diagram of a memory sharing device provided in an embodiment of this application;

[0041] Figure 8 A schematic diagram of the architecture provided in an embodiment of this application;

[0042] Figure 9 This is a schematic diagram of an architecture provided in an embodiment of this application. Detailed Implementation

[0043] The embodiments of this application are described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. As those skilled in the art will understand, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0044] The memory sharing method provided in this application is applied to a computer system, which can be a server, terminal device, virtual machine (VM), or container. The architecture of this computer system can be found in [reference needed]. Figure 1 To understand. Figure 1This is a schematic diagram of the computer system architecture.

[0045] A computing device, also known as a computer system, can logically be divided into a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as processing units, memory, and memory control units, the functions and structure of which will be described in detail later. The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software. Furthermore, in this application embodiment, the computer system can be a handheld device such as a smartphone, or a terminal device such as a personal computer. This application does not particularly limit the type of system, as long as it can read and run the program code containing the memory sharing method of this application embodiment to monitor the sample process according to the memory sharing method of this application embodiment. The executing entity of the memory sharing method in this application embodiment can be a computer system, or a functional module within a computer system capable of calling and executing programs.

[0046] In this application, a program or program code refers to a set of ordered instructions (or code) used to implement a relatively independent function. A process is a single execution of a program and its data on a computer device. Programs typically employ modular design, which breaks down the program's functionality into multiple smaller functional modules. A program contains at least one function, which is a code segment that implements a functional module. Therefore, a function is the basic unit of program functional modularization and can also be considered a subroutine.

[0047] Figure 1 This is a schematic diagram of the architecture of a computing device 100 provided in an embodiment of this application. Figure 1 The computing device shown is used to perform a memory-sharing method. The computing device 100 may include at least one processor 110 and memory 120.

[0048] Optionally, the computer device 110 may also include a system bus, wherein the processor 110 and the memory 120 are respectively connected to the system bus. The processor 110 can access the memory 120 through the system bus; for example, the processor 110 can perform data reading and writing or code execution in the memory 120 through the system bus.

[0049] The processor 110 primarily interprets the instructions (or code) of a computer program and processes data within the computer software. These instructions and data can be stored in memory 120 or cache 116.

[0050] In this embodiment, the processor 110 may be an integrated circuit chip with signal processing capabilities. By way of example and not limitation, the processor 110 may be a general-purpose processor, a system-on-a-chip (SOC), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any combination of the above devices. The general-purpose processor may be a microprocessor, etc. Each processor 110 includes a memory control unit 114 and at least one processing unit 112. The processing unit 112, also referred to as a core, kernel, processing core, or central processing unit (CPU), is the most important component of the processor. The processing unit 112 may be manufactured from single-crystal silicon using a specific manufacturing process. All calculations, command reception, command storage, and data processing are performed by the core. Processing units can independently execute program instructions, utilizing parallel computing capabilities to accelerate program execution. Various processing units have fixed logical structures; for example, a processing unit includes execution units, instruction-level units, and bus interfaces.

[0051] The memory control unit 114 is used to control the data interaction between the memory 120 and the processing unit 112. Specifically, the memory control unit 114 can receive memory access requests from the processing unit 112 and control access to the memory based on the memory access requests. By way of example and not limitation, in the embodiments of this application, the memory control unit may be a device such as a memory management unit (MMU).

[0052] In this embodiment, each memory control unit 114 can address the memory 120 via the system bus. Furthermore, an arbitrator (not shown) can be configured on the system bus to handle and coordinate contention for access by multiple processing units 112.

[0053] In this embodiment, the processing unit 112 and the memory control unit 114 can be connected via internal chip lines, such as address lines, to enable communication between the processing unit 112 and the memory control unit 114.

[0054] Optionally, each processor 110 may also include a cache 116, where the cache is a buffer for data exchange (called a cache). When the processing unit 112 needs to read data, it first looks for the required data in the cache. If the data is found, it is executed directly; otherwise, it looks for the data in memory. Since the cache operates much faster than memory, the purpose of the cache is to help the processing unit 112 run faster.

[0055] Memory 120 provides running space for processes in computing device 100. For example, memory 120 can store computer programs (specifically, program code) used to create processes. Memory, also known as internal memory, is used to temporarily store computational data from processor 110, as well as data exchanged with external storage devices such as hard disks. As long as the computer is running, processor 110 will load the data that needs to be processed into memory for computation, and after the computation is completed, processing unit 112 will then send the result back out.

[0056] By way of example and not limitation, in this embodiment, memory 120 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory 120 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0057] It should be understood that the structure of the computing device 100 listed above is only an illustrative example and this application is not limited thereto. The computing device 100 in the embodiments of this application may include various hardware in the computer system in the prior art. For example, the computing device 110 may also include other storage besides memory 120, such as disk storage.

[0058] To facilitate understanding of the embodiments of this application, some basic concepts involved in this application will be briefly explained.

[0059] 1. Virtual Address: A virtual address is a specific address in the address space that the operating system can recognize or generate. Its size range is determined by the bitness of the operating system running on the processor. For example, if the operating system running on the processor is 32-bit, the virtual address is also 32-bit, and its virtual address range is 0-0xFFFFFFFF (4GB); if the operating system running on the processor is 64-bit, the virtual address is also 64-bit, and its address space is 0-0xFFFFFFFFFFFFFFFF (16EB).

[0060] Virtual addresses can be divided into multiple virtual address spaces according to actual needs, such as user-mode address space and kernel-mode address space. User-mode address space can be accessed by user-mode programs (such as reading, writing, opening, closing, or drawing) and kernel-mode programs (such as process management, memory management, file management, or device management), while kernel-mode address space can only be accessed by kernel-mode programs at runtime.

[0061] 2. Physical address: This can be a specific address within the address space of a hardware storage device such as internal memory.

[0062] 3. Virtual Page: Also known as a page. The MMU can use a paging mechanism to manage the virtual address space in units of virtual pages. Each page can include a virtual address space of a preset size. A virtual address space can include more than one virtual address, and one virtual address space can correspond to a set of page tables. This set of page tables is used to determine the physical address corresponding to the virtual address in the virtual address space. The set of page tables can include a first-level page table, or it can include a first-level page table and at least one second-level page table. The second-level page table can include second-level page tables, third-level page tables, or even lower-level page tables.

[0063] 4. Physical Pages: In the Linux kernel, the primary function of the MMU is to map "virtual addresses" to real "physical addresses." Specifically, the MMU uses physical pages as the basic unit of memory management. Different architectures support different physical page sizes. For example, in a 32-bit architecture, the supported physical page size is 4 kilobytes (KB), in a 64-bit architecture, the supported physical page size is 8 KB, and in a 128-bit architecture, the supported physical page size is 16 KB, and so on.

[0064] Physical page partitioning usually follows certain requirements. A 4KB page is typically called a regular page, while pages larger than 4KB are typically called "large pages." For example, a 2-megabyte (MB) page and a 1-gigabyte (GB) page are both called large pages. Of course, large pages can have other sizes, but they are usually multiples of 4KB.

[0065] 5. User mode: refers to the non-privileged state, which is the execution environment of a normal process. It cannot perform hardware-defined privileged operations, such as directly manipulating page table registers.

[0066] 6. Kernel mode: refers to privileged mode, which is the execution state of the OS kernel and services, and is able to perform privileged operations.

[0067] 7. User Space: Refers to the VA range that processes running in user space can access; it is part of the Visual Studio.

[0068] 8. Kernel Space: Refers to the VA range that can only be accessed by processes in kernel mode, and is part of VS.

[0069] 9. Inter-process communication: refers to the means of exchanging data between processes, including signals, message queues, pipes, shared memory, etc.

[0070] In addition, the following points are provided to facilitate understanding of the embodiments of this application.

[0071] First, in this application, "at least one" refers to one or more, and "more than one" refers to two or more. Furthermore, in the embodiments of this application, "first," "second," and various numerical designations (e.g., "#1," "#2," etc.) are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The sequence numbers of the processes below do not imply an order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. It should be understood that the objects described in this way can be interchanged where appropriate to describe solutions other than those in the embodiments of this application. In addition, in the embodiments of this application, the term "201," etc., is merely an identifier for descriptive convenience and does not limit the order of execution steps.

[0072] Second, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0073] Third, the term "storage" in the embodiments of this application can refer to storage in one or more memories. These memories can be separate installations or integrated into an encoder, decoder, processor, or communication device. Alternatively, some memories can be separately installed, while others are integrated into the decoder, processor, or communication device. The type of memory can be any form of storage medium, and this application does not limit this.

[0074] Fourth, the term "comprising" (also referred to as "includes", "including", "comprises" and / or "comprising") used in the embodiments of this application, when used in this specification, specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0075] Fifth, the word "if" in the embodiments of this application can be interpreted as meaning "when" or "upon" or "in response to determination" or "in response to detection". Similarly, depending on the context, the phrase "if it is determined..." or "if [the stated condition or event] is detected" can be interpreted as meaning "when it is determined..." or "in response to determination..." or "when [the stated condition or event] is detected" or "in response to detection of [the stated condition or event]".

[0076] Sixth, the terminology used in the description of the various examples in the embodiments of this application is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and in the appended claims, the numerical forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0077] Seventh, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0078] In modern computer systems, different processes need to communicate or share data. Traditional inter-process object sharing techniques typically use shared memory. With the evolution of computer architecture towards multi-core processors and distributed systems, multiple processes often need to access the same shared memory, and the shared memory allocated to processes is also dynamically changing.

[0079] Especially when facing applications that require high real-time performance, such as database systems, cloud computing platforms, and distributed computing frameworks, the shared memory mechanisms in existing technologies lack flexible space management capabilities. When expanding or shrinking shared memory, they cannot dynamically expand or shrink memory space according to the actual situation, resulting in resource waste.

[0080] The memory sharing method provided in this application will be described in detail below with reference to the accompanying drawings.

[0081] It should be understood that the memory sharing method provided in the embodiments of this application can be applied to computer systems, for example, Figure 1 The system shown.

[0082] It should also be understood that the embodiments shown below do not particularly limit the specific structure of the execution subject of the method provided in the embodiments of this application. As long as the method provided in the embodiments of this application can be implemented by running a program that records the code of the method provided in the embodiments of this application. For example, the execution subject of the method provided in the embodiments of this application can be a device, or a functional module in the device that can call and execute a program.

[0083] Figure 2 This is a schematic flowchart illustrating a memory sharing method provided in this application. The memory sharing method includes the following steps:

[0084] 201. The first process obtains the index data of the object to be accessed and multiple first metadata nodes; wherein, each of the first metadata nodes contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, and each sub-index is used as the starting address contained in the first metadata node at the corresponding level for address offset, so as to determine an address in the first metadata node at the corresponding level;

[0085] 202. The first process indexes from the highest level first metadata node to the target metadata node in the lowest level of the multiple first metadata nodes, based on the index data and the multiple first metadata nodes.

[0086] In this embodiment, the first process can be the owner (or holder) of a shared object (that is, a block of shared memory, which refers to the same memory area that multiple processes can access through shared memory), and has the authority to allocate, release, read and write the shared object.

[0087] In this embodiment, the second process can be a renter of the shared object, with read-only permissions for the shared object.

[0088] In this application embodiment, the third process can be a memory management process in the kernel, which can be called a mapper. A mapper is a process that has the authority to map physical memory to other processes. Specifically, it has the authority to allocate and release memory in the address space of other processes.

[0089] It should be understood that in the embodiments of this application, the specific roles of the first process, the second process and the third process can correspond one-to-one with specific shared objects, and the roles of different processes can be different when processing different shared objects.

[0090] In this embodiment, the shared object of the first process (and the second process) may be allocated by the third process. Specifically, the third process may allocate information used to index the address of the shared object to the first process (and the second process). In this embodiment, the third process may allocate multiple first metadata nodes and index data as information to index the address of the shared object to the first process.

[0091] Next, we will introduce the process of applying for the identity of a renter for a shared object in the first process:

[0092] In one possible implementation, refer to Figure 4 The first process can register the usage identity (e.g., holder or lessee) of a specific shared object. Specifically, the mapper can first determine the permissions of each shared object (e.g., determine the permissions of the first process). Figure 4 The holder of a shared object has the rights to allocate, release, read, and write, while the lessee has the right to read. The first process can notify the third process to register as the holder of a shared object. After the third process authenticates the first process, it allocates metadata for the first process in its address space and maps it to the address spaces of those with access rights to that shared memory (e.g., the holder and the lessee) using shared memory. The specific process can be described as follows: Figure 4 As shown.

[0093] In one possible implementation, when the holder initially allocates a shared object, it needs to initiate a memory allocation request to the mapper. The mapper then allocates corresponding metadata pages and data pages (i.e., shared memory) for itself and all other user processes, establishing a sharing relationship. (See reference...) Figure 3 To enable flexible scaling of the shared object allocator, this application embodiment can employ a tree-like data structure (also known as a radix tree, such as multiple first metadata nodes in this application embodiment) to implement the shared memory index. Based on this structure, shared memory allocation and reclamation can also be achieved. Therefore, metadata includes intermediate nodes and data nodes of the radix tree, and the physical page containing the metadata is shared by the mapper, holder, and lessee. The page containing the data is shared by the holder and lessee.

[0094] Specifically, the third process can allocate index data and multiple first metadata nodes to the first process. The third process can also allocate index data (the same as that allocated by the first process) and multiple second metadata nodes to the second process. The first process can index the shared memory address based on the index data and multiple first metadata nodes, and the second process can index the shared memory address based on the index data and multiple second metadata nodes. In other words, when the third process allocates memory, it can allocate the same index data to different processes.

[0095] The multiple first metadata nodes can be hierarchical, with each level including at least one first metadata node. For example, the multiple first metadata nodes can be a tree structure (e.g., a radix tree), where each root node can index multiple leaf nodes. For instance, the highest-level first metadata node is the root node, and based on the highest-level first metadata node, multiple leaf nodes (i.e., multiple first metadata nodes of multiple adjacent next-level nodes) can be indexed. Similarly, each first metadata node of the next level (compared to the highest-level first metadata node) can also be indexed to multiple first metadata nodes of multiple adjacent next-level nodes, until the lowest-level first metadata node is indexed. The lowest-level first metadata node can indicate the address of the memory space (e.g., the starting virtual address), based on which shared memory can be accessed.

[0096] For example, refer to Figure 6a , Figure 6a Multiple first metadata nodes can be Ln Node, L1 Node, and so on down to DNode. Ln Node is the highest-level first metadata node, L1 Node is the next-lowest level first metadata node, and DNode is the lowest-level first metadata node. Indexing can proceed from Ln Node to L1 Node, and so on down to DNode. DNode can contain a memory address pointing to shared memory (e.g., ...). Figure 6a In the data chunk, shared memory can be divided into multiple memory units (i.e., object slots), and the address of the memory space contained in the Dnode can point to the starting address of an object slot in the shared memory.

[0097] Specifically, the indexing process described above can be performed using index data. The index data includes multiple sub-indexes, each corresponding to a level of the first metadata node. Each first metadata node contains multiple addresses, with each address in a higher-level first metadata node used to index a first metadata node in the next adjacent level. During indexing, the sub-index can serve as the address offset of the starting address of the corresponding level's first metadata node. The starting address of each first metadata node can be offset by the address of the corresponding sub-index, and the offset address is an address of the first metadata node. Based on this address, a first metadata node in the next adjacent level can be indexed.

[0098] For example, refer to Figure 6aThe index data (specifically, the slow path region within the index data) can contain multiple sub-indexes. Sub-index 1 corresponds to an Ln Node, sub-index 2 corresponds to an L1 Node, and during indexing, sub-index 1 can be used as the address offset of the starting address of an Ln Node. The offset address (belonging to the Ln Node) can then point to an L1 Node. Similarly, sub-index 2 can be used as the address offset of the starting address of an L1 Node. The offset address (belonging to the L1 Node) can then point to an L2 Node, and so on, until a Dnode is indexed. The address of that Dnode (…) Figure 6a The uva in the context can point to an object slot.

[0099] Index data can be used by different processes when indexing shared memory addresses. Since different processes have different address spaces, cross-address space sharing is required. Therefore, pointers cannot be used between upper and lower level metadata, because a pointer in a process's address space describes the virtual address of a physical address in its virtual space. The virtual address of the same physical memory is different in different address spaces, so this value is meaningless to other processes. In this embodiment, a sub-index from index data that is universally applicable across address spaces is used as an offset value, serving as a "pointer" in shared memory, thereby achieving cross-address space sharing.

[0100] In this embodiment, multiple first metadata nodes adopt a tree structure, using "index" instead of "pointer" to achieve cross-address space sharing. In addition, when expanding and shrinking the shared memory space, fine-grained expansion and shrinking can be achieved in small amounts and multiple times, with strong scalability and less memory resource waste. The capacity of the shared memory space can be expanded and shrunk in real time as needed, without the need to pre-allocate large blocks of memory, and can share composite data structures across address spaces.

[0101] In one possible implementation, when expanding memory, for example when the first process needs new memory space, the first process can send a memory allocation request to the third process. Then, the third process can determine the new memory space (memory page) and expand the structure of the radix tree (that is, multiple first metadata nodes) as needed (for example, on the basis of the original multiple first metadata nodes, add new first metadata nodes and corresponding index data, and the newly added first metadata nodes and corresponding index data can index the address of the new memory space). The third node can allocate the newly added information to each process (processes with permission to access the newly allocated shared memory).

[0102] Specifically, when the first process needs new shared memory, it sends a memory allocation request to the third process; the third process allocates newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all multiple processes have permission to access the newly allocated shared memory.

[0103] Next, we will explain under what circumstances the first process needs to request new memory space from the third process.

[0104] In one possible implementation, the first process can determine the need for new shared memory when it finds that there are insufficient allocatable object slots in the free list, where the object slots represent the smallest allocation granularity of the shared memory; the free list contains the allocatable status of each object slot.

[0105] The free list records the allocatability status of each object slot. Allocatability can be understood as whether an object can be allocated on a data page within the address space. If a data page in that address space already holds an object, it is in an unallocable state; otherwise, it is in an allocatable state. Since the shared space allocated to a process may not be contiguous, meaning the object slots are not contiguous, the free list can also contain indices of adjacent object slots to already occupied slots.

[0106] Specifically, when a holder allocates an object, it first checks if there are any available objects in the corresponding size's object slot in the free list of the local address space. If not, the mapper allocates the necessary shared metadata nodes (LnNode, Dnode) and shared data chunks to all holders and renters, completing the growth of the radix tree. The data chunks are then split, with a small metadata area reserved before each chunk to store the metadata (length, allocation status) of all objects within that chunk. The data area of ​​the chunk is then divided into multiple object slots. These are implicitly attached to the free list, meaning the memory where the free object resides stores its index to the objects preceding and following it on the free list, without allocating additional memory to maintain the free list. The overall process can be described as follows: Figure 5 As shown.

[0107] After the shared memory allocator has expanded, it retrieves an available object from the corresponding slot in the free list and allocates it to the user. When there are no free objects of the available size, the larger object is split into smaller, allocatable objects. During subsequent object allocations, if there are allocatable objects in the free list, the holder automatically handles the allocation and metadata area updates, which the lessee can directly perceive through the shared memory. No additional inter-process communication is required during this process.

[0108] Simultaneously, the holder can also initiate the release of shared objects, a process that is the reverse of the object allocation process. The process can be as follows: Figure 6b As shown, when a single object is released, the object is added to the free list (free_list). When all objects in the entire data page are in the free_list, the page mapping of the data page in the address spaces of all holders / tenants is removed.

[0109] When the holder releases an object, it directly adds the corresponding object to the free list, without requiring additional inter-process communication. When the mapper needs to reclaim memory, it scans each shared data block and metadata node. If there are subtrees in the radix tree that do not maintain any allocated objects, it unmaps these shared memory blocks in turn and finally reclaims the physical memory.

[0110] In one possible implementation, the index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; thus, the first process can access the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0111] Since address indexing based on sub-indexes and multiple first metadata nodes in the index data needs to be determined level by level, the time and computing power overhead is large, hence it can be called a slow path index. The advantage of slow paths is that there is no capacity limit. In this embodiment, fast path indexing can also be supported. Specifically, the index data also includes fast addressing information, which can indicate the address offset of the starting address of the first metadata node at the highest level of the address of the object to be accessed. For example, referring to... Figure 4 , Figure 4 The object index is divided into two parts: the fast path index segment and the slow path index segment. The fast path index segment stores the global offset of the fast path object relative to the root node of the first metadata node (that is, the first metadata node at the highest level). When accessing the object subsequently, the index can be completed simply by adding the global offset of the object to the starting address of the fast path. The advantage of the fast path index is that it has lower time and computational overhead.

[0112] Optionally, the fast path size can be configured as needed. The relationship between the length x (in bits) of the fast path index segment and the size fsize (in bytes) of the fast path reserved mapping area is: fsize = 2 x .

[0113] The shared object allocator's object index supports both high-speed addressing (fast path) and unlimited capacity (slow path), and can be flexibly configured by the user. It can simultaneously address the diverse needs of various application scenarios regarding shared object usage, and the system has significant optimization potential. Fast and slow paths are flexibly configured as needed; the fast path object indexing speed is close to that of a single-address-space allocator, while the slow path has unlimited capacity. This allows for a balance between allocator performance and memory consumption in different scenarios.

[0114] Read and write operations are encapsulated by the shared object allocator, which provides a unified interface to the outside world. The addressing and read / write synchronization mechanisms inside the allocator are completely transparent to users. If the address spaces of holders and renters are merged in subsequent system version evolutions, there is no need to modify the code that calls the shared object read and write interface, thus achieving compatibility with system configuration.

[0115] Managed object synchronization mechanisms are supported throughout the object's lifecycle. The object operation interface supports both managed and self-managed modes. The interface remains consistent across all user process address spaces, whether merged or separate, eliminating the need for complex synchronization strategy modifications and enabling flexible configuration that allows for both separation and combination. It is user-friendly and flexible, requiring only a single address space perspective during development. No repetitive coding is needed in scenarios involving merged or separate address spaces for multiple processes / system components, further supporting flexible configuration that allows for both separation and combination.

[0116] 203. The first process accesses the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0117] In one possible implementation, the first process can perform a read operation on the object to be accessed based on the address contained in the target metadata node; and the space of the object to be accessed includes a version number identifier; the success of the read operation is determined by whether the version number identifier before the read operation and the version number identifier after the read operation are the same.

[0118] This application embodiment supports the managed object (coobj) pattern, where a version number can be configured in the memory control where each shared object (e.g., the object to be accessed) resides. When a process reads an object, it can read the object's version number before and after the read. If the version numbers match, the read is successful; otherwise, it retryes. The holder can atomically modify the version number before and after modifying the object's content.

[0119] The following, combined with Figures 7 to 9This application provides a detailed description of the memory sharing apparatus provided in its embodiments. It should be understood that the descriptions of the apparatus embodiments correspond to the descriptions of the method embodiments; therefore, any content not described in detail can be found in the above method embodiments. For brevity, some content is omitted.

[0120] Figure 7 This diagram illustrates the structure of a memory sharing device 700 according to an embodiment of this application. The memory sharing device 700 can be applied to a processing core running processes, and can be used to execute the aforementioned memory sharing method, for example, to execute... Figure 2 The method shown. The memory sharing device 700 may include:

[0121] The first process acquisition module 701 is used to acquire the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, and each sub-index is used as the starting address contained in the first metadata node at the corresponding level for address offset, so as to determine an address in the first metadata node at the corresponding level.

[0122] The specific description of the acquisition module 701 can be found in the description of step 201 in the above embodiment, and the similarities will not be repeated here.

[0123] The index module 702 of the first process indexes from the highest-level first metadata node to the target metadata node among the lowest-level first metadata nodes based on the index data and the plurality of first metadata nodes.

[0124] The specific description of the index module 702 can be found in step 202 of the above embodiment, and the similarities will not be repeated here.

[0125] The access module 703 of the first process performs data access on the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

[0126] The specific description of the access module 703 can be found in step 203 of the above embodiment, and the similarities will not be repeated here.

[0127] In one possible implementation, the physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

[0128] In one possible implementation, the index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the access module of the first process is further configured to perform data access on the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

[0129] In one possible implementation,

[0130] The transceiver module of the first process is used to send a memory allocation request to the third process when new shared memory is needed;

[0131] The device also includes:

[0132] The allocation module of the third process is used to allocate newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all multiple processes have permission to access the newly allocated shared memory.

[0133] In one possible implementation, the transceiver module of the first process is used for:

[0134] When there are insufficient object slots in the free list to be allocated, a memory allocation request is sent to a third process; where the object slot is the smallest allocation granularity of shared memory; the free list contains the allocation status of each object slot.

[0135] In one possible implementation, the access module of the first process is used for:

[0136] The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed;

[0137] After the read operation, the second version number identifier included in the space of the object to be accessed is read;

[0138] The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

[0139] This application also provides a chip system 800, such as... Figure 8 As shown, the chip system 800 includes at least one processor and at least one interface circuit. As an example, when the chip system 800 includes one processor and one interface circuit, the processor can be... Figure 8The processor 810 shown in the solid box (or the processor 810 shown in the dashed box) can have an interface circuit that is... Figure 8 The interface circuit 820 is shown in the solid box (or the interface circuit 820 is shown in the dashed box).

[0140] When the chip system 800 includes two processors and two interface circuits, then the two processors include Figure 8 The processor 810 shown in the solid box and the processor 810 shown in the dashed box, these two interface circuits include Figure 8 Interface circuit 820 is shown in both solid and dashed boxes. This is not a limitation. Processor 810 and interface circuit 820 can be interconnected via lines. For example, interface circuit 820 can be used to receive signals (e.g., instructions stored in memory). As another example, interface circuit 820 can be used to send signals to other devices (e.g., processor 810).

[0141] For example, interface circuit 820 can read instructions stored in memory and send those instructions to processor 810. When the instruction is executed by processor 810, it can cause a memory-sharing device or a device accessing memory to perform the steps in the above embodiments. Of course, the chip system 800 may also include other discrete devices, and this application embodiment does not specifically limit this.

[0142] Another embodiment of this application provides a computer-readable storage medium storing instructions that, when executed on a memory-sharing device, perform the various steps of the method flow shown in the above-described method embodiments. In some embodiments, the disclosed method can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0143] Figure 9 A conceptual partial view of a computer program product provided in an embodiment of this application is shown schematically. The computer program product includes a computer program for executing computer processes on a computing device.

[0144] In one embodiment, a computer program product is provided using a signal bearer medium 900. The signal bearer medium 900 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 2 The described function or part of the function. Therefore, for example, refer to... Figure 2 One or more features of 201-203 can be assumed by one or more instructions associated with the signal carrying medium 900. Furthermore, Figure 9 The program instructions in the document also describe example instructions.

[0145] In some examples, the signal carrying medium 900 may include a computer-readable medium 901, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0146] In some implementations, the signal carrying medium 900 may include a computer recordable medium 902, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0147] In some implementations, the signal-bearing medium 900 may comprise a communication medium 903, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). The signal-bearing medium 900 may be transmitted by a wireless communication medium 903 (e.g., a wireless communication medium conforming to the IEEE 1502.11 standard or other transmission protocols). One or more program instructions may be, for example, computer-executable instructions or logic implementation instructions.

[0148] In some examples, such as targeting Figure 2 The memory sharing method can be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 901, a computer-recordable medium 902, and / or a communication medium 903.

[0149] It should be understood that the arrangements described herein are for illustrative purposes only. Therefore, those skilled in the art will understand that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements may be omitted depending on the desired outcome. Furthermore, many of the described elements are functional entities that can be implemented as discrete or distributed components, or in any suitable combination and location with other components.

[0150] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When executed on a computer and when the computer execution instructions are executed, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0151] Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or it can include one or more data storage devices such as servers or data centers that can be integrated with that medium. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0152] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A memory sharing method, characterized in that, The method includes: The first process acquires the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, each sub-index corresponds to a first metadata node at a level, and each sub-index is used as an address offset of the starting address contained in the first metadata node at the corresponding level to determine an address in the first metadata node at the corresponding level; The first process indexes from the highest-level first metadata node to the target metadata node among the lowest-level first metadata nodes, based on the index data and the plurality of first metadata nodes. The first process accesses the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

2. The method according to claim 1, characterized in that, The physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

3. The method according to claim 1 or 2, characterized in that, The index data further includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the first metadata node at the highest level; the method further includes: The first process accesses the data of the object to be accessed based on the starting address of the first metadata node at the highest level and the fast addressing information.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: When the first process needs new shared memory, it sends a memory allocation request to the third process. The third process allocates newly added index data and newly added first metadata node to multiple processes, including the first process; the newly added index data and the newly added first metadata node are used to index the newly allocated shared memory, wherein all the multiple processes have permission to access the newly allocated shared memory.

5. The method according to claim 4, characterized in that, When the first process needs new shared memory, it sends a memory allocation request to the third process, including: When the first process determines that there are insufficient allocatable object slots in the free list, it sends a memory allocation request to the third process; wherein, the object slot is the smallest allocation granularity of shared memory; and the free list contains the allocatable status of each object slot.

6. The method according to any one of claims 1 to 5, characterized in that, The first process accesses data from the object to be accessed based on the address contained in the target metadata node, including: The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed; the method further includes: After the read operation, the second version number identifier included in the space of the object to be accessed is read; The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

7. A memory sharing device, characterized in that, The device includes: The first process acquisition module is used to acquire the index data of the object to be accessed and multiple first metadata nodes; wherein, each first metadata node contains multiple addresses, and each address in the first metadata node at a higher level is used to index to a first metadata node at the next adjacent level; the index data includes multiple sub-indexes, each sub-index corresponds to a first metadata node at a level, and each sub-index is used as an address offset of the starting address contained in the first metadata node at the corresponding level to determine an address in the first metadata node at the corresponding level; The indexing module of the first process is used to index from the highest-level first metadata node among the multiple first metadata nodes to the target metadata node among the lowest-level multiple first metadata nodes, based on the index data and the multiple first metadata nodes. The access module of the first process is used to access the data of the object to be accessed based on the address contained in the target metadata node; wherein the object to be accessed is shared memory.

8. The apparatus according to claim 7, characterized in that, The physical space corresponding to the address contained in the target metadata node is shared memory that can be accessed by both the first process and the second process, and the index data used by the second process when accessing the shared memory is the same index data used by the first process when accessing the shared memory.

9. The apparatus according to claim 7 or 8, characterized in that, The index data also includes fast addressing information, which indicates the address offset of the address of the object to be accessed relative to the starting address of the highest-level first metadata node; the access module of the first process is further configured to allow the first process to access the object to be accessed based on the starting address of the highest-level first metadata node and the fast addressing information.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The transceiver module of the first process is used to send a memory allocation request to the third process when new shared memory is needed; The device further includes: The allocation module of the third process is used to allocate newly added index data and newly added first metadata nodes to multiple processes, including the first process; the newly added index data and the newly added first metadata nodes are used to index the newly allocated shared memory, wherein all the multiple processes have permission to access the newly allocated shared memory.

11. The apparatus according to claim 10, characterized in that, The transceiver module of the first process is used for: When there are insufficient object slots in the free list to be allocated, a memory allocation request is sent to a third process; wherein, the object slot is the smallest allocation granularity of shared memory; and the free list contains the allocation status of each object slot.

12. The apparatus according to any one of claims 7 to 11, characterized in that, The access module of the first process is used for: The first process performs a read operation on the object to be accessed based on the address contained in the target metadata node; and reads the first version number identifier included in the space of the object to be accessed; After the read operation, the second version number identifier included in the space of the object to be accessed is read; The success of the read operation is determined by whether the first version number identifier and the second version number identifier are the same.

13. A computing device, characterized in that, Including processor and memory; The processor is configured to execute instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1 to 6.

14. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device, the computing device performs the method as described in any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 6.

16. A chip, characterized in that, It includes at least one processing unit and an interface circuit, the interface circuit being used to provide program instructions or data to the at least one processing unit, the at least one processing unit being used to execute the program instructions to implement the method of any one of claims 1 to 6.

Images