Relaxed consistency distributed shared memory synchronization method and system based on bounded lock

Abstract

The application provides a bounded lock-based relaxed consistency distributed shared memory synchronization method and system, which comprises the following steps: taking the range of the shared memory accessed by an application program in a critical region as additional semantic information after the application program is granted a distributed lock, applying the latest change in the range to the local before the application program accesses the shared memory, creating a corresponding shadow memory area in the local, comparing the content of the memory in the range with the content of the shadow memory area when the application program releases the distributed lock, obtaining the memory change in the range, and adding the change into the request for releasing the distributed lock, and the change content is transmitted to the next holder of the lock. Compared with the prior art, the application has the characteristics of low latency, pure user mode, scalability and the like.

Description

Technical Field

[0001] This invention relates to the field of distributed systems technology, and more specifically, to a relaxed consistency distributed shared memory synchronization method and system based on bounded locks. Background Technology

[0002] In the field of distributed systems, using distributed shared memory for data sharing and transfer is an important technology. This abstraction of distributed shared memory allows data to be shared and transferred transparently between machines without the need for data serialization and deserialization operations.

[0003] Currently, academia and industry have proposed several methods for implementing the abstraction of distributed shared memory in real-world computer distributed systems. These methods focus on ensuring the transparency of the distributed shared memory abstraction to upper-layer applications while maintaining memory consistency across machines.

[0004] Currently, one approach to implementing distributed shared memory is to follow strong consistency distributed shared memory. This approach follows classic cache consistency protocols such as MSI. Each time consistency is maintained, memory updates are broadcast to all machines in the distributed system in units of pages. While this approach can ensure the transparency of distributed shared memory to upper-layer applications and guarantee cross-machine memory consistency, its drawbacks include high frequency of consistency maintenance, coarse granularity, and poor scalability due to synchronization across all machines. Even with the help of high-speed network channels, such as remote memory direct access technology and programmable switches, to optimize the consistency maintenance process, the overhead of maintaining consistency remains significant.

[0005] Another implementation of distributed shared memory is relaxed consistent shared memory, which leverages the semantic information of the application, namely the correlation between the application's control synchronization and data synchronization: the application needs to acquire / release locks when entering and leaving critical areas, and access shared data in a single thread within the critical areas. In this implementation, synchronization of changes to shared data only occurs when the application acquires / releases locks, which can significantly reduce the overhead of maintaining consistency. Its disadvantage is that in order to transparently record the changes made by the application to shared data in critical areas, the write permissions of the entire shared area need to be modified when acquiring / releasing locks, which incurs significant time overhead.

[0006] Patent document CN101207515A discloses a method for processing, implementing, and synchronizing shared memory across multiple machines. The processing method includes the following steps: A1. Predefining a memory data object; A2. When any node in the distributed network management system starts up, it creates the memory data object in its local memory; A3. Each host determines that the memory data object of a certain node has changed, and then updates its local memory data object accordingly. However, this patent cannot completely solve the existing technical problems. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a relaxed consistency distributed shared memory synchronization method and system based on bounded locks.

[0008] The relaxed consistency distributed shared memory synchronization method based on bounded locks provided by the present invention includes:

[0009] Step 1: Before accessing shared data, the application calls the interface to request a distributed lock. The distributed shared memory library obtains the range of the shared memory region that the application needs to access in the critical region based on the context of the lock request.

[0010] Step 2: When the application is granted a lock, apply all changes within that scope to local memory;

[0011] Step 3: If the size of the memory range being accessed is greater than a memory page, proceed to step 4; otherwise, proceed to step 5.

[0012] Step 4: The distributed shared memory library removes write permissions for this memory range, and then proceeds to step 6;

[0013] Step 5: Allocate a shadow memory region locally for this memory range, copy the initial content of the memory range to the shadow memory region, and record the starting address and size of the memory range and the starting address of the shadow memory region.

[0014] Step 6: The application accesses the distributed shared memory within this scope. If a page permission error occurs during this process, proceed to Step 7; if the application completes access to the shared memory, proceed to Step 8.

[0015] Step 7: Restore write access to the memory page, allocate a shadow memory region, copy the initial content of the memory page to the shadow memory region, record the starting address and size of the memory page and the starting address of the shadow memory region, and return to step 6 to continue execution;

[0016] Step 8: Generate modified content units based on the shadow memory region information recorded in Step 5 and Step 7, and add these modified content units to the list of existing modified content units in the shared memory within this range;

[0017] Step 9: The application calls the interface to release the distributed lock.

[0018] Preferably, a modified content unit consists of an address in a distributed shared memory space and a modified content of 8 bytes in size, and all modified content units in the memory range specified by the application are added to the network packet that grants the lock.

[0019] Preferably, the size of the memory range is passed in by the application through the interface for requesting the distributed lock; the size of the shadow memory region is the size of a memory page.

[0020] Preferably, in the process of generating modified content units, the distributed shared memory library first determines whether the content of a shadow memory region is the same as the content in the corresponding shared memory. If they are not the same, then a modified content unit is generated.

[0021] Preferably, the distributed shared memory library adds all changes to the shared memory within the current scope to the request to release the distributed lock, and the distributed lock manager sends the change unit within that scope to the next application thread that is granted the lock.

[0022] The relaxed consistency distributed shared memory synchronization system based on bounded locks provided by the present invention includes:

[0023] Module M1: Before accessing shared data, the application calls the interface to request a distributed lock. The distributed shared memory library obtains the range of the shared memory region that the application needs to access in the critical region based on the context of the lock request.

[0024] Module M2: When the application is granted a lock, all changes within that scope are applied to local memory;

[0025] Module M3: If the size of the accessed memory range is greater than a memory page, then module M4 is triggered; otherwise, module M5 is triggered.

[0026] Module M4: The distributed shared memory library removes write permissions for this memory range, and then triggers module M6;

[0027] Module M5: Allocates a shadow memory region locally for this memory range, copies the initial content of the memory range to the shadow memory region, and records the starting address and size of the memory range and the starting address of the shadow memory region.

[0028] Module M6: If an application accesses the distributed shared memory within this scope and a page permission error occurs during this process, module M7 is triggered; if the application completes its access to the shared memory, module M8 is triggered.

[0029] Module M7: Restore write access to the memory page, allocate a shadow memory region, copy the initial contents of the memory page to the shadow memory region, record the starting address and size of the memory page and the starting address of the shadow memory region, and trigger module M6;

[0030] Module M8: Generates modified content units based on the shadow memory region information recorded by modules M5 and M7, and adds these modified content units to the list of existing modified content units in the shared memory within this range;

[0031] Module M9: The application calls the interface to release the distributed lock.

[0032] Preferably, a modified content unit consists of an address in a distributed shared memory space and a modified content of 8 bytes in size, and all modified content units in the memory range specified by the application are added to the network packet that grants the lock.

[0033] Preferably, the size of the memory range is passed in by the application through the interface for requesting the distributed lock; the size of the shadow memory region is the size of a memory page.

[0034] Preferably, in the process of generating modified content units, the distributed shared memory library first determines whether the content of a shadow memory region is the same as the content in the corresponding shared memory. If they are not the same, then a modified content unit is generated.

[0035] Preferably, the distributed shared memory library adds all changes to the shared memory within the current scope to the request to release the distributed lock, and the distributed lock manager sends the change unit within that scope to the next application thread that is granted the lock.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) Compared with distributed shared memory that follows strong consistency, this invention is based on relaxed consistency. Shared data synchronization is only performed when the application acquires / releases the lock, and changes to the shared data are only transmitted between the corresponding lock holders, which significantly reduces the overhead of maintaining consistency and improves the scalability of the system.

[0038] (2) Compared with traditional relaxed consistent distributed shared memory, this invention significantly reduces the scope of changes to write permissions of shared memory regions by analyzing the scope of access to shared memory in critical regions, thereby reducing the time overhead of lock request / lock release operations.

[0039] (3) Compared with traditional relaxed consistent distributed shared memory, in this invention, when the scope of accessing shared memory is less than a memory page, the write permissions of the memory page containing the memory scope will not be changed. Instead, a corresponding shadow memory region will be directly generated, so that the application will not trigger page permission errors during operation, thereby further improving the running efficiency of the application. Attached Figure Description

[0040] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0041] Figure 1 This is a flowchart illustrating the relaxed consistency distributed shared memory content synchronization of this invention. Detailed Implementation

[0042] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0043] Example 1

[0044] This invention provides a relaxed consistency distributed shared memory synchronization method based on bounded locks. It uses a key-value pair storage system as the upper-layer application of distributed shared memory. Taking the process of an application thread in this key-value pair storage system modifying the value of a key-value pair with key 100 to 5 as an example, it combines... Figure 1 The following describes in detail the reading and writing of specific addresses in distributed shared memory:

[0045] Step 1: Before accessing shared data, the application calls the interface to request a distributed lock. The distributed shared memory library obtains the range of the shared memory region that the application needs to access in the critical region based on the context of the lock request.

[0046] Step 2: When the application is granted a lock, apply all changes within that scope to local memory;

[0047] Step 3: If the size of the memory range being accessed is greater than a memory page, proceed to step 4; otherwise, proceed to step 5.

[0048] Step 4: The distributed shared memory library removes write permissions for this memory range, and then proceeds to step 6;

[0049] Step 5: Allocate a shadow memory region locally for this memory range, copy the initial content of the memory range to the shadow memory region, and record the starting address and size of the memory range and the starting address of the shadow memory region.

[0050] Step 6: The application accesses the distributed shared memory within this scope. If a page permission error occurs during this process, proceed to Step 7; if the application completes access to the shared memory, proceed to Step 8.

[0051] Step 7: Restore write access to the memory page, allocate a shadow memory region, copy the initial content of the memory page to the shadow memory region, record the starting address and size of the memory page and the starting address of the shadow memory region, and return to step 6 to continue execution;

[0052] Step 8: Generate modified content units based on the shadow memory region information recorded in Step 5 and Step 7, and add these modified content units to the list of existing modified content units in the shared memory within this range;

[0053] Step 9: The application calls the interface to release the distributed lock.

[0054] In step 1, before accessing the key-value pair, the thread calls the interface to request a distributed lock. The starting address of the access range of the thread is 0x55c0, and the size is 24 bytes. The distributed shared memory library sends the distributed lock request to the distributed lock manager.

[0055] In step 2, when the thread is granted the lock, since no changes have been made to the shared memory within the current scope, no units of change need to be applied to the local memory. A unit of change consists of an address in the distributed shared memory space and a change of 8 bytes in size. All units of change in the memory scope specified by the application are added to the network packet that grants the lock.

[0056] In step 3, since the access range size is 24 bytes, which is less than a memory page, execution jumps to step 5. The size of the memory range is passed in by the application through the interface requesting the distributed lock.

[0057] In step 4, the distributed shared memory library only removes write permissions for shared memory within the scope specified by the application, without adjusting the access permissions for the remaining shared memory.

[0058] In step 5, the distributed shared memory library allocates a shadow memory region and copies the initial contents of the shared memory range into this shadow memory region. The distributed shared memory library records the starting address of the range (0x55c0), its size (24 bytes), and the starting address of the shadow memory region.

[0059] In step 6, the application thread accesses the shared memory within this scope and writes the value of the key-value pair to 5. No page fault occurs during this process, so the process jumps to step 8. The distributed shared memory library registers special page permission fault handling functions with the operating system when the application starts.

[0060] In step 7, the size of the shadow memory region is the size of a memory page.

[0061] In step 8, the distributed shared memory library generates modified content units based on the information recorded in step 5, and adds the generated modified content units to the existing list of modified content units for that shared memory range. During the generation of modified content units, for a shadow memory region, the distributed shared memory library first checks whether its content is the same as the content in the corresponding shared memory. If they are different, a modified content unit is generated.

[0062] In step 9, the application thread calls the interface to release the distributed lock. The distributed shared memory library adds the list of modified content units generated in step 8 to the request. The distributed lock manager then sends this list of modified content units to the next application thread that is granted the lock.

[0063] Example 2

[0064] The present invention also provides a relaxed consistent distributed shared memory synchronization system based on bounded locks. The relaxed consistent distributed shared memory synchronization system based on bounded locks can be implemented by executing the process steps of the relaxed consistent distributed shared memory synchronization method based on bounded locks. That is, those skilled in the art can understand the relaxed consistent distributed shared memory synchronization method based on bounded locks as a preferred embodiment of the relaxed consistent distributed shared memory synchronization system based on bounded locks.

[0065] The relaxed consistency distributed shared memory synchronization system based on bounded locks provided by the present invention includes: Module M1: Before accessing shared data, the application calls an interface to request a distributed lock, and the distributed shared memory library obtains the range of the shared memory region that the application needs to access within the critical region according to the context of the lock request; Module M2: When the application is granted a lock, all modified content units within the range are applied to local memory; Module M3: If the size of the accessed memory range is greater than a memory page, module M4 is triggered; otherwise, module M5 is triggered; Module M4: The distributed shared memory library removes the write permission for the memory range, and then triggers module M6; Module M5: A shadow memory region is allocated locally for the memory range, the initial content of the memory range is copied to the shadow memory region, and the memory range is... The starting address, size, and starting address of the shadow memory region are recorded; Module M6: When the application accesses the distributed shared memory within this range, if a page permission error occurs during this process, module M7 is triggered; if the application completes the access to the shared memory, module M8 is triggered; Module M7: Restores the write permission of the memory page, allocates a shadow memory region, copies the initial content of the memory page to the shadow memory region, records the starting address, size, and starting address of the shadow memory region, and triggers module M6; Module M8: Generates modified content units based on the shadow memory region information recorded by modules M5 and M7, and adds these modified content units to the list of existing modified content units in the shared memory within this range; Module M9: The application calls the interface to release the distributed lock.

[0066] A unit of modified content consists of an address in the distributed shared memory space and a modified content of 8 bytes. All units of modified content within the memory range specified by the application are added to the network packet granting the lock. The size of the memory range is passed in by the application through the interface requesting the distributed lock; the size of the shadow memory region is the size of a memory page.

[0067] In the process of generating modified content units, the distributed shared memory library first checks whether the content of a shadow memory region is the same as the content in the corresponding shared memory. If they are different, a modified content unit is generated. The distributed shared memory library adds all the modified content of the shared memory in the current scope to the request to release the distributed lock. The distributed lock manager then sends the modified content units in that scope to the next application thread that is granted the lock.

[0068] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0069] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A relaxed consistency distributed shared memory synchronization method based on bounded locks, characterized in that, include: Step 1: Before accessing shared data, the application calls the interface to request a distributed lock. The distributed shared memory library obtains the range of the shared memory region that the application needs to access in the critical region based on the context of the lock request. Step 2: When the application is granted a lock, apply all changes within that scope to local memory; Step 3: If the size of the memory range being accessed is greater than a memory page, proceed to step 4; otherwise, proceed to step 5. Step 4: The distributed shared memory library removes write permissions for this memory range, and then proceeds to step 6; Step 5: Allocate a shadow memory region locally for this memory range, copy the initial content of the memory range to the shadow memory region, and record the starting address and size of the memory range and the starting address of the shadow memory region. Step 6: The application accesses the distributed shared memory within this scope. If a page permission error occurs during this process, proceed to Step 7. If the application has completed its access to shared memory, proceed to step 8; Step 7: Restore write access to the memory page, allocate a shadow memory region, copy the initial content of the memory page to the shadow memory region, record the starting address and size of the memory page and the starting address of the shadow memory region, and return to step 6 to continue execution; Step 8: Generate modified content units based on the shadow memory region information recorded in Step 5 and Step 7, and add these modified content units to the list of existing modified content units in the shared memory within this range; Step 9: The application calls the interface to release the distributed lock; A modified content unit consists of an address in a distributed shared memory space and a modified content of 8 bytes in size. All modified content units in the memory range specified by the application are added to the network packet that grants the lock. In the process of generating modified content units, the distributed shared memory library first checks whether the content of a shadow memory region is the same as the content in the corresponding shared memory. If they are not the same, then a modified content unit is generated.

2. The relaxed consistency distributed shared memory synchronization method based on bounded locks according to claim 1, characterized in that, The size of the memory region is passed in by the application through the interface for requesting the distributed lock; the size of the shadow memory region is the size of a memory page.

3. The relaxed consistency distributed shared memory synchronization method based on bounded locks according to claim 1, characterized in that, The distributed shared memory library adds all changes to the shared memory within the current scope to the request to release the distributed lock. The distributed lock manager then sends the changes within that scope to the next application thread that has been granted the distributed lock.

4. A relaxed consistency distributed shared memory synchronization system based on bounded locks, characterized in that, include: Module M1: Before accessing shared data, the application calls the interface to request a distributed lock. The distributed shared memory library obtains the range of the shared memory region that the application needs to access in the critical region based on the context of the lock request. Module M2: When the application is granted a lock, all changes within that scope are applied to local memory; Module M3: If the size of the accessed memory range is greater than a memory page, then module M4 is triggered; otherwise, module M5 is triggered. Module M4: The distributed shared memory library removes write permissions for this memory range, and then triggers module M6; Module M5: Allocates a shadow memory region locally for this memory range, copies the initial content of the memory range to the shadow memory region, and records the starting address and size of the memory range and the starting address of the shadow memory region. Module M6: If an application accesses the distributed shared memory within this scope and a page permission error occurs during this process, module M7 is triggered. If the application completes its access to shared memory, module M8 is triggered; Module M7: Restore write access to the memory page, allocate a shadow memory region, copy the initial contents of the memory page to the shadow memory region, record the starting address and size of the memory page and the starting address of the shadow memory region, and trigger module M6; Module M8: Generates modified content units based on the shadow memory region information recorded by modules M5 and M7, and adds these modified content units to the list of existing modified content units in the shared memory within this range; Module M9: The application calls the interface to release the distributed lock; A modified content unit consists of an address in a distributed shared memory space and a modified content of 8 bytes in size. All modified content units in the memory range specified by the application are added to the network packet that grants the lock. In the process of generating modified content units, the distributed shared memory library first checks whether the content of a shadow memory region is the same as the content in the corresponding shared memory. If they are not the same, then a modified content unit is generated.

5. The relaxed consistency distributed shared memory synchronization system based on bounded locks according to claim 4, characterized in that, The size of the memory region is passed in by the application through the interface for requesting the distributed lock; the size of the shadow memory region is the size of a memory page.

6. The relaxed consistency distributed shared memory synchronization system based on bounded locks according to claim 4, characterized in that, The distributed shared memory library adds all changes to the shared memory within the current scope to the request to release the distributed lock. The distributed lock manager then sends the changes within that scope to the next application thread that has been granted the distributed lock.

Images