A memory synchronization method, device and equipment of a distributed system and a medium

Abstract

The application discloses a memory synchronization method and device of a distributed system, equipment and a medium, relates to the technical field of distributed computing, and comprises the following steps: obtaining a target memory modification instruction, modifying a memory block to be modified to obtain a modified memory block updated to a second state; updating version vector information of a target node, sending a memory synchronization instruction to other nodes, and updating to a third state, so that other nodes compare version vectors based on the memory synchronization instruction and feed back to the target node; counting the comparison result, and when it is determined that the counting result meets the condition, sending a synchronization data packet to other nodes in a multicast manner, so that other nodes synchronize the memory and feed back the synchronization result to the target node; integrating the synchronization result, and when it is determined that the integration result meets the condition, updating the physical memory of the target distributed system based on the modified memory block and updating to the first state, so as to complete the memory synchronization operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed computing, in particular to a memory synchronization method and device of a distributed system, equipment and medium. BACKGROUND

[0002] In real-time distributed systems, memory synchronization technology is the core foundation of realizing the collaborative work of multiple nodes. Traditional reflective memory cards realize deterministic low-latency memory synchronization through a dedicated fiber network and customized hardware, but have inherent defects such as high cost (single card price exceeds ten thousand yuan) and difficult expansion (requires a dedicated fiber switch). The above defects are usually solved by borrowing software. Existing software solutions include full synchronization schemes based on TCP (Transmission Control Protocol) / IP (Internet Protocol), which simulate hardware behavior through kernel drivers, but have high latency; difference synchronization systems based on UDP (User Datagram Protocol) realize fixed block and point-to-point transmission, but have low network utilization; and industrial protocol schemes rely on special network cards to realize deterministic transmission, but are limited in the number of nodes. These schemes have defects such as high latency, low network utilization, and limited expansion capability, resulting in low efficiency of memory synchronization.

[0003] In summary, how to solve the dependence of memory synchronization on specific hardware and ensure the efficiency of memory synchronization is a technical problem to be solved at present. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a memory synchronization method, device, equipment and medium of a distributed system, which can solve the dependence of memory synchronization on specific hardware and ensure the efficiency of memory synchronization. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a memory synchronization method of a distributed system, applied to a target node in a target distributed system, wherein the target distributed system includes a plurality of nodes, each of the plurality of nodes has a corresponding buffer memory area in its local memory, and the target node is a node in the plurality of nodes that obtains a target memory modification instruction through a preset interface; wherein the method comprises:

[0006] obtaining a target memory modification instruction through the preset interface, and modifying a to-be-modified memory block based on the target memory modification instruction to obtain a modified memory block whose block state is updated from a first state to a second state; the to-be-modified memory block is a memory block in the buffer memory area of the target node corresponding to the modification content of the target memory modification instruction, and the block state of the memory block is the first state.

[0007] updating version vector information of the target node to obtain updated version vector information, sending a memory synchronization instruction comprising the updated version vector information to other nodes, and updating a block state of the modified memory block from the second state to a third state, so that the other nodes perform version vector comparison based on the memory synchronization instruction and feed back corresponding comparison results to the target node; the other nodes are other nodes of the several nodes except the target node;

[0008] counting the comparison results, and when it is determined that a corresponding statistical result meets a first preset node confirmation condition, sending a synchronization data packet comprising block information of the modified memory block to the other nodes in a multicast manner, so that the other nodes perform memory synchronization according to the synchronization data packet, and feed back corresponding synchronization results to the target node after completing the synchronization;

[0009] integrating the synchronization results obtained, and when it is determined that an integrated result meets a second preset node confirmation condition, updating the physical memory of the target distributed system based on the modified memory block, and updating the block state of the modified memory block from the third state to the first state, to complete the memory synchronization operation this time.

[0010] Optionally, before the target memory modification instruction is obtained through the preset interface, the method further comprises:

[0011] when the target distributed system is initialized, establishing a corresponding memory space of a preset size as a corresponding buffer memory area for each of the several nodes;

[0012] dividing the buffer memory area into a plurality of memory blocks based on a preset memory block division strategy, and establishing a mapping relationship between the plurality of memory blocks and the physical memory of the target distributed system by using a memory mapping mechanism, so as to map the physical memory to the plurality of memory blocks, so that the several nodes perform memory synchronization operations through the plurality of memory blocks;

[0013] Correspondingly, the updating the physical memory of the target distributed system based on the modified memory block comprises:

[0014] determining a target physical memory corresponding to the modified memory block in the physical memory of the target distributed system according to the mapping relationship;

[0015] updating the target physical memory by using the modification content of the modified memory block.

[0016] Optionally, the modifying the to-be-modified memory block based on the target memory modification instruction comprises:

[0017] determining the corresponding modification content according to the target memory modification instruction, and the to-be-modified memory block corresponding to the modification content in the buffer memory area of the target node;

[0018] modifying the corresponding content of the to-be-modified memory block based on the modification content to obtain a corresponding modification result;

[0019] if the modification result indicates that the modification of the corresponding content of the to-be-modified memory block fails, modifying the corresponding content of the to-be-modified memory block again based on the modification content until the modification succeeds, updating the block state of the to-be-modified memory block from the first state to the second state to obtain a corresponding modified memory block; the first state is an initial block state indicating that the memory block is not modified; and the second state is a block state indicating that the memory block is modified and is to be synchronized.

[0020] Optionally, the comparing the version vectors based on the memory synchronization instruction and feeding back a corresponding comparison result to the target node comprises:

[0021] obtaining the memory synchronization instruction sent by the target node;

[0022] determining the updated version vector in the memory synchronization instruction and determining a to-be-compared version vector of the other node;

[0023] comparing the to-be-compared version vector and the updated version vector to obtain a corresponding comparison result;

[0024] determining a target version number in which the version number in the to-be-compared version vector is inconsistent with the corresponding version number in the updated version vector based on the comparison result, and updating the to-be-compared version vector;

[0025] determining a to-be-synchronized memory block according to the target version number and feeding back the comparison result to the target node;

[0026] Correspondingly, the synchronizing the memory according to the synchronization data packet and feeding back a corresponding synchronization result to the target node after the synchronization is completed comprises:

[0027] receiving the synchronization data packet sent by the target node and performing verification on the synchronization data packet to obtain a corresponding verification result;

[0028] If the check result represents that the synchronization data packet passes the check, the synchronization data packet is parsed to obtain the block information in the synchronization data packet.

[0029] Memory synchronization operation is performed on the to-be-synchronized data block based on the block information, and after the synchronization is completed, a corresponding synchronization result is fed back to the target node.

[0030] Optionally, the comparison result is counted, and after it is determined that a corresponding statistical result meets a first preset node confirmation condition, a synchronization data packet including the block information of the modified memory block is sent to the other nodes in a multicast manner, including:

[0031] The comparison result fed back by the other nodes is obtained, and the comparison result is counted to obtain a corresponding statistical result; the statistical result includes a first result quantity of the comparison result;

[0032] The number of nodes of the other nodes is determined, and the first result quantity in the statistical result is compared with the number of nodes;

[0033] If the first result quantity is consistent with the number of nodes, it is determined that the statistical result meets a first preset node confirmation condition, a synchronization data packet including the block information of the modified memory block is sent to the other nodes in a multicast manner, and the transmission situation of the synchronization data packet is continuously monitored;

[0034] If the transmission situation represents that the synchronization data packet is lost, the synchronization data packet is retransmitted.

[0035] Optionally, the obtained synchronization result is integrated, and when it is determined that an integrated result meets a second preset node confirmation condition, the physical memory of the target distributed system is updated based on the modified memory block, and the block state of the modified memory block is updated from the third state to the first state, to complete the memory synchronization operation, including:

[0036] The synchronization result fed back by the other nodes is obtained, and the synchronization result is integrated to obtain a corresponding integrated result; the integrated result includes a second result quantity of the synchronization result;

[0037] The number of nodes of the other nodes is determined, and the second result quantity in the integrated result is compared with the number of nodes;

[0038] If the second result quantity is not less than half of the number of nodes, it is determined that the integrated result meets a second preset node confirmation condition, and the physical memory of the target distributed system is updated based on the modified memory block;

[0039] updating a block state of the modified memory block from the third state to the first state to complete the current memory synchronization operation; the third state is a block state representing that a memory block is in synchronization.

[0040] Optionally, the memory synchronization method of the distributed system further comprises:

[0041] continuously recording modification information corresponding to each of the plurality of memory blocks in the buffer memory area;

[0042] generating a memory access heat map for the plurality of memory blocks based on the modification information according to a preset heat map generation frequency;

[0043] adjusting a block granularity of the buffer memory area according to the memory access heat map, and adjusting the plurality of memory blocks in the buffer memory area according to the block granularity.

[0044] In a second aspect, the present application provides a memory synchronization device of a distributed system, which is applied to a target node in a target distributed system, the target distributed system comprising a plurality of nodes, each of the plurality of nodes comprising a corresponding buffer memory area in its local memory, and the target node being a node in the plurality of nodes that obtains a target memory modification instruction through a preset interface; wherein the device comprises:

[0045] a memory block modification module, configured to obtain a target memory modification instruction through the preset interface, and modify a to-be-modified memory block based on the target memory modification instruction to obtain a modified memory block with a block state updated from a first state to a second state; the to-be-modified memory block being a memory block in the buffer memory area of the target node corresponding to a modification content of the target memory modification instruction and having the first state;

[0046] a first block state updating module, configured to update version vector information of the target node to obtain updated version vector information, send a memory synchronization instruction comprising the updated version vector information to other nodes, and update a block state of the modified memory block from the second state to a third state, so that the other nodes compare version vectors based on the memory synchronization instruction and feed back corresponding comparison results to the target node; the other nodes being nodes other than the target node in the plurality of nodes;

[0047] The data packet sending module is configured to count the comparison results, and send a synchronization data packet including the block information of the modified memory block to the other nodes by using a multicast mode when it is determined that the corresponding counting result meets a first preset node confirmation condition, so that the other nodes perform memory synchronization according to the synchronization data packet, and feed back a corresponding synchronization result to the target node after the synchronization is completed.

[0048] The second block state updating module is configured to integrate the obtained synchronization results, and update the physical memory of the target distributed system based on the modified memory block when it is determined that the integrated result meets a second preset node confirmation condition, and update the block state of the modified memory block from the third state to the first state, so as to complete the memory synchronization operation.

[0049] In a third aspect, the present application provides an electronic device, comprising:

[0050] A memory configured to save a computer program;

[0051] A processor configured to execute the computer program to implement the memory synchronization method of the distributed system.

[0052] In a fourth aspect, the present application provides a computer readable storage medium configured to save a computer program; wherein the computer program is executed by a processor to implement the memory synchronization method of the distributed system.

[0053] In the present application, the target memory modification instruction is acquired through the preset interface, and the to-be-modified memory block is modified based on the target memory modification instruction to obtain a modified memory block whose block state is updated from a first state to a second state; the to-be-modified memory block is a memory block in the buffer memory area of the target node, whose block state is the first state and which corresponds to the modification content of the target memory modification instruction; the version vector information of the target node is updated to obtain updated version vector information, a memory synchronization instruction including the updated version vector information is sent to other nodes, and the block state of the modified memory block is updated from the second state to a third state, so that the other nodes perform version vector comparison based on the memory synchronization instruction and feed back the corresponding comparison result to the target node; the other nodes are nodes other than the target node among the plurality of nodes; the comparison results are counted, and when it is determined that the corresponding statistical result meets a first preset node confirmation condition, a synchronization data packet including block information of the modified memory block is sent to the other nodes in a multicast manner, so that the other nodes perform memory synchronization according to the synchronization data packet and feed back the corresponding synchronization result to the target node after completing the synchronization; the obtained synchronization results are integrated, and when it is determined that the obtained integration result meets a second preset node confirmation condition, the physical memory of the target distributed system is updated based on the modified memory block, and the block state of the modified memory block is updated from the third state to the first state to complete the memory synchronization operation. As can be seen from the above, the present application first acquires a target memory modification instruction through a preset interface, modifies the to-be-modified memory block in the buffer memory area of the target node whose block state is the first state according to the target memory modification instruction, updates its state to the second state to obtain a modified memory block, then updates the version vector information of the target node, generates a memory synchronization instruction including the updated version vector information, sends it to other nodes, and updates the block state of the modified memory block to the third state for version vector comparison and result feedback by the other nodes, subsequently counts the corresponding comparison results, when the first preset node confirmation condition is met, multicasts a synchronization data packet including block information of the modified memory block to the other nodes, so that the other nodes feed back the result after completing the synchronization, and finally integrates the synchronization results, if the second preset node confirmation condition is met, updates the physical memory of the system using the modified memory block, and restores its state to the first state to complete the memory synchronization.In this way, through the above process of the application, the traditional memory management function realized by hardware is completely softwareized; the network transmission efficiency is optimized through the intelligent block strategy to reduce the delay; only the modified memory block is synchronized instead of the entire memory region, improving the network utilization; through the explicit memory block state management, version vector comparison and node confirmation mechanism, the consistency and accuracy of memory modification in the distributed system are ensured, the conflicts and errors in the memory synchronization process are reduced, and the multicast packet mode can support dynamic multicast topology, avoiding the limitation of physical wiring, thereby solving the dependence defect of memory synchronization for specific hardware and ensuring the efficiency of memory synchronization. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0055] Figure 1 A flow chart of a memory synchronization method of a distributed system disclosed by the present application;

[0056] Figure 2 A core class structure diagram of a shared memory management system disclosed by the present application;

[0057] Figure 3 A timing diagram of memory data modification and synchronization disclosed by the present application;

[0058] Figure 4 A memory block state conversion diagram disclosed by the present application;

[0059] Figure 5 A memory block state conversion diagram disclosed by the present application;

[0060] Figure 6 A comparison flow diagram of a memory block version vector disclosed by the present application;

[0061] Figure 7 A memory block state conversion diagram disclosed by the present application;

[0062] Figure 8 A flow diagram of application data access disclosed by the present application;

[0063] Figure 9 A memory synchronization device structure diagram of a distributed system disclosed by the present application;

[0064] Figure 10A structure diagram of an electronic device disclosed in the application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the application will be apparently and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative work belong to the protection scope of the application.

[0066] The existing software solutions include a full-quantity synchronization scheme based on TCP / IP, which simulates hardware behavior through a kernel driver, but has high delay; a differential synchronization system based on UDP, which is implemented in a fixed block and point-to-point transmission manner, and has low network utilization; and an industrial protocol scheme, which relies on a special network card to realize deterministic transmission, but is limited in the number of nodes. These schemes respectively have defects of high delay, low network utilization, and limited expansion capability, and the efficiency of memory synchronization is low.

[0067] To overcome the above technical problems, the application provides a memory synchronization method of a distributed system, to solve the defect of dependence on specific hardware for memory synchronization and ensure the efficiency of memory synchronization.

[0068] Referring to Figure 1 As shown in the figure, the embodiment of the application discloses a memory synchronization method of a distributed system, which is applied to a target node in a target distributed system, the target distributed system includes a plurality of nodes, each local node of the plurality of nodes includes a corresponding buffer memory area, and the target node is a node in the plurality of nodes that obtains a target memory modification instruction through a preset interface; wherein the method includes:

[0069] Step S11, obtaining a target memory modification instruction through the preset interface, and modifying a to-be-modified memory block based on the target memory modification instruction to obtain a modified memory block whose block state is updated from a first state to a second state; the to-be-modified memory block is a memory block corresponding to the modification content of the target memory modification instruction and having the first state in the buffer memory area of the target node.

[0070] In this embodiment, after obtaining the target memory modification instruction through the preset interface, the to-be-modified memory block corresponding to the modification content of the instruction and having the first state is found in the buffer memory area of the target node, and the to-be-modified memory block is modified based on the target memory modification instruction, so that the block state of the memory block is updated from the first state to the second state, and the modified memory block is obtained. The preset interface is an application program running on the target node and initiating the target memory modification instruction.

[0071] It should be noted that before the target memory modification instruction is acquired, the embodiment can use a virtualization technology to construct a software-defined shared memory management system, that is, the target distributed system is initialized, and the processing flow is as follows: when the target distributed system is initialized, a preset size of memory space is established for each of the plurality of nodes as a corresponding buffer memory area; the buffer memory area is divided into a plurality of memory blocks based on a preset memory block strategy, and a mapping relationship between the plurality of memory blocks and the physical memory of the target distributed system is established by using a memory mapping mechanism, so as to map the physical memory to the plurality of memory blocks, so that the plurality of nodes perform memory synchronization operations through the plurality of memory blocks. That is, when the target distributed system is initialized, a preset size of memory space is established for each of the plurality of nodes as a respective buffer memory area, for example, a 16 MB (megabyte) buffer area, and then the buffer memory area is divided into a plurality of memory blocks according to a preset memory block strategy, for example, the 16 MB buffer area is divided into 4096 4 KB (kilobyte) memory blocks, and a mapping relationship between the plurality of memory blocks and the physical memory of the system is established by using a memory mapping mechanism, so that the physical memory is mapped to the plurality of memory blocks. A logical unified memory space across nodes is created by using an operating system level memory mapping mechanism, the physical memory is abstracted as a virtual shared area, so that each node performs memory synchronization operations through the memory blocks, each modification operation triggers multicast transmission of the corresponding memory block, and low synchronization delay is achieved. For example, a logical unified address space is created at the operating system kernel layer, and a mmap (memory map) system call is used to map the physical memory to the logical address space. It should be noted that the embodiment can also establish a 128 KB block management table independent of the buffer memory area, so as to manage the plurality of memory blocks. Table 1 below is an example of a block management table structure provided by the present application.

[0072] Table 1 Block management table structure example

[0073]

[0074] It should be further pointed out that, as Figure 2A core class structure diagram of a shared memory management system provided by the present application is shown. Main API (Application Programming Interface) interfaces are provided, including initialization, memory read / write and configuration functions. Among them, HL_SOFT_RFM is the shared memory management system, the MemoryManager class is responsible for memory block management and modification tracking, and the NetworkEngine class handles multicast transmission and data reception. The relationship between the classes shows the modular design of the system, and reflects the division of labor and cooperation of the three core functions of virtual memory management, difference detection and multicast transmission. As shown in Table 2 below is the function information table of the HL_SOFT_RFM class.

[0075] Table 2 Function information table of the HL_SOFT_RFM class

[0076]

[0077] Among them, the config parameter represents the corresponding setting parameter; the offset parameter represents the position offset, which represents the position offset of the data to be written to the corresponding position of the buffer in the SetValue function, and represents the position offset of the data to be read in the buffer in the GetValue function; the value represents the data to be written to the buffer, for example, SetValue(100, 255) represents the operation of setting the data of the 100th byte (from index 0) of the buffer to 255; buffer is a container for storing variables of data read from the buffer, for example, GetValue(100, buffer) represents the operation of reading the data of the 100th byte (from index 0) of the buffer and storing the data into buffer; size represents the size of the set memory. As shown in Table 3 below is the structure information table of the MemoryManager class.

[0078] Table 3 Structure information table of the MemoryManager class

[0079]

[0080] As shown in Table 4 below is the structure information table of the NetworkEngine class.

[0081] Table 4 Structure information table of the NetworkEngine class

[0082]

[0083] It should be noted that, as Figure 3A timing diagram of memory data modification and synchronization provided by the present application is shown. The processing flow of modifying the to-be-modified memory block is as follows: determining the corresponding modification content according to the target memory modification instruction, and the to-be-modified memory block corresponding to the modification content in the buffer memory area of the target node; modifying the corresponding content of the to-be-modified memory block based on the modification content to obtain a corresponding modification result; if the modification result represents that the corresponding content of the to-be-modified memory block fails to be modified, modifying the corresponding content of the to-be-modified memory block again based on the modification content until the modification is successful, updating the block state of the to-be-modified memory block from the first state to the second state to obtain a modified memory block. The first state is an initial block state representing that the memory block is not modified. The second state is a block state representing that the memory block is modified and is to be synchronized. For example, Figure 4 A memory block state conversion diagram provided by the present application is shown. That is, after the target memory modification instruction is intercepted, the corresponding modification content is determined according to the target memory modification instruction, and the to-be-modified memory block corresponding to the modification content in the buffer memory area of the target node is determined. The corresponding content of the to-be-modified memory block is modified based on the modification content to obtain a modification result. If the modification result represents that the modification fails, the modification is performed again according to the modification content until the modification is successful. Then, the block state of the to-be-modified memory block is updated from the initial first state representing that the memory block is not modified to the second state representing that the memory block is modified and is to be synchronized to obtain a modified memory block. It should be noted that the system of the present embodiment has the function of concurrent processing, for example, Figure 5The memory block state conversion diagram provided by the application is shown. If a new modification occurs in the synchronization process, continuous modification operation can be performed, and the efficiency of memory synchronization is improved. In this way, the embodiment creates a logical unified memory space across nodes through the memory mapping mechanism of the operating system, maps the buffer memory area and the underlying physical memory, implements the unification of the address space across processes, ensures that the reading and writing of the application program to the buffer area can be efficiently mapped to actual physical memory operation, adopts the design of the buffer area, avoids the low efficiency caused by network delay, node communication overhead and other reasons when directly operating the physical memory through the network, directly accesses the local buffer area, reduces the number of cross-network request times, ensures the memory access efficiency, allocates a dedicated buffer memory area for each node for local data temporary storage, network transmission preprocessing and the like, avoids the resource competition between nodes, clearly records the modification state of the memory block with the help of state updating, provides an explicit state basis for subsequent memory synchronization and the like, helps to maintain the orderliness and accuracy of the memory data in the distributed system, adopts a retry mechanism for modification failure, guarantees the final success of the modification, supports concurrent processing of modification, improves the efficiency of memory synchronization, constructs a transparent memory access interface, enables the application program to access the distributed shared memory like operating the local memory, without needing to care about the underlying network transmission and synchronization details, not only maintains high compatibility with the reflective memory card driver API, facilitates the migration of existing applications, and increases more flexible memory management and configuration functions.

[0084] In step S12, the version vector information of the target node is updated to obtain updated version vector information, memory synchronization instructions including the updated version vector information are sent to other nodes, and the block state of the modified memory block is updated from the second state to a third state, so that the other nodes perform version vector comparison based on the memory synchronization instructions, and the corresponding comparison results are fed back to the target node. The other nodes are other nodes in the plurality of nodes except the target node.

[0085] In the embodiment, considering that the data synchronization is performed in the UDP multicast mode, it cannot be guaranteed that the data packets arrive in order, and in addition, due to network fluctuations, when the data is frequently modified, the data of the first data packet may arrive late, so it is necessary to add a version vector for confirmation. Specifically, the version vector information of the target node is updated to obtain updated version vector information, memory synchronization instructions including the information are sent to other nodes in the plurality of nodes except the target node, and the block state of the modified memory block is updated from the second state of the modified to-be-synchronized state to a third state, so that the other nodes perform version vector comparison based on the memory synchronization instructions, and the comparison results are fed back to the target node.

[0086] It should be noted that each memory block has a 4-byte version number, and is globally incremented, and the embodiment can maintain a version vector for each node in the block management table, for recording the version number of each memory block on the node. When performing data synchronization, the vector comparison can be used to determine the memory block to be synchronized, such as Figure 6 The comparison process of the memory block version vector provided by the application is shown in the figure. The processing flow of the version vector comparison of the other node is as follows: obtaining the memory synchronization instruction sent by the target node; determining the updated version vector in the memory synchronization instruction, and determining the version vector to be compared of the other node; comparing the version vector to be compared with the updated version vector to obtain the corresponding comparison result; determining the target version number in the version vector to be compared, which is inconsistent with the corresponding version number in the updated version vector based on the comparison result, and updating the version vector to be compared; determining the corresponding memory block to be synchronized according to the target version number, and feeding back the comparison result to the target node. That is, the other node first obtains the memory synchronization instruction, determines the updated version vector therein and the version vector to be compared of itself, compares the two to obtain a comparison result, finds out a target version number in the version vector to be compared which is inconsistent with the corresponding version number in the updated version vector based on the comparison result, updates the version vector to be compared, and determines the corresponding memory block to be synchronized according to the target version number, and feeds back the comparison result to the target node. In this way, the embodiment takes into account the characteristics of UDP data transmission and the reasons for network fluctuations, and sends the memory synchronization instruction containing the version vector information to the other node before sending the data packet, uses the version vector to confirm, provides a verification basis for the accuracy of data synchronization, and improves the accuracy of memory synchronization; by updating the version vector information and sending the synchronization instruction, the other node can know the memory modification of the target node in time; updating the block state of the memory block to the third state marks that the memory block has entered the synchronization interaction stage, which helps the nodes to cooperatively complete the consistency maintenance of the memory data; the version vector comparison accurately locates the version number with differences, and then determines the memory block to be synchronized, ensuring the pertinence of the synchronization operation, avoiding resource waste caused by indiscriminate synchronization, and reducing network load.

[0087] Step S13, statistics are performed on the comparison result, and when it is determined that the corresponding statistical result meets the first preset node confirmation condition, a synchronization data packet including the block information of the modified memory block is sent to the other node in a multicast manner, so that the other node performs memory synchronization according to the synchronization data packet, and feeds back the corresponding synchronization result to the target node after completing the synchronization.

[0088] In the embodiment, the comparison results are counted, and when the counting result meets the first preset node confirmation condition, a synchronization data packet containing the modified memory block information is sent to the other nodes by multicast mode, so that the other nodes perform memory synchronization according to the synchronization data packet, and the synchronization result is fed back to the target node after the synchronization is completed. The synchronization data packet includes a header, the block information, data load and checksum. The header contains basic information of the synchronization data packet, and is used for the receiving party to quickly identify and analyze the synchronization data packet. The block information is used to describe the specific attributes of the memory block, and helps the receiving party to locate and judge the new and old degree of the data. The data load includes the actual memory block data that needs to be synchronized. The checksum is used to verify whether the synchronization data packet has an error in the transmission process. Table 5 shows a header structure table of a synchronization data packet provided by the present application. Table 6 shows a block information structure table of a synchronization data packet provided by the present application. Table 7 shows a data load structure table of a synchronization data packet provided by the present application. Table 8 shows a checksum structure table of a synchronization data packet provided by the present application.

[0089] Table 5 Checksum structure table of synchronization data packet

[0090]

[0091] Table 6 Checksum structure table of synchronization data packet

[0092]

[0093] Table 7 Checksum structure table of synchronization data packet

[0094]

[0095] Table 8 Checksum structure table of synchronization data packet

[0096]

[0097] It should be noted that the target node processes as follows after obtaining the comparison result: obtaining the comparison result fed back by the other nodes, and performing statistics on the comparison result to obtain a corresponding statistical result; the statistical result includes a first result quantity of the comparison result; determining the node quantity of the other nodes, and comparing the first result quantity in the statistical result with the node quantity; if the first result quantity is consistent with the node quantity, it is determined that the statistical result meets a first preset node confirmation condition, a synchronization data packet including the block information of the modified memory block is sent to the other nodes in a multicast manner, and the transmission situation of the synchronization data packet is continuously monitored; if the transmission situation represents that the synchronization data packet is lost, the synchronization data packet is retransmitted. That is, as shown in Figure 7 A memory block state conversion schematic diagram provided by the present application is shown. The target node obtains the comparison result, and performs statistics thereon to obtain a statistical result including a first result quantity thereof, then determines the node quantity of the other nodes, compares the first result quantity with the node quantity, if both are consistent, it indicates that the other nodes have all received the memory synchronization instruction, it is determined that the statistical result meets a first preset node confirmation condition, then a synchronization data packet including the block information of the modified memory block is sent to the other nodes in a multicast manner, and the data packet transmission situation is continuously monitored, if there is a loss, retransmission operation is performed.

[0098] It should be further pointed out that the processing flow of the other nodes after receiving the synchronization data packet is as follows: receiving the synchronization data packet sent by the target node, and checking the synchronization data packet to obtain a corresponding check result; if the check result represents that the synchronization data packet passes the check, the synchronization data packet is parsed to obtain the block information in the synchronization data packet; based on the block information, the memory synchronization operation is performed on the to-be-synchronized data block, and after the synchronization is completed, the corresponding synchronization result is fed back to the target node. That is, after receiving the synchronization data packet, it is checked, if the check passes, the synchronization data packet is parsed to obtain the block information therein, based on the block information, the memory synchronization operation is performed on the to-be-synchronized data block, and after the synchronization is completed, the synchronization result is fed back to the target node. It should be pointed out that before the data packet is multicast, the embodiment can first construct a dynamic multicast group based on the IGMPv3 protocol (a network communication protocol), when sending, the transmission layer is encapsulated by the UDP protocol, and the application layer adds the RMCP (Remote Management Control Protocol) header (including the version number, timestamp, and retry number field). At the same time, since the UDP multicast has no built-in reliability mechanism, the sender cannot know whether the receiver has received the data, therefore, the receiving node can send a NACK (negative acknowledgement) packet through a reverse channel to trigger selective retransmission. In this way, the embodiment ensures the full-node confirmation of the data synchronization instruction by counting the comparison results and setting the confirmation condition, avoids the occurrence of inconsistent data and chaotic node version number, improves the reliability; the data packet is transmitted in a multicast manner, improving the efficiency of data packet transmission; the other nodes feed back the synchronization result, so that the target node can know the synchronization completion in time, forming a complete closed loop, further ensuring the comprehensiveness and effectiveness of the memory synchronization in the distributed system, and improving the collaborative efficiency and stability of the system; the packet retransmission mechanism ensures the integrity of data transmission, avoiding the synchronization failure of other nodes due to transmission problems; the other nodes check the synchronization data packet after receiving it, ensuring the integrity and accuracy of the received data, and avoiding the interference of invalid or error data on the memory synchronization.

[0099] In step S14, the obtained synchronization result is integrated, and when the obtained integration result meets the second preset node confirmation condition, the physical memory of the target distributed system is updated based on the modified memory block, and the block state of the modified memory block is updated from the third state to the first state, to complete the memory synchronization operation.

[0100] In this embodiment, the synchronization results obtained from the other nodes are integrated, and when it is determined that the integrated result meets the second preset node confirmation condition, the physical memory of the target distributed system is updated based on the modified memory block, and the block state is updated from the third state to the first state, thereby completing the memory synchronization operation.

[0101] Specifically, the synchronization results fed back by the other nodes are obtained, and the synchronization results are integrated to obtain a corresponding integrated result; the integrated result includes a second result quantity of the synchronization results; the number of nodes of the other nodes is determined, and the second result quantity in the integrated result is compared with the number of nodes; if the second result quantity is not less than half of the number of nodes, it is determined that the integrated result meets the second preset node confirmation condition, and the physical memory of the target distributed system is updated based on the modified memory block; the block state of the modified memory block is updated from the third state to the first state, thereby completing the memory synchronization operation; the third state is a block state representing that a memory block is in synchronization. That is, the synchronization results fed back by the other nodes are obtained and integrated to obtain an integrated result including a second result quantity of the results, after determining the number of nodes of the other nodes, the second result quantity is compared with the number of nodes, if it is not less than half of the number of nodes, it means that the synchronization data packet has been received by more than half of the nodes, and it can be basically determined that the data can be ultimately spread to the whole cluster through secondary synchronization between nodes (such as unconfirmed nodes can obtain from confirmed nodes subsequently), it is determined that the integrated result meets the second preset node confirmation condition, the physical memory of the target distributed system is updated based on the modified memory block, and the block state is updated from the third state representing synchronization to the first state, thereby completing the memory synchronization operation.

[0102] It should be noted that the process of updating the physical memory is as follows: according to the mapping relationship, the target physical memory corresponding to the modified memory block in the physical memory of the target distributed system is determined; and the target physical memory is updated by using the modification content of the modified memory block. That is, according to the mapping relationship, the target physical memory for saving the content of the modified memory block in the target distributed system is determined, and the target physical memory is updated by using the modification content of the modified memory block, the physical memory part to be updated is accurately located through the mapping relationship, the accuracy of the physical memory update is ensured, and the misoperation on irrelevant memory areas is avoided; the content of the modified memory block is updated, the consistency of the physical memory data and the modification result is ensured, and the physical memory of the distributed system can be synchronized with the modification of the memory block in time.

[0103] It needs to be further pointed out that the embodiment can also monitor the memory modification in real time and dynamically adjust the granularity of the block. The processing flow is as follows: continuously record the modification information corresponding to each of the memory blocks in the buffer memory area; generate a memory access heat map for the memory blocks based on the preset heat map generation frequency; and adjust the block granularity of the buffer memory area according to the memory access heat map, so as to adjust the memory blocks in the buffer memory area according to the block granularity. The modification information includes the modification times or frequency of the memory blocks and the time of the last modification. The block granularity can include three levels of 1KB, 4KB and 16KB. That is, the modification information corresponding to each of the memory blocks in the buffer memory area is continuously recorded, a memory access heat map for the memory blocks is generated based on a preset heat map generation frequency, for example, 1ms (millisecond) period, and the block granularity of the buffer memory area is dynamically adjusted according to the heat map, and then the memory blocks in the buffer memory area are adjusted according to the adjusted block granularity. Specifically, the memory blocks with high access can be finely divided, and the memory blocks with low access can be coarsely divided.

[0104] It needs to be pointed out that the application program is a software process running on the node, which accesses the virtual shared area through the API provided by the system, such as the API for reading and writing shared memory. Figure 8 Fig. 2 shows a flowchart of application data access provided by the present application. According to the operation type called by the application program, the corresponding access flow is performed. If the operation type is a write type, the memory modification and synchronization operation are performed by the above method. If the operation type is a read type, the corresponding query data is obtained from the local memory of the target node, and the query data is returned to the application program. In this way, the embodiment sets the condition that the second result quantity is not less than half of the node quantity as the confirmation standard, which ensures that most nodes have completed synchronization and the data can be spread to the whole cluster through the secondary synchronization between nodes, reduces the network overhead and delay, and balances the reliability and efficiency. The natural defect of UDP is avoided, the advantage of efficient transmission is retained, and the synchronization efficiency is improved. The state of the memory block is restored to the initial first state, which prepares for the next memory operation and forms a complete closed-loop management, effectively ensuring the consistency of the physical memory data and the synchronization result of each node, and enhancing the reliability and flexibility of the system memory synchronization. The modification information is continuously recorded and the memory access heat map is generated, which can intuitively reflect the access heat and modification frequency of each memory block. Based on this, the block granularity is adjusted, the division of the buffer memory area is more in line with the actual access demand, the space allocation and use efficiency of the buffer memory area are optimized, the flexibility and adaptability of the memory management in the distributed system are enhanced, and the efficient memory operation is ensured.

[0105] From the above, the embodiment of the application first obtains a target memory modification instruction through a preset interface, modifies a memory block in a buffer memory area of a target node corresponding to the first state to be modified according to the target memory modification instruction, updates the state of the memory block to the second state, obtains a modified memory block, then updates the version vector information of the target node, generates a memory synchronization instruction containing the updated version vector information, sends the memory synchronization instruction to other nodes, and updates the block state of the modified memory block to the third state, so that the other nodes perform version vector comparison and feed back the results, then statistics the corresponding comparison results, multicasts a synchronization data packet containing the block information of the modified memory block to the other nodes when the first preset node confirmation condition is met, so that the other nodes complete synchronization and feed back the results, finally integrates the synchronization results, and if the second preset node confirmation condition is met, updates the physical memory of the system by using the modified memory block and restores the state of the modified memory block to the first state, and completes this memory synchronization. In this way, through the above process of the embodiment of the application, on the one hand, a logical unified memory space across nodes is created through a memory mapping mechanism of an operating system, a buffer memory area is mapped with an underlying physical memory to realize unification across process address spaces, and efficient mapping of reading and writing of an application program on a buffer area to actual physical memory operation is ensured; on the other hand, the design of the buffer area is adopted to avoid low efficiency caused by network delay, node communication overhead and the like when directly operating the physical memory through the network, local buffer areas are directly accessed, the number of cross-network request times is reduced, and memory access efficiency is ensured; on the other hand, a dedicated buffer memory area is allocated for each node for local data temporary storage, network transmission preprocessing and the like, and resource competition among nodes is avoided; on the other hand, the state is updated to clearly record the modification state of the memory block, and an explicit state basis is provided for subsequent memory synchronization and the like, which is helpful to maintaining the orderliness and accuracy of memory data in a distributed system; on the other hand, a transparent memory access interface is constructed, an application program can access distributed shared memory like operating local memory, without needing to care about underlying network transmission and synchronization details, high compatibility with a reflective memory card driver API is maintained, existing application migration is facilitated, and more flexible memory management and configuration functions are increased; on the other hand, considering the characteristics of UDP data transmission and network fluctuation, a memory synchronization instruction containing version vector information is first sent to other nodes before sending a data packet, version vector is used for confirmation, a check basis is provided for the accuracy of data synchronization, and the accuracy of memory synchronization is improved; on the other hand, the version vector information is updated and the synchronization instruction is sent, so that the other nodes can know the memory modification of the target node in time; the block state of the memory block is updated to the third state, the memory block is marked as entering the synchronization interaction stage, and this is helpful to collaborative maintenance of memory data consistency by nodes;On the one hand, the version number with differences is accurately located through the comparison of the version vector, and then the memory block to be synchronized is determined, thereby ensuring the pertinence of the synchronization operation, avoiding the resource waste caused by indiscriminate synchronization, and reducing the network load. On the one hand, the data synchronization instruction is ensured to be confirmed by all nodes by counting the comparison results and setting the confirmation condition, thereby avoiding the occurrence of inconsistent data and chaotic version numbers of nodes, and improving the reliability. On the one hand, the multicast mode is used for data packet transmission, thereby improving the efficiency of data packet transmission. On the one hand, the other nodes feed back the synchronization results, so that the target node can know the synchronization completion in time, form a complete closed loop, and further ensure the comprehensiveness and effectiveness of the memory synchronization in the distributed system, and improve the collaborative efficiency and stability of the system. On the one hand, the data transmission integrity is ensured by the packet loss retransmission mechanism, thereby avoiding the synchronization failure of other nodes caused by transmission problems. On the one hand, the other nodes check the received data after receiving the synchronization data packet, thereby ensuring the integrity and accuracy of the received data, and avoiding the interference of invalid or error data on the memory synchronization. On the one hand, the condition that the second result quantity is not less than half of the node quantity is set as the confirmation standard, thereby ensuring that most nodes have completed synchronization, and the data can be basically determined to be spread to the whole cluster through secondary synchronization between nodes, thereby reducing the network overhead and delay, balancing the reliability and efficiency, avoiding the system collapse risk caused by the natural defects of UDP, retaining the advantages of efficient transmission of UDP, and improving the synchronization efficiency. On the one hand, the state of the memory block is restored to the initial first state, thereby preparing for the next memory operation, forming a complete closed loop management, and effectively ensuring the consistency of the physical memory data and the synchronization results of the nodes. On the one hand, the modification information is continuously recorded and the memory access heat map is generated, thereby directly reflecting the access heat and modification frequency of each memory block. Based on this, the blocking granularity is adjusted, so that the division of the buffer memory area is more in line with the actual access demand, the space allocation and use efficiency of the buffer memory area are optimized, the flexibility and adaptability of the memory management in the distributed system are enhanced, and the efficient memory operation is ensured. On the other hand, the traditional memory management function realized by hardware is completely software, thereby solving the dependence defect of the memory synchronization on specific hardware and ensuring the efficiency of the memory synchronization.

[0106] Correspondingly, referring to Figure 9 The embodiment of the application also provides a memory synchronization device of a distributed system, which is applied to a target node in a target distributed system, the target distributed system comprising a plurality of nodes, each of the plurality of nodes comprising a corresponding buffer memory area in the local node, and the target node being a node in the plurality of nodes which obtains a target memory modification instruction through a preset interface; wherein the device comprises:

[0107] The memory block modification module 11 is configured to acquire a target memory modification instruction through the preset interface, and modify a to-be-modified memory block based on the target memory modification instruction to obtain a modified memory block whose block state is updated from a first state to a second state; the to-be-modified memory block is a memory block corresponding to a modification content of the target memory modification instruction and having the first state in the buffer memory area of the target node.

[0108] The first block state updating module 12 is configured to update version vector information of the target node to obtain updated version vector information, send a memory synchronization instruction including the updated version vector information to other nodes, and update a block state of the modified memory block from the second state to a third state, so that the other nodes perform version vector comparison based on the memory synchronization instruction and feed back corresponding comparison results to the target node; the other nodes are nodes other than the target node in the plurality of nodes.

[0109] The data packet sending module 13 is configured to count the comparison results, and when it is determined that a corresponding counting result meets a first preset node confirmation condition, send a synchronization data packet including block information of the modified memory block to the other nodes in a multicast manner, so that the other nodes perform memory synchronization according to the synchronization data packet, and feed back corresponding synchronization results to the target node after completing the synchronization.

[0110] The second block state updating module 14 is configured to integrate the obtained synchronization results, and when it is determined that an integrated result meets a second preset node confirmation condition, update a physical memory of the target distributed system based on the modified memory block, and update the block state of the modified memory block from the third state to the first state, to complete the memory synchronization operation.

[0111] As can be seen from the above, the embodiment of the present application first acquires a target memory modification instruction through a preset interface, modifies a memory block in a corresponding buffer memory area of a target node in a first state to be modified according to the target memory modification instruction, updates the state of the memory block to a second state, to obtain a modified memory block, then updates version vector information of the target node, generates a memory synchronization instruction containing the updated version vector information, sends the memory synchronization instruction to other nodes, and updates the block state of the modified memory block to a third state, so that the other nodes perform version vector comparison and feed back results, then statistics corresponding comparison results, multicasts a synchronization data packet containing block information of the modified memory block to the other nodes when a first preset node confirmation condition is met, so that the other nodes complete synchronization and feed back results, and finally integrates synchronization results, and if a second preset node confirmation condition is met, updates a physical memory of the system by using the modified memory block and restores the state of the modified memory block to the first state, to complete this memory synchronization. In this way, through the above process of the embodiment of the present application, the memory management function realized by hardware in the prior art is completely softwareized; the network transmission efficiency is optimized by using an intelligent block division strategy to reduce delay; only the modified memory block is synchronized instead of the entire memory region, to improve network utilization; the consistency and accuracy of memory modification in a distributed system are ensured by using a clear memory block state management, version vector comparison and node confirmation mechanism, to reduce conflicts and errors in the memory synchronization process, and the multicast data packet mode can support a dynamic multicast topology, to avoid the limitation of physical wiring, thereby solving the dependence defect of memory synchronization on specific hardware and ensuring the efficiency of memory synchronization.

[0112] In some specific embodiments, the memory synchronization method of the distributed system can further include:

[0113] a space establishing unit, configured to establish a preset size of memory space as a corresponding buffer memory area for each of the nodes when the target distributed system is initialized;

[0114] a relationship establishing unit, configured to divide the buffer memory area into a plurality of memory blocks based on a preset memory block division strategy, and establish a mapping relationship between the plurality of memory blocks and the physical memory of the target distributed system by using a memory mapping mechanism, to map the physical memory to the plurality of memory blocks, so that the plurality of nodes perform memory synchronization operations through the plurality of memory blocks;

[0115] Correspondingly, the second block state updating module 14 can specifically include:

[0116] a memory determining unit, configured to determine a target physical memory corresponding to the modified memory block in the physical memory of the target distributed system according to the mapping relationship;

[0117] The first memory updating unit is configured to update the target physical memory by using the modification content of the modified memory block.

[0118] In some embodiments, the memory block modification module 11 can specifically include:

[0119] The memory block determination unit is configured to determine the modification content according to the target memory modification instruction, and determine the to-be-modified memory block corresponding to the modification content in the buffer memory area of the target node.

[0120] The content modification unit is configured to modify the corresponding content of the to-be-modified memory block based on the modification content to obtain a corresponding modification result.

[0121] The first state updating unit is configured to modify the corresponding content of the to-be-modified memory block again based on the modification content until the modification is successful if the modification result indicates that the modification of the corresponding content of the to-be-modified memory block fails, update the block state of the to-be-modified memory block from the first state to a second state, and obtain a corresponding modified memory block; the first state is an initial block state indicating that the memory block is not modified; and the second state is a block state indicating that the memory block is modified and is to be synchronized.

[0122] In some embodiments, the first block state updating module 12 can specifically include:

[0123] The instruction acquisition unit is configured to acquire the memory synchronization instruction sent by the target node.

[0124] The vector determination unit is configured to determine the updated version vector in the memory synchronization instruction and determine the to-be-compared version vector of the other node.

[0125] The vector comparison unit is configured to compare the to-be-compared version vector and the updated version vector to obtain a corresponding comparison result.

[0126] The vector updating unit is configured to determine a target version number in the to-be-compared version vector, in which the version number is inconsistent with the corresponding version number in the updated version vector, based on the comparison result, and update the to-be-compared version vector.

[0127] The comparison result feedback unit is configured to determine a to-be-synchronized memory block according to the target version number, and feed back the comparison result to the target node.

[0128] Correspondingly, the data packet sending module 13 can specifically include:

[0129] A data packet checking unit is configured to receive the synchronization data packet sent by the target node and check the synchronization data packet to obtain a corresponding checking result.

[0130] A data packet analyzing unit is configured to analyze the synchronization data packet to obtain the block information in the synchronization data packet if the checking result indicates that the synchronization data packet passes the check.

[0131] A synchronization result feedback unit is configured to perform a memory synchronization operation on the data block to be synchronized based on the block information and feed back a corresponding synchronization result to the target node after the synchronization is completed.

[0132] In some embodiments, the data packet sending module 13 specifically can include:

[0133] A result statistics unit is configured to obtain the comparison result fed back by the other nodes and perform statistics on the comparison result to obtain a corresponding statistics result; the statistics result includes a first result number of the comparison result.

[0134] A first number comparison unit is configured to determine the number of nodes of the other nodes and compare the first result number in the statistics result with the number of nodes.

[0135] A data packet sending unit is configured to determine that the statistics result meets a first preset node confirmation condition if the first result number is consistent with the number of nodes, send a synchronization data packet including the block information of the modified memory block to the other nodes in a multicast manner, and continuously monitor the transmission situation of the synchronization data packet.

[0136] A data packet retransmission unit is configured to perform a retransmission operation on the synchronization data packet if the transmission situation indicates that the synchronization data packet is lost.

[0137] In some embodiments, the second block state updating module 14 specifically can include:

[0138] A result integration unit is configured to obtain the synchronization result fed back by the other nodes and integrate the synchronization result to obtain a corresponding integration result; the integration result includes a second result number of the synchronization result.

[0139] A second number comparison unit is configured to determine the number of nodes of the other nodes and compare the second result number in the integration result with the number of nodes.

[0140] a second memory updating unit, configured to determine that the integrated result meets a second preset node confirmation condition if the second result quantity is not less than half of the node quantity, and update the physical memory of the target distributed system based on the modified memory block;

[0141] a second state updating unit, configured to update a block state of the modified memory block from the third state to the first state, so as to complete the memory synchronization operation in this time;

[0142] In some specific embodiments, the memory synchronization apparatus of the distributed system can further include:

[0143] an information recording unit, configured to continuously record modification information corresponding to each of the plurality of memory blocks in the buffer memory area;

[0144] a heat map generating unit, configured to generate a memory access heat map for the plurality of memory blocks based on the modification information according to a preset heat map generation frequency;

[0145] a granularity adjusting unit, configured to adjust a block granularity of the buffer memory area according to the memory access heat map, and adjust the plurality of memory blocks in the buffer memory area according to the block granularity.

[0146] Further, the embodiment of the present application further discloses an electronic device, Figure 10 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the use range of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps in the memory synchronization method of the distributed system disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment can be an electronic computer.

[0147] In the embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited here; the input / output interface 25 is used to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited here.

[0148] In addition, the memory 22 can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc. as a carrier for storing resources, and the resources stored thereon can include an operating system 221, a computer program 222, etc. The storage mode can be temporary storage or permanent storage.

[0149] The operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the memory synchronization method of the distributed system executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0150] Further, the present application also discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the memory synchronization method of the distributed system disclosed above. For the specific steps of the method, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be described here.

[0151] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the same or similar parts between the embodiments, please refer to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and please refer to the method part for the relevant content.

[0152] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of the examples have been described in the above description. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0153] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0154] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the stated element.

[0155] The above detailed description of the technical solutions provided by the present application has been provided, and the principles and implementation manners of the present application have been described by applying specific examples. The above description of the examples is only for the purpose of helping to understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description of the content of the specification should not be understood as a limitation of the present application.

Claims

1. A memory synchronization method for a distributed system, characterized in that, The method is applied to a target node in a target distributed system, wherein the target distributed system includes several nodes, each of which has a corresponding buffer memory area in its local memory, and the target node is the node among the several nodes that obtains a target memory modification instruction through a preset interface; wherein, the method includes: The target memory modification instruction is obtained through the preset interface, and the memory block to be modified is modified based on the target memory modification instruction to obtain the modified memory block whose block status is updated from the first state to the second state; the memory block to be modified is the memory block in the buffer memory area of ​​the target node that corresponds to the modification content of the target memory modification instruction and whose block status is the first state. The version vector information of the target node is updated to obtain the updated version vector information. A memory synchronization instruction including the updated version vector information is sent to other nodes, and the block status of the modified memory block is updated from the second state to the third state, so that the other nodes can perform version vector comparison based on the memory synchronization instruction and feed back the corresponding comparison results to the target node; the other nodes are the nodes other than the target node among the plurality of nodes. The comparison results are statistically analyzed, and after determining that the corresponding statistical results meet the first preset node confirmation conditions, the synchronization data packet including the block information of the modified memory block is sent to the other nodes via multicast, so that the other nodes can perform memory synchronization according to the synchronization data packet, and after the synchronization is completed, the corresponding synchronization results are fed back to the target node. The obtained synchronization results are integrated, and when the obtained integration results meet the second preset node confirmation condition, the physical memory of the target distributed system is updated based on the modified memory block, and the block status of the modified memory block is updated from the third state to the first state to complete this memory synchronization operation. The process of integrating the obtained synchronization results, and when the obtained integration results meet the second preset node confirmation condition, updating the physical memory of the target distributed system based on the modified memory blocks, and updating the block status of the modified memory blocks from the third state to the first state to complete this memory synchronization operation, includes: The synchronization results fed back by the other nodes are obtained, and the synchronization results are integrated to obtain a corresponding integrated result; the integrated result includes the second result number of the synchronization results; Determine the number of nodes in the other nodes, and compare the number of the second result in the integration result with the number of nodes; If the number of the second result is not less than half of the number of nodes, then the integration result is determined to meet the second preset node confirmation condition, and the physical memory of the target distributed system is updated based on the modified memory block; The block state of the modified memory block is updated from the third state to the first state to complete this memory synchronization operation; the third state is the block state that indicates that the memory block is being synchronized.

2. The memory synchronization method for a distributed system according to claim 1, characterized in that, Before obtaining the target memory modification instruction through the preset interface, the method further includes: During the initialization of the target distributed system, a corresponding preset size memory space is established for each of the several nodes as the corresponding buffer memory area; The buffer memory area is divided into several memory blocks based on a preset memory block partitioning strategy, and a memory mapping mechanism is used to establish a mapping relationship between the several memory blocks and the physical memory of the target distributed system, so as to map the physical memory to the several memory blocks, so that the several nodes can perform memory synchronization operations through the several memory blocks. Accordingly, updating the physical memory of the target distributed system based on the modified memory block includes: Based on the mapping relationship, determine the target physical memory in the physical memory of the target distributed system that corresponds to the modified memory block; The target physical memory is updated using the modified content of the modified memory block.

3. The memory synchronization method for a distributed system according to claim 1, characterized in that, The step of modifying the memory block to be modified based on the target memory modification instruction to obtain a modified memory block whose block state is updated from the first state to the second state includes: The target memory modification instruction determines the corresponding modification content, and the memory block to be modified in the buffer memory area of ​​the target node that corresponds to the modification content; Based on the modified content, the corresponding content of the memory block to be modified is modified to obtain the corresponding modification result; If the modification result indicates that the corresponding content of the memory block to be modified has failed to be modified, then the corresponding content of the memory block to be modified is modified again based on the modified content until the modification is successful. The block status of the memory block to be modified is updated from the first state to the second state to obtain the corresponding modified memory block. The first state is the initial block status indicating that the memory block has not been modified. The second state is the block status indicating that the memory block has been modified and is waiting for memory synchronization.

4. The memory synchronization method for a distributed system according to claim 1, characterized in that, The step of performing version vector comparison based on the memory synchronization instructions and feeding back the corresponding comparison results to the target node includes: Obtain the memory synchronization instruction sent by the target node; Determine the updated version vector in the memory synchronization instruction, and determine the version vectors to be compared for the other nodes; The version vector to be compared and the updated version vector are compared to obtain the corresponding comparison results; Based on the comparison results, the target version number in the version vector to be compared that is inconsistent with the corresponding version number in the updated version vector is determined, and the version vector to be compared is updated. The corresponding memory block to be synchronized is determined based on the target version number, and the comparison result is fed back to the target node; Accordingly, the step of performing memory synchronization based on the synchronization data packet, and feeding back the corresponding synchronization result to the target node after synchronization is completed, includes: The system receives the synchronization data packet sent by the target node and verifies the synchronization data packet to obtain the corresponding verification result. If the verification result indicates that the synchronization data packet passes the verification, then the synchronization data packet is parsed to obtain the block information in the synchronization data packet; Based on the block information, a memory synchronization operation is performed on the data block to be synchronized, and after the synchronization is completed, the corresponding synchronization result is fed back to the target node.

5. The memory synchronization method for a distributed system according to claim 1, characterized in that, The step of performing statistical analysis on the comparison results, and after determining that the corresponding statistical results meet the first preset node confirmation condition, sending a synchronization data packet including the block information of the modified memory block to the other nodes via multicast, includes: Obtain the comparison results fed back by the other nodes, and perform statistics on the comparison results to obtain corresponding statistical results; the statistical results include the first result number of the comparison results; Determine the number of nodes in the other nodes, and compare the first result number in the statistical results with the number of nodes; If the number of the first results is consistent with the number of nodes, then the statistical results are determined to meet the first preset node confirmation condition. The synchronization data packet including the block information of the modified memory block is sent to the other nodes via multicast, and the transmission status of the synchronization data packet is continuously monitored. If the transmission status indicates that the synchronization data packet has been lost, then the synchronization data packet will be retransmitted.

6. The memory synchronization method for a distributed system according to any one of claims 1 to 5, characterized in that, Also includes: Continuously record the modification information corresponding to each of the several memory blocks in the buffer memory area; Based on a preset heatmap generation frequency, a memory access heatmap for the plurality of memory blocks is generated using the modified information. The block granularity of the buffer memory region is adjusted according to the memory access heatmap, so as to adjust the plurality of memory blocks in the buffer memory region according to the block granularity.

7. A memory synchronization device for a distributed system, characterized in that, An apparatus is applied to a target node in a target distributed system, wherein the target distributed system includes several nodes, each of which has a corresponding buffer memory area in its local memory, and the target node is the node among the several nodes that obtains a target memory modification instruction through a preset interface; wherein the apparatus includes: The memory block modification module is used to obtain a target memory modification instruction through the preset interface, and modify the memory block to be modified based on the target memory modification instruction to obtain a modified memory block whose block status is updated from a first state to a second state; the memory block to be modified is the memory block in the buffer memory area of ​​the target node that corresponds to the modification content of the target memory modification instruction and whose block status is the first state. The first state update module is used to update the version vector information of the target node to obtain the updated version vector information, send a memory synchronization instruction including the updated version vector information to other nodes, and update the block status of the modified memory block from the second state to the third state, so that the other nodes can perform version vector comparison based on the memory synchronization instruction and feed back the corresponding comparison results to the target node; the other nodes are the nodes other than the target node among the plurality of nodes; The data packet sending module is used to perform statistics on the comparison results, and after determining that the corresponding statistical results meet the first preset node confirmation conditions, it sends the synchronization data packet including the block information of the modified memory block to the other nodes via multicast, so that the other nodes can perform memory synchronization according to the synchronization data packet, and after completing the synchronization, feed back the corresponding synchronization results to the target node. The second state update module is used to integrate the obtained synchronization results, and when it is determined that the obtained integration results meet the second preset node confirmation conditions, it updates the physical memory of the target distributed system based on the modified memory block, and updates the block state of the modified memory block from the third state to the first state to complete this memory synchronization operation. The second state update module includes: The result integration unit is used to obtain the synchronization results fed back by the other nodes and integrate the synchronization results to obtain a corresponding integration result; the integration result includes the second result number of the synchronization results; The second quantity comparison unit is used to determine the number of nodes of the other nodes and compare the second result quantity in the integration result with the number of nodes; The second memory update unit is used to determine that the integration result meets the second preset node confirmation condition if the number of the second result is not less than half of the number of nodes, and to update the physical memory of the target distributed system based on the modified memory block. The second state update unit is used to update the block state of the modified memory block from the third state to the first state to complete the memory synchronization operation; the third state is the block state that indicates that the memory block is being synchronized.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the memory synchronization method for a distributed system as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the memory synchronization method of a distributed system as described in any one of claims 1 to 6.

Images