Mirror management method, distributed system, electronic device and medium

Abstract

The application discloses a mirror image management method, a distributed system, an electronic device and a storage medium. The method comprises the following steps: determining that a mirror image generation condition is met, generating a mirror image and a mirror image index based on a local applied log, and updating a mapping relationship between an identifier of a first node and the mirror image index which is maintained locally by using the generated mirror image index; generating a first type of log by using the mirror image index and the identifier of the first node, and sending the first type of log to a shared log system, so that after other nodes in a distributed system read the first type of log, a mapping relationship between a node identifier and the mirror image index which is maintained on the other nodes is updated, thereby realizing a basic function of mirror image management of the distributed system and reducing the difficulty of implementing mirror image management by using a state machine. Moreover, since the mirror image generation is performed only when it is determined that the mirror image generation condition is met, system performance fluctuation caused by frequent mirror image generation can be prevented.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an image management method, a distributed system, an electronic device and a medium. BACKGROUND

[0002] In the current distributed system, after decoupling the consensus algorithm and the state machine by using the shared log, the technical architecture of the distributed system is changed to a masterless architecture, that is, the identities of all nodes in the system are consistent, so the functions originally realized by the consensus algorithm need to be realized by the state machine, including the image management function, and the communication between nodes is changed from direct communication to indirect communication via the shared log, and the indirect communication will cause hysteresis, so it is relatively difficult to realize the image management function by the state machine. SUMMARY

[0003] The present application aims at the deficiencies of the prior art and provides an image management method, a distributed system, an electronic device and a storage medium, which achieves the goal by the following technical scheme.

[0004] The first aspect of the present application provides an image management method applied to a first node in a distributed system, the first node being any node in the distributed system, and the method comprising:

[0005] determining that an image generation condition is met, generating an image and an image index based on a local applied log, and updating a mapping relationship between an identifier of the first node and the image index maintained locally by using the generated image index;

[0006] generating a first type of log by using the image index and the identifier of the first node, and sending the first type of log to a shared log system.

[0007] The second aspect of the present application provides a distributed system, comprising a shared log system and a plurality of nodes, wherein each node is configured to execute the steps of the method according to the first aspect.

[0008] The third aspect of the present application provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the steps of the method according to the first aspect.

[0009] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to realize the steps of the method according to the first aspect.

[0010] Based on the image management method and the distributed system according to the first aspect and the second aspect, the present application has at least the following advantages:

[0011] By adding separate mirror management logic outside the state machine of each node in the distributed system, i.e., determining whether the node itself meets the mirror generation condition, generating a mirror and a mirror index under the condition that the mirror generation condition is met, and updating the mapping relationship between the node identifier and the mirror index maintained locally, and submitting a first type of log containing the newly generated mirror index and the node identifier to the shared log system, the basic function of mirror management of the distributed system is realized, and the difficulty of implementing mirror management by the state machine is reduced. And because mirror generation is performed only under the condition that the mirror generation condition is met, system performance fluctuations caused by frequent mirror generation can be prevented. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the principles of the present application, and do not limit the present application in any manner. In the drawings:

[0013] Figure 1 A structure schematic diagram of a distributed system according to an exemplary embodiment of the present application is shown;

[0014] Figure 2 An embodiment flowchart of a mirror management method according to an exemplary embodiment of the present application is shown;

[0015] Figure 3 A mirror management implementation schematic diagram according to an exemplary embodiment of the present application is shown;

[0016] Figure 4 A heartbeat mechanism implementation schematic diagram according to an exemplary embodiment of the present application is shown;

[0017] Figure 5 A log compression implementation schematic diagram according to an exemplary embodiment of the present application is shown;

[0018] Figure 6 A hardware structure schematic diagram of an electronic device according to an exemplary embodiment of the present application is shown;

[0019] Figure 7 A structure schematic diagram of a storage medium according to an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0023] In a masterless distributed system, each of the multiple nodes has the same identity. (See also...) Figure 1 As shown, the distributed system includes a first node and a second node with the same identity. Each node contains a state machine representing the structured state of the logs applied to this node in memory, as well as some functional modules.

[0024] Furthermore, the distributed system also includes a shared log system for storing shared logs. This shared log system has log ordering capabilities, allowing each node to concurrently submit logs to it. The shared log system sorts the received logs according to their log timestamps, adds a log index to each log based on the sorting result, and then sends it to a shared queue. Figure 1 As shown, the logs stored in the shared queue have log indices of 140, 141, 142, 143, 144, and 145, respectively.

[0025] The operation logic of the above distributed system is that: after a client establishes a communication connection with any node in the system, the client sends a task request (for example, a write request) to the node through the communication connection, the node generates an original log and transmits it to a function module when receiving the task request, the function module adds information in the original log according to the function requirement and then submits it to a shared log system, and the shared log system sorts the received log and adds it to the end of a shared queue in the form of a shared log.

[0026] Each node in the system continuously sends an application request to the shared log system to read the log, and the node reads the log and applies it to a state machine after the log flows through the function module, and if the node judges that the read log is the log of itself, the node returns the execution result of the log to the corresponding client.

[0027] It is worth noting that the client can freely establish a communication connection with the node in the distributed system and disconnect, and even if connected to different nodes, the data read by the client can be ensured to be consistent.

[0028] Based on the operation logic of the distributed system with the above given masterless architecture, in order to realize the mirror management function, the application realizes the mirror management logic by additionally setting a mirror management module outside the state machine of each node, and the upper layer of the mirror management module is still the state machine.

[0029] In the mirror management process, if the first node determines that the mirror generation condition is met, the mirror and the mirror index are generated based on the locally applied log, the mapping relationship between the identifier of the first node and the mirror index maintained locally is updated using the generated mirror index, and then the first type of log is generated using the mirror index and the identifier of the first node, and the first type of log is sent to the shared log system;

[0030] Further, when the second node reads the first type of log generated by the first node from the shared log system, the mapping relationship between the identifier of the first node and the mirror index maintained locally is updated using the mirror index carried by the read first type of log.

[0031] Those skilled in the art can understand that the above mirror management process is an example of two nodes, and generally the distributed system contains multiple nodes, so the first node can be regarded as any node in the distributed system, and the second node can be regarded as any node other than the first node.

[0032] In order to enable those skilled in the art to better understand the scheme of the application, the mirror management logic of the mirror management module will be described clearly, completely and comprehensively in conjunction with the drawings in the embodiments of the application.

[0033] Embodiment one

[0034] Figure 2For the embodiment flowchart of the mirror image management method according to an exemplary embodiment of the present application, applied to a first node in a distributed system, the first node is any node in the distributed system, such as the node A or the node B shown in the above Figure 1 The mirror image management method includes the following steps:

[0035] Step 201: Determine that the mirror image generation condition is met, generate a mirror image and a mirror image index based on the local applied log, and update the mapping relationship between the identifier of the first node and the mirror image index maintained locally by using the generated mirror image index.

[0036] The mirror image generation condition defines the trigger condition for generating a mirror image, and the system performance loss caused by frequent generation of mirror images can be prevented by the mirror image generation condition.

[0037] Illustratively, for the mirror image generation process, since the state machine represents the structured state of the applied log in the memory, the interface provided by the state machine can be called to generate a mirror image and a mirror image index based on the local applied log. The mirror image is the structured information and data of the state machine organized and stored in a fixed format. The mirror image index is the maximum log index value in all applied logs participating in the mirror image generation, that is, the log index of the last applied log included in the mirror image.

[0038] In the embodiment of the present application, the mirror image index of the last time the mirror image is generated on all nodes in the distributed system is maintained in any node in the distributed system, that is, a mapping relationship set between a node identifier and a mirror image index is saved. Therefore, after the first node generates a mirror image, the mapping relationship between the identifier of the first node and the mirror image index maintained locally needs to be updated by using the newly generated mirror image index, so that the mirror image index corresponding to the identifier of the first node always remains the latest mirror image index.

[0039] In a possible implementation manner, the trigger condition for generating a mirror image can include active triggering and passive triggering, and the determination process of the two conditions is described in detail as follows:

[0040] The first one is the determination process of the active trigger condition: the mirror image index corresponding to the identifier of the first node is obtained from the mapping relationship maintained locally, and the log index of the current application log is obtained. According to the difference between the obtained mirror image index and the log index being greater than a target threshold, it is determined that the active trigger condition in the mirror image generation condition is met.

[0041] The current application log is a log newly read by the first node from the shared log system, and the active triggering condition is defined by determining that the difference between the mirror index of the last time the first node generates a mirror and the log index of the current application log is greater than the target threshold. The minimum number of logs between two mirror generations can be specified to prevent system performance fluctuations caused by frequent mirror generation.

[0042] Optionally, the target threshold can be the sum of a preset value and a random value. Since the random value changes in real time, this can avoid system performance fluctuations caused by simultaneous mirror generation by each node in the distributed system.

[0043] The second is the determination process of the passive triggering condition: when receiving a mirror generation request, extracting the specified index from the mirror generation request, obtaining the mirror index corresponding to the identifier of the first node from the locally maintained mapping relationship, and obtaining the log index of the current application log, according to the specified index being less than the log index and greater than the mirror index, it is determined that the passive triggering condition in the mirror generation condition is satisfied.

[0044] The mirror generation request is a request triggered by an upper module, and the specified index carried by the mirror generation request is also an index value specified by the upper module. Since the log index of the current application log indicates the log index contained in the current state machine, if the specified index is less than the log index of the current application log, it means that the mirror generation will not miss data, and in addition, the specified index is also greater than the mirror index, which means that a certain number of logs have been read and applied since the last mirror generation.

[0045] It should be noted that after the mirror is generated, the mirror can be stored in the form of a file, or it can also be stored in the shared log system.

[0046] Step 202: Generate a first type of log using the mirror index and the identifier of the first node, and send the first type of log to the shared log system.

[0047] The first type of log includes the mirror index and the identifier of the first node, and also includes the log type: mirror type. After sending the first type of log to the shared log system, other nodes in the distributed system read the first type of log, and update the mapping relationship between the node identifier and the mirror index maintained on the other nodes, so that the mirror index of the first node maintained by each node in the distributed system is consistent.

[0048] It should be noted that for the first node, the first type of log generated by the second node in the distributed system can also be read from the shared log system, and the mirror index carried by the read first type of log is used to update the mapping relationship between the identifier of the second node and the mirror index maintained locally.

[0049] The second node is a node in the distributed system other than the first node.

[0050] In an exemplary scenario, referring to Figure 3 As shown, the mirror image index of the last time the node 2037 generates a mirror image in the distributed system is 160, and the mirror image is generated after it is determined that the mirror image generation condition is met, the mirror image index is changed from 160 to 285, a first type of log containing the mirror image index Idx: 285, the node identification ID: 2037, and the log type Type: SP is submitted to the log sharing system, the log sharing system adds a log index of 286 to it, and the node 1012 reads the first type of log generated by the node 2037 from the shared log system, applies the first type of log to the local, and updates the mirror image index of the node 2037 maintained locally to 285.

[0051] It should be noted that the mapping relationship between the node identification and the mirror image index maintained by each node in the distributed system can also add an update time, that is, when the log containing the node identification and the mirror image index is read, the update time in the corresponding mapping relationship is updated. Of course, if a specified time interval has elapsed since the update time and the log containing the corresponding node identification and mirror image index has not been read, it can be considered that the corresponding node is offline, and the corresponding mapping relationship is deleted. When the log containing the corresponding node identification and mirror image index is read next time, it can be considered that the corresponding node is online, and the mapping relationship between the node identification and the mirror image index is re-established.

[0052] Further, in addition to reading the first type of log to maintain the mapping relationship between the node and the mirror image index, each node in the distributed system can also use a heartbeat mechanism to maintain the mapping relationship between the node and the mirror image index, that is, the first node sends a heartbeat type log containing the node identification and the mirror image index to the shared log system at intervals, and each time the first node reads the heartbeat type log generated by the second node, it also updates the mapping relationship between the second node identification and the mirror image index maintained locally using the mirror image index carried by the read log.

[0053] Referring to Figure 4 As shown, the node 1012 generates a heartbeat log containing the node identification ID: 1012, the mirror image index Idx: 145, and the log type Type: HB, and submits it to the shared log system, and the shared log system adds a log index of 284 to it after sorting. The node 2037 generates a heartbeat log containing the node identification ID: 2037, the mirror image index Idx: 160, and the log type Type: HB, and submits it to the shared log system, and the shared log system adds a log index of 285 to it after sorting.

[0054] After the node 1012 reads the heartbeat log with log index 284, in the process of applying the heartbeat log, it is found that the heartbeat log is self-generated and can be ignored. Then, after the node 1012 reads the heartbeat log with log index 285, in the process of applying the heartbeat log, the node identifier ID: 2037 and the mirror index mapping relationship maintained locally are updated by using the mirror index Idx: 160 carried in the heartbeat log, and the corresponding update time is updated.

[0055] After the node 2037 reads the heartbeat log with log index 284, in the process of applying the heartbeat log, the node identifier ID: 1012 and the mirror index mapping relationship maintained locally are updated by using the mirror index Idx: 145 carried in the heartbeat log, and the corresponding update time is updated. Then, after the node 2037 reads the heartbeat log with log index 285, in the process of applying the heartbeat log, it is found that the heartbeat log is self-generated and can be ignored.

[0056] At this point, the above Figure 2 The mirror management process shown in the figure is implemented by adding separate mirror management logic to the state machine of each node in the distributed system, that is, determining whether the node itself meets the mirror generation condition, generating a mirror and a mirror index when it is determined that the mirror generation condition is met, updating the node identifier and the mirror index mapping relationship maintained locally, and submitting a first type of log containing the newly generated mirror index and the node identifier to the shared log system, so that other nodes in the distributed system read the first type of log and update the node identifier and the mirror index mapping relationship maintained on the other nodes, thereby realizing the basic function of mirror management of the distributed system, and reducing the difficulty of implementing mirror management by the state machine. And because mirror generation is performed only when the mirror generation condition is met, it can prevent system performance fluctuations caused by frequent mirror generation.

[0057] Embodiment Two

[0058] In addition to the mirror generation of the above Figure 2 The mirror management function of the distributed system also involves log compression. The log compression implementation process for mirror management includes the following steps:

[0059] Step 203: Determine the compression index based on the current log compression condition.

[0060] The compression index refers to the log position to be compressed, that is, all nodes in the distributed system will not read the log before the compression index.

[0061] In one possible implementation, the trigger condition of log compression can include active triggering and passive triggering. The compression index determination process under the two conditions is described in detail as follows:

[0062] The first log compression active trigger condition is that after generating the mirror image and the mirror image index based on the locally applied log, the log compression is immediately attempted, that is, the minimum mirror image index is obtained from the locally maintained mapping relationship, and the minimum mirror image index is determined as the compression index.

[0063] In the distributed system, all nodes continuously read and apply logs from the shared log system. Therefore, compared with other nodes in the distributed system, the node corresponding to the minimum mirror image index in the mapping relationship is the node with the least applied log. Determining the minimum mirror image index as the compression index can ensure that all nodes in the distributed system will not read the log before the compression index.

[0064] The second log compression passive trigger condition is that the specified index is extracted from the received log compression request, the minimum mirror image index is obtained from the locally maintained mapping relationship, and then the minimum value of the specified index and the minimum mirror image index is determined as the compression index.

[0065] The log compression request is a request triggered by the upper module, and the specified index carried by the log compression request is also an index value specified by the upper module. By selecting the minimum value between the minimum mirror image index in the mapping relationship and the specified index as the compression index, it can be ensured that all nodes in the distributed system will not read the log before the compression index.

[0066] Step 204: generating a second type of log containing the compression index, and sending the second type of log to the shared log system.

[0067] Specifically, in addition to the compression index, the second type of log also includes the log type: request compression and the node identifier of the first node. After sending the second type of log to the shared log system, the nodes in the distributed system read the second type of log, and when determining that the node is the node of the current minimum mirror image index, the log compression operation is performed, the log is compressed to the log position indicated by the compression index, and the logs before the log position indicated by the compression index are all deleted.

[0068] It should be noted that after the first node reads the second type of log from the shared log system, it determines whether the mirror image index corresponding to the identifier of the first node is the minimum value based on the locally maintained mapping relationship. If not, no processing is performed. If yes, the log compression operation is performed based on the compression index carried by the read second type of log, and then a third type of log is generated using the compression index and the identifier of the first node, and the third type of log is sent to the shared log system.

[0069] The third type of log includes the log type: success compression log in addition to the compression index and the identification of the first node. After the third type of log is sent to the shared log system, the nodes in the distributed system read the third type of log, and the mapping relationship between the node identification and the mirror index maintained on the nodes is updated, so that the mirror index of the first node maintained by each node of the distributed system is consistent.

[0070] In an exemplary scenario, referring to FIG. 10, passive triggering is taken as an example. It is assumed that the index specified in the log compression request received by the node 2037 is A, and the minimum mirror index in the locally maintained mapping relationship is B. The smaller one of A and B is taken as the compression index C. Then, the second type of log including the node identification ID: 2037, the compression index C, and the log type Type: Try is submitted to the shared log system. Figure 5

[0071] Then, when the node 1012 reads the second type of log from the shared log system and determines that the mirror index 145 corresponding to the node 1012 is the minimum mirror index according to the locally maintained mapping relationship, the node 1012 performs a log compression operation based on the compression index carried by the read second type of log. Then, the third type of log is generated using the compression index C and the identification 1012 of the node 1012, and the third type of log is sent to the shared log system. The third type of log includes the log type Type: Done.

[0072] Based on the above log compression implementation process, by using the node with the minimum mirror index to perform the log compression operation, the system performance fluctuation caused by repeated log compression can be avoided.

[0073] Embodiment Three

[0074] In addition to the mirror generation and log compression, the mirror management function of the distributed system also involves node recovery. The node recovery implementation process for mirror management includes the following steps:

[0075] Step 205: According to the first node state meeting the preset recovery condition, a communication connection is established with a second node in the distributed system.

[0076] The second node is any node in the distributed system except the first node. By establishing a communication connection with the second node, data on the second node can be read.

[0077] ​The preset recovery condition can be that the first node restarts due to a crash or power failure, or that the log reading speed of the first node is significantly slower than the log generation speed of the shared log system, although the log reading operation is correct, but the read log can be invalid, for example, the node with slow reading speed reads the heartbeat log of other nodes from the shared log system, which can be a heartbeat log a long time ago, and the node processes the meaningless log at this time.

[0078] In an optional embodiment, for the implementation process of the first node log reading speed significantly slower than the log generation speed of the shared log system, the ratio between the log reading speed and the log generation speed of the shared log system is determined every preset time interval, and the first node state is determined to meet the preset recovery condition according to the fact that the ratio determined twice in succession is less than the preset threshold and the ratio determined for the second time is less than the ratio determined for the first time.

[0079] By using the ratio of the log reading speed and the log generation speed determined twice in succession for judgment, the accuracy of the judgment result can be ensured.

[0080] Specifically, for the acquisition method of the log reading speed, the first node maintains a timer to count the number of logs read by the first node from 0 to a certain time, and then calculates the ratio of the number of logs and the timer counting time as the log reading speed.

[0081] For the acquisition method of the log generation speed, since the shared log system receives logs sent by each node in the distributed system, the shared log system calculates the log generation speed. Specifically, the shared log system generates an anchor package every preset time interval, and the ratio between the number of logs received between the two anchor packages and the preset time interval is taken as the log generation speed, so that each node in the distributed system can read the log generation speed every preset time interval to compare and judge with the log reading speed on the node itself.

[0082] Step 206: reading the mirror and mirror index generated on the second node through the communication connection.

[0083] Step 207: applying the read mirror to the local and reading the log from the shared log system according to the read mirror index for processing.

[0084] Based on the above node recovery implementation process, when each node in the distributed system determines that its state meets the preset recovery condition, it automatically enters the recovery process and recovers to a state that can serve the client, thereby ensuring the accuracy of the mirror generation and log compression mechanism.

[0085] The embodiment of the present application also provides an electronic device corresponding to the mirror management method provided in the foregoing embodiment, to execute the mirror management method.

[0086] Figure 6 A hardware structure diagram of an electronic device according to an exemplary embodiment of the present application is shown in the present application, which comprises a communication interface 601, a processor 602, a memory 603 and a bus 604; wherein the communication interface 601, the processor 602 and the memory 603 complete mutual communication through the bus 604. The processor 602 can execute the mirror management method described above by reading and executing machine executable instructions corresponding to the control logic of the mirror management method in the memory 603, and the specific content of the method can be referred to the foregoing embodiment, which will not be repeated here.

[0087] The memory 603 mentioned in the present application can be any electronic, magnetic, optical or other physical storage system, and can contain stored information such as executable instructions, data, etc. Specifically, the memory 603 can be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), any type of storage disk (such as an optical disk, a DVD, etc.), or similar storage medium, or a combination thereof. The communication connection between the system network element and at least one other network element can be achieved through at least one communication interface 601 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.

[0088] The bus 604 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 603 is used to store programs, and the processor 602 executes the programs after receiving execution instructions.

[0089] The processor 602 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 602. The processor 602 mentioned above can be a general processor, including a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution.

[0090] The electronic device provided by the embodiments of the present application and the mirror management method provided by the embodiments of the present application have the same beneficial effects as the method adopted, run or implemented by the electronic device.

[0091] The embodiments of the present application also provide a computer readable storage medium corresponding to the mirror management method provided by the preceding embodiments. Please refer to Figure 7 As shown in the figure, the computer readable storage medium is an optical disc 30, and a computer program (i.e. program product) is stored on the optical disc 30. When the computer program is run by a processor, the mirror management method provided by any of the preceding embodiments is executed.

[0092] It should be noted that examples of the computer readable storage medium can also include, but are not limited to, a phase change memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), other types of random access memory (RAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a flash memory or other optical or magnetic storage medium, which will not be described one by one here.

[0093] The computer readable storage medium provided by the above embodiments of the present application and the mirror management method provided by the embodiments of the present application have the same beneficial effects as the method adopted, run or implemented by the application program stored in the computer readable storage medium.

[0094] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the application that come within the scope of the general inventive concepts based on the description and the appended claims. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the application indicated by the following claims.

[0095] It should also be noted that the terms "comprising", "comprises" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without further limitation, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0096] The above only is the preferred embodiment of the present application, and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A mirror management method, characterized in that, The first node is applied to a distributed system with a masterless architecture. The first node is any node in the distributed system. Each node in the distributed system contains a state machine that represents the structured state of the logs applied by the node in memory. The state machine is used to generate a mirror and a mirror index based on the locally applied logs when the corresponding interface is called. Each node stores the image index of all nodes in the distributed system when the image was last generated; the method includes: Once the conditions for image generation are met, an image and an image index are generated based on the locally applied logs. The generated image index is used to update the mapping relationship between the identifier of the first node and the image index maintained locally. The image is used to organize and store the structural information and data of the state machine in a fixed format. The image index is used to indicate the maximum log index value among all applied logs participating in image generation. The first type of log is generated using the mirror index and the identifier of the first node, and then the first type of log is sent to the shared log system.

2. The method according to claim 1, characterized in that, The determination that the mirror generation conditions are met includes: Retrieve the image index corresponding to the identifier of the first node from the locally maintained mapping relationship, and retrieve the log index of the current application log; The image generation condition is determined to be met if the difference between the image index and the log index is greater than the target threshold.

3. The method according to claim 1, characterized in that, The determination that the mirror generation conditions are met includes: Extract the specified index from the received image generation request; Retrieve the image index corresponding to the identifier of the first node from the locally maintained mapping relationship, and retrieve the log index of the current application log; The image generation conditions are determined to be met based on the fact that the specified index is less than the log index and greater than the image index.

4. The method according to claim 1, characterized in that, The method further includes: Read the first type of log generated by the second node in the distributed system from the shared log system, and update the mapping relationship between the identifier of the second node and the mirror index maintained locally using the mirror index carried by the read first type of log; The second node is a node in the distributed system other than the first node.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: A compression index is determined based on the currently met log compression conditions, and the compression index indicates the log position to which compression is to be performed; A second type of log containing the compressed index is generated, and the second type of log is sent to the shared log system.

6. The method according to claim 5, characterized in that, The process of determining the compression index based on the currently satisfied log compression conditions includes: After generating the image and image index based on the locally applied logs, the minimum image index is obtained from the locally maintained mapping relationship, and this minimum image index is determined as the compressed index; and / or, Extract the specified index from the received log compression request, and obtain the minimum mirror index from the locally maintained mapping relationship; determine the minimum value between the specified index and the minimum mirror index as the compression index.

7. The method according to claim 5, characterized in that, The method further includes: Read the second type of log from the shared log system, and determine whether the mirror index corresponding to the identifier of the first node is the minimum value based on the mapping relationship maintained locally; If so, then perform log compression operation based on the compression index carried by the second type of log, generate a third type of log using the compression index and the identifier of the first node, and send the third type of log to the shared log system.

8. The method according to any one of claims 1-4, characterized in that, The method further includes: If the state of the first node meets the preset recovery conditions, a communication connection is established with the second node in the distributed system. The second node is a node in the distributed system other than the first node. The image and image index generated on the second node are read through the communication connection; The image is applied locally, and logs are read from the shared log system for processing based on the read image index.

9. The method according to claim 8, characterized in that, Based on the first node's state meeting preset recovery conditions, including: At preset time intervals, determine the ratio between the log reading speed and the log generation speed of the shared log system. If the ratio determined in two consecutive tests is less than the preset threshold, and the ratio determined in the second test is less than the ratio determined in the first test, the state of the first node is determined to meet the preset recovery conditions.

10. A distributed system, characterized in that, The system includes a shared log system and multiple nodes, wherein each node is used to perform the steps of the method as described in any one of claims 1-9.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the method as claimed in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the steps of the method as described in any one of claims 1-9.

Images