Distributed collaboration method and device based on MySQL storage, equipment and medium
Through the distributed collaboration method based on MySQL storage, using file directory tree storage and version number design, the challenges of coordination and communication between nodes in distributed systems are solved, stability and sequential consistency are achieved, and system complexity is reduced. Operation and maintenance costs avoid single point of failure.
Patent Information
- Application Number
- CN202410908750.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2024-12-10
AI Technical Summary
Distributed systems have challenges in coordination and communication between nodes, especially in the face of failure recovery and retry. The introduction of third-party middleware will increase architectural complexity and operation and maintenance costs, and the existing technology has not effectively solved these problems. .
Using a distributed collaboration method based on MySQL storage, node information is stored in a file directory tree, and the changes of nodes and their children are monitored. Information updates and modifications are implemented through the version number design of child nodes and parent nodes to reduce external Middleware dependency, using MySQL database to implement distributed locking and synchronization mechanisms.
The stability and sequential consistency of distributed systems are achieved, the system complexity and operation and maintenance costs are reduced, single point of failure is avoided, and the information of child nodes and parent nodes is updated and modified easily.
Smart Images
Figure CN119106084A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a distributed collaboration method, device, equipment and medium based on MySQL storage. Background Art
[0002] As the scale of the Internet and the complexity of applications continue to increase, traditional stand-alone systems can no longer meet the needs of large-scale applications. Therefore, distributed systems came into being. Distributed systems break down tasks into multiple independent parts and execute these tasks simultaneously on multiple computer nodes, thereby achieving higher performance, availability, and scalability.
[0003] However, building a distributed system is not easy and involves many challenges. One of the important challenges is the coordination and communication between nodes. In a distributed environment, each node may be in different physical locations, network conditions, and operating states. Therefore, a mechanism is needed to ensure that each node can work together, share information, and effectively recover and retry in the face of failures. Various distributed coordination middlewares have emerged, among which the most widely used ones are: zookeeper, etcd, consul, eureka, etc. However, the introduction of third-party middleware will increase the complexity of the architecture and is not conducive to system output and delivery. The increase in architectural complexity will lead to reduced stability. The introduction of one more middleware will introduce certain unstable factors, and the operation and maintenance costs are further increased because the middleware requires operation and maintenance personnel to operate and maintain it.
[0004] To address the above problems, no effective solution has been proposed yet. Summary of the invention
[0005] Based on this, it is necessary to provide a distributed collaboration method, device, equipment and medium based on MySQL storage for the above technical problems. The distributed collaboration method based on MySQL storage of the present invention naturally does not have a single point of failure problem, naturally supports technical sequential consistency, and at the same time reduces dependence on external middleware, reduces system complexity and improves stability.
[0006] According to a first aspect of the present invention, a distributed collaboration method based on MySQL storage is provided, comprising: a connection step, defining and creating a connection, wherein each java instance object is regarded as a logical link; a storage step, storing nodes in a file directory tree-like manner; and a monitoring step, monitoring information changes of a node and its child nodes.
[0007] In some embodiments, it also includes a configuration management step, where each node obtains the latest configuration information to achieve unified configuration management.
[0008] In some embodiments, the node has node metadata information, including: node creation time, node type, node data, and data version.
[0009] In some embodiments, the information change includes: changing node data and deleting nodes;
[0010] Every time the node data changes, the version number of the node data version is changed;
[0011] Each time the child node data is changed, the version number of the data version of the child node is changed, and the version number of the data version of the parent node of the child node is changed.
[0012] When the node is deleted, if the node has a parent node, the version number of the data version of the parent node is changed.
[0013] Through such a version number change design, it is possible to more conveniently update and modify the information of the child node and the parent node of the child node, without having to traverse all the child nodes under the parent node to know whether there are any changes to the child nodes under the parent node.
[0014] In some embodiments, the node types include: temporary nodes, permanent nodes;
[0015] The temporary node is a node type that is destroyed when the logical connection of the created node is disconnected;
[0016] The permanent node is a node type that is not destroyed when the logical connection that created the node is disconnected.
[0017] In some embodiments, the logical connection has a unique identifier, and determines whether to maintain the heartbeat as needed;
[0018] The situation where the heartbeat needs to be maintained includes: creating a temporary node.
[0019] In some embodiments, a MySQL database is used to store data information, and a distributed lock and synchronization mechanism is implemented by using temporary sequential nodes in the MySQL database.
[0020] According to a second aspect of the present invention, a distributed collaborative device based on MySQL storage is provided, comprising: a connection module for defining and creating connections, wherein each Java instance object is regarded as a logical link; a storage module for storing nodes in a file directory tree-like manner; and a monitoring module for monitoring information changes of nodes and their child nodes.
[0021] According to a third aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.
[0022] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned embodiments of the method are implemented.
[0023] The distributed coordination method implemented based on MySQL in the present invention naturally does not have a single point of failure problem, naturally supports technical sequential consistency, and reduces the dependence on external middleware, reduces system complexity, and improves stability, and has the characteristics of extremely low access cost and low maintenance cost. Through the design of the version number of the child node and the parent node, the information of the child node and its parent node can be updated and modified more conveniently. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 is a flow chart of some embodiments of the distributed collaboration method based on MySQL storage of the present invention;
[0025] Figure 2 is a flowchart of a temporary node going offline in some embodiments of the present invention;
[0026] Figure 3 is a schematic diagram of monitoring node changes in some embodiments of the present invention;
[0027] Figure 4 is a flow chart of monitoring information changes in some embodiments of the present invention;
[0028] Figure 5 is a schematic diagram of a logical architecture in some embodiments of the present invention;
[0029] Figure 6 It is a schematic diagram of the structure of some embodiments of the distributed collaboration device based on MySQL storage of the present invention;
[0030] Figure 7 It is a diagram of the internal structure of a computer device used to implement some embodiments of the present invention. DETAILED DESCRIPTION
[0031] Embodiments of the present invention will be described more fully below with reference to the accompanying drawings, in which embodiments of the present invention are shown. However, the present invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.
[0032] The terms used herein are used only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an", "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that when used herein, the term "comprising" specifies the presence of the claimed features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof.
[0033] Unless otherwise defined, the terms (including technical and scientific terms) used herein have the same meanings as commonly understood by those of ordinary skill in the art to which the present disclosure belongs. The terms used herein should be interpreted as having the meanings consistent with their meanings in the context of this specification and in the relevant field, and should not be interpreted in an idealized or overly formal sense, unless specifically defined as such herein.
[0034] Logical connection: A connection established between different computers or devices through network protocols to achieve data transmission and communication.
[0035] Figure 1 The flowcharts of some embodiments of the distributed collaboration method based on MySQL storage of the present invention are shown.
[0036] like Figure 1 As shown, the method includes:
[0037] Connection step S102, defining and creating a connection, wherein each Java instance object is regarded as a logical link;
[0038] In some embodiments, each instance creation means creating a logical connection of Datakeeper, and the connection creation implicitly executes two logics: creating a unique ID (UUID) for each logical connection; and each connection decides on demand whether to maintain the heartbeat.
[0039] Each Java instance has its own logical connection, which ensures that their operations are independent of each other and do not affect each other, thus improving the stability and concurrency of the system.
[0040] The pseudo code for its implementation is as follows:
[0041] / / Create a connection
[0042] DataKeeperConnection dataKeeperConnection=
[0043] new DataKeeperConnection(dataSource,true);
[0044] / / Create a connection
[0045] DataKeeperConnection dataKeeperConnection2=
[0046] new DataKeeperConnection(dataSource);
[0047] The situations where the heartbeat needs to be maintained specifically include: creating a temporary type of node (the current client application is offline, and a temporary node needs to be created offline, which is generally used to maintain runtime information).
[0048] Figure 2 A flowchart of temporary node offline in some embodiments of the present invention is shown. Figure 2 As shown, create DataKeeperConnection; in the loop, first check whether the heartbeat information exists. If it exists, update the heartbeat time. If it does not exist, create the heartbeat information, then check and clean up the expired heartbeat; if you actively go offline, clean up the heartbeat and the created temporary node data.
[0049] Storage step S104, storing the nodes in a file directory tree format;
[0050] In some embodiments, the node metadata information of each node includes: the creation time of the node; the node type; the node data (text in JSON format); the data version (each version is changed + 1). Among them, the node types are divided into two types: temporary nodes (nodes that are destroyed when the logical connection that created the node is disconnected); permanent nodes (node types that are not destroyed when the logical connection that created them is destroyed); in addition, permanent subnodes cannot be created under temporary nodes.
[0051] Monitoring step S106, monitoring information changes of the node and its child nodes.
[0052] In some embodiments, the following situations may occur when the information of a monitoring node and its child nodes are changed: 1. The current node data is changed, that is, the current node version number is changed. 2. The child node data is changed, that is, the child node version number is changed. 3. When a node is deleted: To delete a node, the parent node version number needs to be changed (the parent node current version number + the monitoring version number of the deleted node).
[0053] Figure 3 FIG. 2 is a schematic diagram showing a monitoring node change in some embodiments of the present invention. Figure 3As shown in the figure, when there is no child node under the node, the listening version is the current version of the node; when there is a child node under the node, the listening version is the sum of the node version and its child node version. This design allows the information of the child node and the parent node of the child node to be updated and modified conveniently when the information of the child node changes, avoiding the need to traverse all the child nodes under the parent node to know whether there is a change in the child node under the parent node and then modify the parent node information.
[0054] Figure 4 FIG. 4 is a flowchart showing a change in monitoring information in some embodiments of the present invention. Figure 4 As shown, add a node change listener and query the current version number of the node (v0); in the loop, query the node version number (v1) in MySQL. If the node version number (v1) in MySQL is greater than the DataKeeperClient (client) version number, update the version in the memory to the database version (v1) and publish a change event.
[0055] Figure 5 Schematic diagram of the logical architecture in some embodiments of the present invention is shown. Figure 5 As shown, the host application is divided into a connection module, a storage module, and a monitoring module. The host application writes data into the storage layer, and the storage layer returns the data change notification to the host application.
[0056] The functional points that can be achieved by the present invention are shown in Table 1.
[0057] Table 1:
[0058] Function describe Machine registration, heartbeat Registering machine information Create nodes and data Temporary nodes, persistent nodes Deleting a Node Deleting a Node Does the node exist? Does the node exist? Update node data Update node data Get node data Get node data Get the child nodes of a node Get the child nodes of a node Listening Node Listening Node
[0059] The present invention implements the core capabilities required in the distributed system based on MySQL storage, including:
[0060] Distributed system configuration management: used to manage the configuration information of distributed systems, such as storing database connection parameters, registration address information, etc. Each node can obtain the latest configuration information, thereby achieving unified configuration management.
[0061] Naming service: It can implement the naming service of the distributed system and provide a unified namespace for each node. For example, it can register information such as service port and node address so that the client can dynamically discover the service.
[0062] Distributed locks and synchronization: Distributed locks and temporary sequence nodes are provided, which can be used to implement distributed locks and synchronization mechanisms, which ensure mutual exclusive access and sequential execution of resources in a distributed environment.
[0063] Figure 6A schematic diagram of the structure of some embodiments of the distributed collaboration device based on MySQL storage of the present invention is shown.
[0064] like Figure 6 As shown, the distributed collaboration device based on MySQL storage in the embodiment includes:
[0065] A connection module 100 is used to define and create a connection, wherein each Java instance object is regarded as a logical link;
[0066] Storage module 200, used to store nodes in a file directory tree format;
[0067] The monitoring module 300 is used to monitor information changes of a node and its sub-nodes.
[0068] For the specific definition of a distributed collaborative device based on MySQL storage, please refer to the definition of a distributed collaborative method based on MySQL storage above, which will not be repeated here. Each module in the above-mentioned distributed collaborative device based on MySQL storage can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0069] The present invention also provides a computer device, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned distributed collaboration method based on MySQL storage is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc. Those skilled in the art can understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0070] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned distributed collaboration method based on MySQL storage is implemented.
[0071] Those skilled in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0072] So far, the embodiments of the present invention have been described in detail. In order to avoid obscuring the concept of the present invention, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solutions disclosed herein.
[0073] Although some specific embodiments of the present invention have been described in detail by way of examples, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should be understood by those skilled in the art that the above embodiments may be modified or some technical features may be replaced by equivalents without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.
Claims
1. A distributed collaboration method based on MySQL storage, characterized in that: include: The connection step defines and creates a connection, where each Java instance object is considered as a logical link; The storage step uses a file directory tree to store nodes; The monitoring step monitors the information changes of the node and its child nodes.
2. The distributed collaboration method based on MySQL storage according to claim 1, characterized in that: Also includes, In the configuration management step, each node obtains the latest configuration information to achieve unified configuration management.
3. The distributed collaboration method based on MySQL storage according to claim 1, characterized in that: The node has node metadata information, including: node creation time, node type, node data, and data version.
4. The distributed collaboration method based on MySQL storage according to claim 3, characterized in that: The information change includes: changing node data and deleting nodes; Every time the node data changes, the version number of the node data version is changed; Each time the child node data is changed, the version number of the data version of the child node is changed, and the version number of the data version of the parent node of the child node is changed. When the node is deleted, if the node has a parent node, the version number of the data version of the parent node is changed.
5. The distributed collaboration method based on MySQL storage according to claim 3, characterized in that: The node types include: temporary nodes and permanent nodes; The temporary node is a node type that is destroyed when the logical connection of the created node is disconnected; The permanent node is a node type that is not destroyed when the logical connection that created the node is disconnected.
6. The distributed collaboration method based on MySQL storage according to claim 1, characterized in that: The logical connection has a unique identifier, and whether to maintain the heartbeat is determined according to the need; The situation where the heartbeat needs to be maintained includes: creating a temporary node.
7. The distributed collaboration method based on MySQL storage according to claim 1, characterized in that: The MySQL database is used to store data information, and the distributed lock and synchronization mechanism is implemented by using temporary sequential nodes in the MySQL database.
8. A distributed collaboration device based on MySQL storage, characterized in that: include: The connection module is used to define and create connections, where each Java instance object is considered as a logical link; A storage module is used to store nodes in a file directory tree format; The monitoring module is used to monitor the information changes of a node and its child nodes.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.