Multi-processing system and method for controlling multi-processing system

Abstract

To reduce delay required for synchronization by forming consensus between nodes collectively in advance the version of data to be accessed.SOLUTION: A multiple processing system includes a plurality of nodes each including a processor, a memory, and a communication device, and connects the plurality of nodes via a network. The plurality of nodes each perform one or more tasks of producing a unique output for an input. The nodes each include: a task execution control unit that accepts an event and starts a transaction of performing the tasks corresponding to the event; and a state management unit that manages version of data in a database accessed by the tasks. The task execution unit collectively forms consensus between the nodes of the version of data which the tasks refer to in the database or the version of data updated by the tasks, prior to starting processing of the tasks.SELECTED DRAWING: Figure 21

Description

[Technical field]

[0001] The present invention relates to a multi-processing system to which the technology of a fault-tolerant system is applied. [Background technology]

[0002] 2. Description of the Related Art Fault-tolerant systems are known in which the same processing is carried out between multiple computers (nodes) and operations can be continued without interruption even if a failure occurs in one of the computers.

[0003] State machine replication is known as an example of multiplexing. State machine replication performs deterministic processing for the same input to each replicated node, producing the same output and maintaining the same state, and achieving seamless master-slave system switching when a node fails.

[0004] For an application configured to execute multiple tasks in parallel, a technique is known that ensures overall determinism by matching input contents to each task between nodes, as disclosed in, for example, Patent Document 1. In Patent Document 1, when accessing shared data, the access order is determined by a leader-follower type consensus protocol and processing is executed, thereby ensuring determinism between replicated nodes. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2012 / 127652 Summary of the Invention [Problem to be solved by the invention]

[0006] In the above Patent Document 1, the order of access to the shared data (= state) between tasks is matched between nodes each time the access is made, thereby ensuring determinism. In other words, in Patent Document 1, communication occurs between the leader node and the follower node each time an access request is made to the data shared between tasks, so there is a risk that the load for synchronization will increase depending on the workload.

[0007] Therefore, the present invention has been made in consideration of the above problems, and aims to reduce the delay required for synchronization by forming an agreement among nodes in advance on the version of the shared data to be accessed. [Means for solving the problem]

[0008] The present invention is a multi-processing system having a plurality of nodes each including a processor, a memory, and a communication device, the plurality of nodes being connected via a network, and the plurality of nodes each executing one or more tasks that provide a unique output in response to an input, wherein the nodes have a task execution control unit that receives an event and initiates a transaction to execute the task corresponding to the event, and a state management unit that manages the version of data in a database accessed by the task, and the task execution control unit forms an agreement among the nodes collectively on the version of the data referenced by the task in the database or the version of the data updated by the task before starting processing of the task. Effect of the Invention

[0009] Therefore, the present invention can reduce the delay required for synchronization by simultaneously managing multiple versions of data (states) and forming a consensus among nodes in advance on the version of the data to be accessed. By configuring each node in a leader-follower system, the leader node starts executing a task before the consensus is formed, and the follower node starts executing a task after the consensus is formed, so that at least the leader node can avoid the delay caused by waiting for the consensus to be formed.

[0010] The details of at least one implementation of the subject matter disclosed herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the disclosed subject matter will become apparent from the following disclosure, drawings, and claims. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example of a multiple-system computer system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a node according to the first embodiment of the present invention. [Diagram 3] 5 is a flowchart illustrating an example of processing performed in the node according to the first embodiment of the present invention. [Figure 4] FIG. 2 illustrates an example of a task according to the first embodiment of the present invention. [Diagram 5] FIG. 2 illustrates an example of a receiving process performed in the node according to the first embodiment of the present invention. [Figure 6] FIG. 11 illustrates an example of a message header according to the first embodiment of the present invention. [Figure 7] FIG. 2 illustrates an example of the functions of a state management unit according to the first embodiment of the present invention. [Figure 8] FIG. 2 illustrates an example of a database management table according to the first embodiment of the present invention. [Figure 9] FIG. 11 illustrates an example of the table management table according to the first embodiment of the present invention. [Figure 10] FIG. 2 illustrates an example of a database table according to the first embodiment of the present invention. [Figure 11] 11 is a flowchart illustrating an example of start processing of a read-only transaction according to the first embodiment of the present invention. [Figure 12] FIG. 1 illustrates the first embodiment of the present invention and shows an example of a handle. [Figure 13]13 is a flowchart illustrating an example of a request to start a read-only transaction to a specified table according to the first embodiment of the present invention. [Figure 14] 13 is a flowchart illustrating an example of a request to end a read-only transaction according to the first embodiment of the present invention. [Figure 15] 13 is a flowchart illustrating an example of a start request for a read / write transaction according to the first embodiment of the present invention. [Figure 16] FIG. 1 illustrates the first embodiment of the present invention and shows an example of a handle. [Figure 17] 13 is a flowchart illustrating an example of a request to end a read / write transaction according to the first embodiment of the present invention. [Figure 18] FIG. 2 illustrates an example of consensus formation performed between nodes according to the first embodiment of the present invention. [Figure 19] FIG. 11 is a sequence diagram illustrating an example of a task initialization process according to the first embodiment of the present invention. [Figure 20] FIG. 2 illustrates an example of a task management table according to the first embodiment of the present invention. [Figure 21] FIG. 4 is a sequence diagram illustrating an example of task processing according to the first embodiment of the present invention. [Figure 22] 11 is a flowchart illustrating an example of an event registration process according to the first embodiment of the present invention. [Diagram 23] FIG. 2 illustrates an example of an event management queue according to the first embodiment of the present invention. [Figure 24] 11 is a flowchart illustrating an example of a transaction start process performed in the leader node according to the first embodiment of the present invention. [Diagram 25] 11 is a flowchart illustrating an example of a transaction start process performed in a follower node according to the first embodiment of the present invention. [Figure 26] FIG. 1 illustrates an example of a transmission process according to the first embodiment of the present invention. [Figure 27] 13 is a flowchart illustrating an example of processing performed by the state management unit according to the second embodiment of the present invention. [Figure 28] FIG. 11 illustrates a second embodiment of the present invention, showing an example of a handle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. EXAMPLES

[0013] 1 is a block diagram showing an example of a multi-processing system according to a first embodiment of the present invention. In the illustrated example, three nodes 1-1 to 1-3 are configured in a leader-follower configuration, and the leader node 1-1 is replicated as follower nodes 1-2 to 1-3 in the multi-processing system.

[0014] In the following description, when there is no need to distinguish between leader nodes and follower nodes, the symbol "1" is used, with the "-" and everything after it omitted. The same applies when the symbols of other components include "-".

[0015] Each client application 2 is connected to a proxy 3 running on the same client computer 4 , and the proxy 3 is connected to each node 1 via a network 5 .

[0016] In the multiplexed computer system of this embodiment, duplicated server applications 20 (see FIG. 2) run in the same state on each of the duplicated nodes 1-1 to 1-3. The client application 2 sends a request message 300 addressed to the server application 20 to the proxy 3, which duplicates it and distributes it to each node 1. The server applications 20 running on each node 1 each perform the same processing, and return the processing results to the client application 2 via the proxy 3. The proxy 3 runs on the same client computer 4 as each client application 2. For this reason, the proxy 3 does not become a single point of failure.

[0017] The outline of the multiplexing system of this embodiment is as follows: First, the client application 2 transmits a request message addressed to a server application replicated on each node 1, and the proxy 3 acting as an intermediary receives this (S1). The proxy 3 replicates the request message received from the client application 2 and distributes it to each node 1 (S2).

[0018] The leader node 1-1 forms an agreement with the follower nodes 1-2 and 1-3 so that the server application running on each node behaves deterministically (S3). Note that, since a well-known or publicly known distributed consensus algorithm such as RAFT can be adopted to form an agreement between the nodes 1, it will not be described in detail in this embodiment.

[0019] The server application on each node 1 processes the request message from the client application 2 with deterministic behavior based on the above-mentioned agreement result, and returns the result to the proxy 3 (S4).

[0020] The proxy 3 selects one of the processing results from each node 1 based on a preset policy (majority vote, etc.) and responds to the client application 2 (S5).

[0021] In this embodiment, the client application 2 and the multiplexed nodes 1 can continue processing without losing access requests or processing results even if a failure occurs in any of the nodes 1.

[0022] As described below, node 1 in this embodiment simultaneously manages multiple versions of state (shared data = database 100) using the MVCC (Multiversion Concurrency Control) method, and can reduce the load required for synchronization by forming an agreement between nodes 1 on the version of the data to be accessed in advance.

[0023] <Node configuration> 2 is a block diagram showing an example of the node 1. The leader node 1-1 and the follower nodes 1-2 and 1-3 have the same configuration, so they will be described below as the node 1.

[0024] The node 1 is a computer including a processor 11, a memory 12, and a communication interface (or a communication device) 13. The communication interface 13 is connected to a network 5, and communicates with a client computer 4 and other nodes 1.

[0025] A server application 20 and a multiplexing processing unit 30 are loaded into the memory 12 and executed by the processor 11. A shared memory 90 shared by a task group 21 (described later) is also set in the memory 12, and a database 100 for storing the state of the server application 20 is arranged in the shared memory 90. In addition to the above-mentioned states, the database 100 also holds a database management table 200 and table management tables 210-1 to 210-3, which are used by the state management unit 60.

[0026] The memory 12 also stores a task management table 220 , which is used by the multiplexing unit 30 .

[0027] In this embodiment, an example is shown in which each multiplexing processor 30 accesses the database 100 stored in the shared memory 90 of the node 1, but the present invention is not limited to this. For example, the database 100 may be stored in a storage device (not shown) accessible from each node 1.

[0028] The server application 20 can be composed of one or more tasks 21-A to 21-N. Each task 21 receives a message from the proxy 3 as an input, and performs deterministic processing (updating the state or sending it to the outside) based on the state (shared data) stored in the database 100 managed by the state management unit 60.

[0029] The multiplexing processing unit 30 includes an external communication unit 40, a task execution control unit 50, a state management unit 60, an inter-node communication unit 70, and an event management queue 80. Each functional unit of the external communication unit 40, the task execution control unit 50, the state management unit 60, and the inter-node communication unit 70 is loaded into the memory 12 as a program.

[0030] The processor 11 operates as a functional unit that provides a predetermined function by executing processing according to the program of each functional unit. For example, the processor 11 functions as a task execution control unit 50 by executing processing according to a task execution control program. The same applies to other programs. Furthermore, the processor 11 also operates as a functional unit that provides each function of multiple processes executed by each program. A computer and a computer system are devices and systems that include these functional units.

[0031] The external communication unit 40 accepts messages (service requests to the server application 20) from the client computer 4 via the proxy 3 and notifies the task execution control unit 50, and also responds to the client application 2 via the proxy 3 with the processing results of each task 21.

[0032] The task execution control unit 50 manages events (timer events, message reception, etc.) and drives the task 21 based on the events. When driving this task 21, the task execution control unit 50 controls the transaction of the state accessed by the task through the state management unit 60. In addition, the task execution control unit 50 uses the inter-node communication unit 70 to collectively form an agreement between the nodes 1 on the processing order of one event (the version of the data to be accessed).

[0033] The state management unit 60 manages the state of the stateful server application 20 for each version. The state version is expressed as a transaction number, which is incremented by one each time a transaction is started.

[0034] The inter-node communication unit 70 executes a consensus protocol between the nodes 1. The event management queue 80 sequentially holds events that drive the tasks 21. The event management queue 80 can be configured, for example, as a FIFO.

[0035] The database management table 200 stored in the memory 12 holds the committed transaction numbers and the last started transaction numbers, and is managed by the state management unit 60. The database 100 can have one or more tables. The table management tables 210-1 to 210-3 manage the transaction states for each table in the database 100. These are managed by the state management unit 60.

[0036] The task management table 220 is managed by the task execution control unit 50, and manages the event that triggers the activation of each task, and the name of the table that the task is to update.

[0037] <Task model> 3 is a flowchart showing an example of the behavior of a task 21 running on node 1. The server application 20 of this embodiment is composed of one or more event-driven tasks 21. After being started, each task 21 first performs initialization such as securing resources (S6).

[0038] Next, each task 21 waits for an event (S7). An event is a timer event, reception of a message, satisfaction of a predetermined condition, or the like, and is set in advance. When a task 21 receives an event, it executes a unique process for the input (a request message received from the client application 2, a state in the database 100, etc.) and outputs a unique process result (S8). That is, the task 21 outputs the process result as a transmission message or updates the state of the database 100.

[0039] Then, when the task 21 completes the processing for one event, it returns to waiting for an event in step S7 and repeats the above processing. In this embodiment, the state (data) to be updated is unique to each task 21, and the same state is not updated by multiple tasks 21. Therefore, only one task 21 updates a specific state, and no update conflict occurs. Other tasks can also reference the state.

[0040] 4 is a diagram showing an example of an event drive pattern of task 21. Task A (21-A) is driven by its own periodic timer event. The timer event information can include the target task ID, period, wake-up time, etc. Task A (21-A) is executed repeatedly every 100 ms.

[0041] For task 21-A, whose event is timer-driven, a consensus can be reached among nodes 1 collectively on the version of the state to be accessed before the timer starts to operate (count up) after the timer is started. This can reduce the risk of processing delays and missed cycles due to waiting for consensus to be reached, including among follower nodes 1-2 and 1-3.

[0042] Task B (21-B) drives task C (21-C) by a timer event. In the example shown, task C (21-C) is executed 100 ms after task B (21-B) sets the timer. The target task ID and wake-up time can be set as information for the timer event.

[0043] Task D (21-D) is driven by a message reception event. The target task ID, received message, etc. can be set as event information. Task E (21-E) wakes up task F (21-F) when a certain condition is met. The target task ID, wake-up cause, etc. can be set as event information.

[0044] <External Communication Unit 40> 5 is a diagram showing an example of a receiving process performed by the external communication unit 40 of the node 1. The proxy 3 receives a request message 300 addressed to the server application 20 from the client application 2, adds a management header 310 to the message 300, and transfers the same message 300 to each node 1 on which the server application 20 is running.

[0045] 6 is a diagram showing an example of the header 310. The header 310 includes at least a connection ID 311 for distinguishing between a plurality of connections between the client computer 4 and the node 1, a message ID 312 which is a serial number of a message transmitted using the same connection, and a size 313 of the original message.

[0046] The connection ID 311 is assigned a unique ID, for example, as a tuple of the IP address of the client computer 4 and a serial number that is incremented each time a connection with the proxy 3 is established.

[0047] Upon receiving the request message 300, the external communication unit 40 waits for the completion of reception of one message's worth of data by referring to the message size 313 in the management header 310. Upon receiving one message's worth of data, the external communication unit 40 refers to the connection ID 311 to identify the task ID of the task 21 waiting for the message 300.

[0048] The external communication unit 40 generates an event for driving the task 21, and adds the message 300, from which the management header 310 has been removed, to the event management queue 80 in a manner linked to the generated event.

[0049] <State Management Unit 60> 7 is a diagram showing an example of the functions of the state management unit 60. The state management unit 60 manages multiple states in an MVCC database 100. MVCC has a function for reading and writing multiple versions of a data set in parallel.

[0050] The state management unit 60 manages the entire state shared between the tasks 21 in the database 100 on the shared memory 90. The state management unit 60 manages each state on a table-by-table basis for one or more tables included in one database 100. Also, as described above, this embodiment is based on the premise that there is only one task 21 that updates a specific state (table).

[0051] When a request to drive a task 21 occurs, the task execution control unit 50 requests the state management unit 60 to start a transaction for the task 21 to access a state based on a predetermined rule, and in response, obtains a handle for the state (table) and passes the handle to the task 21. Each task 21 references or updates the state via the state management unit 60 through the handle.

[0052] When the node 1 to which the task execution control unit 50 belongs is the leader node 1-1, the task execution control unit 50 starts a consensus formation process collectively for the access targets of the tasks 21 that constitute the transaction in conjunction with the processing. On the other hand, when the node 1 to which the task execution control unit 50 belongs is the follower node 1-2 or 1-3, the task execution control unit 50 requests the state management unit 60 to start a transaction based on the contents of the agreement after the consensus formation process is completed, and drives the task 21.

[0053] When task 21 finishes processing one event and returns to the event waiting state, task execution control unit 50 requests state management unit 60 to end the transaction of task 21. Upon receiving the request, state management unit 60 sets the number of the transaction in question in a management table (described later) and ends the transaction.

[0054] <Management table> 8 is a diagram showing an example of the database management table 200. The database management table 200 is a table for managing the entire database 100, which includes one or more tables 110.

[0055] The database management table 200 includes, in one record, a transaction status 201 that stores the state of a transaction, and an XNO 202 that stores the transaction number.

[0056] The maximum value of the committed transaction number is stored in a record whose transaction status 201 is "Committed." In the example shown, this indicates that transactions up to XNO 202 "102" have been committed. Note that even if a transaction with XNO 202 "104" has been committed, if a transaction with XNO 202 "103" has not yet been committed, then XNO 202 for "Committed" will be "102."

[0057] For records in which the transaction status 201 is "Last," the sequence number of the last read / write transaction (hereinafter, referred to as a RW transaction) that started is set in XNO 202 as the transaction number.

[0058] 9 is a diagram showing an example of the table management table 210. The table management tables 210-1 to 210-3 manage transactions for each table in the database 100. In this embodiment, an example is shown in which the database 100 has three tables, foo, bar, and baz.

[0059] The table management table 210 is made up of the following items: table name 211, Committed 212, and Inprogress 213. Committed 212 stores the transaction number (XNO) of the transaction that was last committed in the table.

[0060] The RW transaction number (XNO) currently in progress in the table is stored in InProgress 213. Note that if the transaction number is undetermined, such as at the start of a transaction, an invalid value (such as -1) is set.

[0061] <Database> FIG. 10 is a diagram showing an example of tables 110-1 to 110-3 of database 100. Table 110-1 has a table name of foo. Table 110-2 has a table name of bar. Table 110-3 has a table name of baz.

[0062] Each table 110 manages one state (shared data). The rows (records) of each table 110 correspond to each element when the state has a list or a Key-Value structure. Each row includes XNO111, Index112, and Value113.

[0063] XNO111 stores the number of the transaction that updated the row. Past rows can be left without being overwritten. For example, index = 1 of bar table 110-2 has left two update histories (XNO = 102, 104).

[0064] Index112 stores a sequential number for identifying rows within table 110. In the case where database 100 is of the Key-Value type, the Key can be stored. Value113 stores the data body. If database 100 is an RDB, it is composed of a column group, and if it is a KVS (Key-Value Store), the Value part is stored.

[0065] <MVCC method> The database 100 of this embodiment manages the data version in the MVCC method as described above and supports RO transactions and RW transactions.

[0066] An RO transaction gives only read access rights to all tables 110 of database 100 or to a specific table 110.

[0067] An RW transaction grants reference and update access rights to a specific table 110 within the database 100. For the table 110 targeted by the RW transaction, until the ongoing RW transaction in that table 110 is committed, the next RW transaction for that table 110 cannot be started. Also, on the premise that there is only one task 21 to update a certain state (table 110), a case where such an RW transaction cannot be started does not occur.

[0068] Also, in the MVCC method of this embodiment, the version of the entire database 100 is managed by a transaction number (XNO). Records within each table 110 are managed in a set with the transaction number XNO that updated the record.

[0069] In an RO transaction for all tables 110 with transaction number XNO = N, it is possible to refer to the committed latest records of all tables 110 where the transaction number XNO is N or less. Note that the RO transaction for a specific table with XNO = N will be described later.

[0070] In an RW transaction with transaction number XNO = N, the specified table 110 can be updated, and the committed latest records of the table 110 can be referred to. When the latest transaction number XNO is N, the state management unit 60 sets the number of the new RW transaction and the latest transaction number XNO to N + 1.

[0071] <RO Transaction Start Processing> FIG. 11 is a flowchart showing an example of the start processing of a read-only (RO) transaction. This processing starts when the state management unit 60 receives a start request for a read-only transaction from the task execution control unit 50.

[0072] The state management unit 60 judges whether or not a transaction number (XNO) is specified in the transaction start request from the task execution control unit 50 (S11). If a transaction number is not specified, the process proceeds to step S12, and if a transaction number is specified, the process proceeds to step S13.

[0073] In step S12, state management unit 60 refers to database management table 200, acquires XNO 202 for which transaction status 201 is "Committed", generates a handle corresponding to that transaction number, and responds to task execution control unit 50. In database management table 200 of Fig. 8, state management unit 60 acquires the transaction number of XNO 202 = 102, and generates a handle that refers to the latest row (record) in each table 110 whose transaction number is 102 or less.

[0074] In step S13, it is determined whether the specified transaction number is equal to XNO 202 whose transaction status 201 in database management table 200 is "Committed." If the specified transaction number is equal to XNO 202, state management unit 60 generates a handle for referencing each table 110 with the transaction number of XNO 202 and responds to task execution control unit 50.

[0075] On the other hand, if the specified transaction number is not equal to XNO 202, the state management unit 60 notifies the task execution control unit 50 of an error and ends the process.

[0076] Through the above process, the state management unit 60 generates a handle for referencing committed data for the entire database 100 and responds to the task execution control unit 50.

[0077] FIG. 12 is a diagram showing an example of a handle. The handle 510 generated in step S14 of FIG. 11 indicates that the target table 511 is the entire database 100, the transaction number is the transaction number specified by the transaction number (XNO), and the access mode to table 110 is read-only (RO).

[0078] FIG. 13 is a flowchart showing an example of a start request for a read-only transaction (hereinafter referred to as an RO transaction) to a specified table. This process starts when the state management unit 60 receives a start request for an RO transaction from the task execution control unit 50 for the specified table 110.

[0079] The state management unit 60 refers to the table management table 210 of the specified table 110 and determines whether the transaction number specified in the transaction start request is less than or equal to the transaction number of InProgress213 - 1, that is, whether it has been committed (S21).

[0080] If the state management unit 60 has been committed, it proceeds to step S22, generates a handle for the table 110, and responds to the task execution control unit 50. If it has not been committed, it proceeds to step S23 and notifies the task execution control unit 50 of an error.

[0081] <RO Transaction End Processing> FIG. 14 is a flowchart showing an example of an end request for an RO transaction. This process is executed when the state management unit 60 receives an end request for an RO transaction from the task execution control unit 50.

[0082] The state management unit 60 releases the handle for which the access has been completed and ends the process (S25).

[0083] Through the above processing, the state management unit 60 executes handle generation and release in the processes of starting and ending an RO transaction.

[0084] <RW Transaction Start Processing> FIG. 15 is a flowchart showing an example of a request to start an RW transaction for a specified table. This process is started when the state management unit 60 receives a request to start an RW transaction for the specified table 110 from the task execution control unit 50.

[0085] The state management unit 60 refers to the table management table 210 of the specified table 110 and determines whether the value of InProgress213 is an invalid value, that is, whether there is an RW transaction in progress (S31). If the value of InProgress213 is an invalid value, it is considered that there is no RW transaction in progress, and the process proceeds to step S32. If the value is not an invalid value (during transaction processing), the process proceeds to step S34.

[0086] In step S32, the state management unit 60 updates the value of XNO202 of the record whose transaction state 201 in the database management table 200 is LAST by adding 1, and uses it as the transaction number of the specified table 110. The state management unit 60 sets the transaction number in Inprogress213 of the table management table 210 of the specified table 110.

[0087] In step S33, a handle for referring to the specified table 110 is generated and responded to the task execution control unit 50.

[0088] On the other hand, in step S34, if the value of InProgress213 is not an invalid value in the table management table 210 of the specified table 110, since the current transaction is in progress, the state management unit 60 notifies the task execution control unit 50 of an error and ends the process.

[0089] By the above processing, when the specified table 110 is not being processed in a transaction, the state management unit 60 generates a handle for accessing the table 110 and responds to the task execution control unit 50.

[0090] FIG. 16 is a diagram showing an example of a handle. The handle 510 generated in step S33 of FIG. 15 indicates that the target table 511 is the specified table 110, the transaction number (XNO) is "105" which is the result of updating the XNO202 corresponding to LAST in the database management table 200, and the access mode to the table 110 is read / write.

[0091] An example of an RW transaction for a specific table 110 will be described below with reference to FIG. 9.

[0092] In the Inprogress213 of the table management tables 210-1 and 210-2 with table names 211 = "foo" and "bar", uncommitted RW transaction numbers are set, so a new RW transaction cannot be started.

[0093] When a request is made to start an RW transaction for the table 110 with table name 211 = "baz", the value obtained by adding 1 to "104" stored in the XNO202 where the transaction state 201 of the database management table 200 is "Last", i.e., "105", is set to the XNO202 of Last in the database management table 200 and to the InProgress213 of the table management table 210 with table name 211 = "baz". Thereafter, when the table 110-3 with table name 213 = "baz" is updated in response to a transaction request, a row with XNO111 = "105" is added to the table 110-3.

[0094] <RW Transaction End Processing> 17 is a flowchart showing an example of a RW transaction end request. This process is started when the state management unit 60 receives a RW transaction end request (commit completion) from the task execution control unit 50.

[0095] The state management unit 60 receives the table 110 and transaction number of the termination request received from the task execution control unit 50, sets the received transaction number in Committed 212 of the table management table 210 for the corresponding table 110, and sets an invalid value in Inprogress 213 (S41).

[0096] The state management unit 60 refers to the database management table 200, and if the value of XNO 202 of a record whose transaction status 201 is "Committed" is the transaction number of Committed 212 in the table management table 210 that accepted the transaction termination request minus 1, it sets the Committed 212 and transaction number in the table management table 210 to XNO 202 of the record whose transaction status 201 is "Committed".

[0097] Next, in steps S43 to S46, state management unit 60 refers to database management table 200, and from the transaction number obtained by adding 1 to the value of XNO 202 of a record whose transaction status 201 is “Committed”, determines whether the transaction number of XNO 202 of a record whose transaction status 201 is “Last” has been committed, and updates database management table 200.

[0098] In step S43, the state management unit 60 sets the variable i to a value obtained by adding 1 to the XNO 202 of the record whose transaction status 201 is "Committed". In step S44, the state management unit 60 refers to the table management table 210 to determine whether or not the transaction number=i has been committed. If it has been committed, the state management unit 60 proceeds to step S45, and if not, ends the process.

[0099] In step S45, the state management unit 60 sets the value of the variable i to the XNO 202 of the record whose transaction status 201 in the database management table 200 is "Committed." In step S46, 1 is added to the variable i, and the process returns to step S44 to repeat the above process.

[0100] Through the above process, the transaction numbers of the committed RW transactions are set in the table management table 210 and the database management table 200, and the setting of the committed transaction is completed.

[0101] <Task execution control unit 50> 18 is a diagram showing an example of consensus formation performed between the nodes 1. The task execution control unit 50 takes the lead in executing the consensus formation process.

[0102] The task execution control unit 50 drives a given task 21 based on an event, and also controls the execution of the task 21 and transactions on the state management unit 60 so as to perform deterministic operations between the nodes 1 .

[0103] First, the task execution control unit 50 executes a consensus protocol via the inter-node communication unit 70 to reach an agreement between the nodes 1 on the event processing order and event information for the events in the event management queue 80 (S51). The event information here refers to a request source task ID 82, a target task ID, event details 84, and a handle 86 contained in an entry registered in the event management queue 80, which will be described later. The handle 86 includes information such as which version (transaction number XNO) of data in the table 110 of the database 100 the task 21 to be driven refers to and updates.

[0104] Next, the task execution control unit 50 of the leader node 1-1 drives the task 21 that is waiting for the event in the event management queue 80 in a FIFO (First In First Out) manner (S52). The task execution control units 50 of the follower nodes 1-2 and 1-3 drive the task 21 only for the agreed-upon event.

[0105] Prior to driving the task 21, the task execution control unit 50 requests the state management unit 60 to start an RO transaction or a RW transaction for the state (shared data) accessed by the task 21. In response to the request, the task execution control unit 50 receives a handle for access from the state management unit 60 (S53).

[0106] This handle is passed to the task 21 by the task execution control unit 50 when the task 21 is driven. The driven task 21 uses the handle passed from the task execution control unit 50 to access a state (data) of a specified version, and when the task 21 enters event waiting again, the task execution control unit 50 releases the handle and commits the RW transaction.

[0107] The task execution control unit 50 provides each task 21 with the functions of task registration, event registration, event waiting, task drive, and external transmission.

[0108] In the task registration, the task 21 constituting the server application 20 to be multiplexed is registered as a managed object of the task execution control unit 50. Then, the task execution control unit 50 calls this function when initializing the task 21. The task execution control unit 50 also registers a list of states (tables) to be updated by the task 21.

[0109] In event registration, the task execution control unit 50 adds to the event management queue 80 an event that drives the calling task 21 or another task 21 .

[0110] In the event wait state, the task execution control unit 50 puts the task 21 into a wait state until an event occurs that drives the calling task 21. In addition, the task execution control unit 50 performs transaction end processing for the handle held by the calling task 21.

[0111] Regarding task driving, when an event occurs that drives a task 21, the task execution control unit 50 drives a specific task 21 that is waiting for an event. At that time, a transaction start request is output to the state management unit 60, which generates a handle for accessing the state (table 110) that the task 21 refers to or updates. The task execution control unit 50 acquires the handle and passes it to the task 21.

[0112] When the node 1 is the leader node 1-1, the task execution control unit 50 of the node 1 collectively forms an agreement with the follower nodes 1-2 and 1-3 on the event processing order in parallel with the driving of the task 21. That is, in the leader node 1-1, by starting the task 21 before the consensus is formed, it is possible to suppress the delay required for consensus formation (synchronization) and output the processing result.

[0113] The leader node 1-1 also forms an agreement with the follower nodes 1-2 and 1-3 as to which version (transaction number XNO) of the data in table 110 of database 100 a task 21 driven by a single event will refer to and which version the data will be updated in.

[0114] By forming a collective agreement with follower nodes 1-2 and 1-3 on the version to be accessed, leader node 1-1 can avoid forming an agreement between nodes 1 every time task 21 accesses database 100, as in the conventional example, and can avoid delays due to synchronization.

[0115] On the other hand, when the node 1 is a follower node 1-2 or 1-3, the task execution control unit 50 drives the target task 21 based on the agreement result after the agreement is reached (including the process of generating the handle).

[0116] In the external transmission, the task execution control unit 50 transmits a response (message 300 including the processing result) to an external device such as the client computer 4 on behalf of the task 21. Furthermore, if consensus on the processing order for the event that drove the requesting task 21 has not been reached, the task execution control unit 50 suspends actual transmission until consensus is reached. Furthermore, if consensus on the processing order for the event that drove the requesting task 21 has failed, the task execution control unit 50 discards the processing result that was scheduled to be transmitted.

[0117] <Task initialization process> 19 is a sequence diagram showing an example of a task initialization process. When the task execution control unit 50 starts the task 21 (S61), it executes an initialization process (S62).

[0118] During the initialization process, each task 21 calls the "task registration" function of the task execution control unit 50 and registers the task 21 in question in the task management table 220 as a task 21 to be multiplexed. The task 21 in question also notifies the task execution control unit 50 of a list of states (table 110) that can be updated in the future. The task execution control unit 50 manages this information in the task management table 220.

[0119] When the initialization process is completed, task 21 enters an event waiting state (S63) and sleeps until a predetermined event (timer, message reception, drive request from another task, etc.) occurs.

[0120] 20 is a diagram showing an example of the task management table 220. The task management table 220 is managed by the task execution control unit 50.

[0121] The task management table 220 includes, in one record, a task ID 221, a waiting event 222, and an update target table 223. The task ID 221 stores an identifier of the task 21 that constitutes the server application 20.

[0122] The waiting event 222 stores an event that is a trigger for starting the task 21. The update target table 223 stores the identifier (or name) of the table 110 of the database 100 that is updated by the execution of the task 21.

[0123] When an event occurs, the task execution control unit 50 can start the task 21 corresponding to the waiting event 222 in the task management table 220 and notify the update target table 223 to the state management unit 60 .

[0124] <Task 1 event processing> 21 is a sequence diagram showing an example of task processing. In the example shown, the task 21 of the leader node 1-1 and the tasks 21 of the follower nodes 1-2 and 1-3 are each started and are in a state of waiting for an event (S71).

[0125] <Leader node processing> First, a process performed by the leader node 1-1 will be described. The leader node 1-1 accepts an event for which the task 21 is waiting. In the leader node 1-1, the task execution control unit 50 registers the event in the event management queue 80 (S72).

[0126] The task execution control unit 50 refers to the task management table 220, identifies the task ID 221 corresponding to the waiting event 222, and requests the state management unit 60 to start a transaction (S73). The transaction is an RO transaction for the table referenced by the task 21 corresponding to the task ID 221 or for all tables, or a RW transaction for the table indicated in the update target table 223.

[0127] The state management unit 60 generates a handle according to the transaction type (RO, RW) and the table 110 to be accessed, and responds to the task execution control unit 50 (S74). The task execution control unit 50 acquires a handle for the state (table 110) to be accessed.

[0128] The task execution control unit 50 forms an agreement with the other nodes 1 as to which version of the state (table 110) the task 21 will refer to or update (S76).

[0129] The task execution control unit 50 drives the task 21 (S77) without waiting for an agreement to be reached with the other node 1. The task execution control unit 50 passes to the task 21 the handle acquired in step S74.

[0130] The task 21 accesses (references, updates) the state (table 110) via the handle received from the task execution control unit 50 (S78). The task 21 accesses the table 110 of the database 100 stored in the shared memory 90 in a non-blocking manner.

[0131] The task 21 requests the task execution control unit 50 to transmit the processing result to the outside (S80).

[0132] After starting the task 21, the leader node 1-1 receives a notification of the formation of a consensus from the follower node 1-2 (S79).

[0133] After confirming that the consensus formation received in step S79 has been completed, the task execution control unit 50 executes transmission to the outside (proxy 3) (S81). If the consensus formation is incomplete, the task execution control unit 50 suspends transmission until it is completed. This prevents different processing results from being transmitted between nodes 1 if the consensus formation fails and another node 1 takes over as leader. If the consensus formation fails, the task execution control unit 50 discards the processing result as described above.

[0134] When the processing of one event is completed, task 21 transitions to an event waiting state and goes into a sleep state again (S82).

[0135] After confirming that consensus has been reached with the other node 1, the task execution control unit 50 outputs a request to terminate the transaction including the task 21 to the state management unit 60 (S83), and the state management unit 60 sets the commit completion in the database management table 200 and the table management table 210.

[0136] The task execution control unit 50 deletes the event for which processing has been completed from the event management queue 80, and ends processing for one event (S84).

[0137] <Follower node processing> Next, a process performed by the follower node 1-2 will be described. In the following description, since the follower nodes 1-2 and 1-3 are similar, only the follower node 1-2 will be described.

[0138] As described above, the task 21 started in the follower node 1-2 is in an event waiting state (S71), and the task execution control unit 50 accepts an event and registers it in the event management queue 80 in step S72.

[0139] In response to the request for consensus formation from the leader node 1-1, the task execution control unit 50 verifies that the event for driving the task 21 has been registered in the event management queue 80, and then returns a request for consensus formation (S79).

[0140] After reaching an agreement, the task execution control unit 50 refers to the task management table 220 based on the agreement, identifies the task ID 221 corresponding to the waiting event 222, and requests the state management unit 60 to start a transaction (S85). The transaction starts with a specified table 511, a specified transaction number 512, and a specified mode 513 (RO or RW) based on the handle 510 information included in the agreement with the leader node 1-1.

[0141] The state management unit 60 generates a handle according to the transaction type (RO, RW) and the table 110 to be accessed, and responds to the task execution control unit 50 (S86). The task execution control unit 50 acquires a handle for the state (table 110) to be accessed.

[0142] The task execution control unit 50 drives the task 21 (S87). The task execution control unit 50 passes to the task 21 the handle acquired in step S86.

[0143] The task 21 accesses (references, updates) the table 110 (state) via the handle received from the task execution control unit 50 (S88). The task 21 accesses the table 110 of the database 100 stored in the shared memory 90 in a non-blocking manner.

[0144] The task 21 requests the task execution control unit 50 to transmit the processing result to the outside (S89). The task execution control unit 50 executes the transmission to the outside (proxy 3) (S89, S90).

[0145] When the processing of one event is completed, task 21 transitions to an event waiting state and goes into a sleep state again (S91).

[0146] The task execution control unit 50 outputs a request to end the transaction of the task 21 to the state management unit 60, and the state management unit 60 sets the commit completion in the database management table 200 and the table management table 210 (S92).

[0147] The task execution control unit 50 deletes the event for which processing has been completed from the event management queue 80, and ends processing for one event (S93).

[0148] <Transaction End Request from Task Execution Control Unit 50> In the above example, the task execution control unit 50 implicitly outputs a transaction end request to the state management unit 60 in conjunction with the event wait of the task 21, but the task 21 may explicitly output a transaction end request.

[0149] For example, when task A (21-A) updates state T1 and subsequently drives task B (21-B), in order to ensure that the update result of state T1 is referenced by task 21-B, task A must explicitly commit the update of state T1 before driving task B.

[0150] After committing state T1, task A cannot update state T1 until the next event-driven transaction is started again. If there is no need for task B to refer to the update results of state T1, there is no need for task A to explicitly commit. In this case, task B will refer to state T1 before the update.

[0151] <Event registration process of the task execution control unit 50> FIG. 22 is a flowchart showing an example of an event registration process performed by the task execution control unit 50. As shown in FIG.

[0152] First, the task execution control unit 50 generates an event based on a request from the task 21 and registers it in the event management queue 80 (S101). The generation of an event will be described later.

[0153] The task execution control unit 50 judges whether the node 1 is the leader node 1-1 or not (S102). If it is the leader node 1-1, the process proceeds to step S103, and if it is the follower node 1-2 or 1-3, the process proceeds to step S106.

[0154] In step S103, the task execution control unit 50 issues a request to the state management unit 60 to start an RO transaction or RW transaction used by the task 21 to be driven based on a predetermined rule, and receives an access handle as a response from the state management unit 60. A reference to this handle is set in the event generated in step S101 above. A method for setting a handle in an event will be described later.

[0155] The task execution control unit 50 of the leader node 1-1 starts consensus building with the follower nodes 1-2 and 1-3 regarding the event processing order and event information (S104). The consensus building is performed in the order in which the events are registered in the leader node 1-1.

[0156] Then, the leader node 1-1 starts driving the target task 21 without waiting for the completion of consensus (S105). When the contents of the RW transaction of task 21 are finally reflected (committed) in the state or when they are to be transmitted externally, the task execution control unit 50 waits for the completion of consensus, so there is no problem in continuing the processing of task 21 up to that point. In the unlikely event that consensus formation fails, the task execution control unit 50 discards the transaction processing results and external transmission data, so that an erroneous processing result is not output. The handle of step S103 is passed to the task 21 driven by the task execution control unit 50.

[0157] In step S106, when the node 1 is the follower node 1-2, the follower node 1-2 must wait for the completion of consensus formation for the event before generating a handle or driving the task 21.

[0158] If a request for consensus has already been received from the leader node 1-1, the follower node 1-2 determines whether consensus is possible (whether an event with the same content exists in the event management queue 80) and responds with the determination result to the other nodes 1 (S106).

[0159] The task execution control unit 50 judges whether or not an agreement has been reached (S107). If the agreement on the event has been reached, the task execution control unit 50 outputs a request to start an RO transaction or RW transaction with a predetermined transaction number to the state management unit 60 in accordance with the contents of the agreement, and obtains a handle as a response from the state management unit 60 (S108).

[0160] Then, the task execution control unit 50 drives the task 21 driven by the event, and passes the handle acquired in step S108 to the task 21 (S109).

[0161] By the above process, an event is registered in the event management queue 80, and the task execution control unit 50 passes the handle from the state management unit 60 to the task 21, and the task 21 is driven.

[0162] <Event management queue 80 configuration> 23 is a diagram showing an example of the event management queue 80. The event management queue 80 includes, in one record, an ENO 81 that stores an event identifier, a request source task ID 82, a target task ID 83, event details 84, consensus formation information 85, a handle 86, and a transmission queue 87.

[0163] ENO 81 stores a serial number assigned by the leader node 1-1 in the order in which events occur. The follower nodes 1-2 and 1-3 generate events without setting ENO 81. They then refer to the consensus information sent from the leader node 1-1 to search for events with matching content, and set the ENO value included in the consensus information in the event management queue 80 of their own node 1.

[0164] The request source task ID 82 stores the identifier of the task that generated the event. The target task ID 83 refers to the event details 84 passed from the request source task ID 82, searches for a task 21 waiting for an event under the same conditions, and sets the identifier of the task 21. The event details 84 stores the type and period of the event shown in FIG.

[0165] The consensus information 85 stores information used in the consensus protocol. It is assumed that any leader-follower type consensus protocol such as RAFT is used for consensus. The consensus information 85 includes at least the consensus content received from other nodes 1, information on whether consensus is complete or not, etc. The consensus content includes information on the event (ENO, requesting task ID 82, target task ID 83, event details 84, handle 86).

[0166] The handle 86 stores information for referencing or updating the state via the state management unit 60. That is, it is zero or more handles 510 shown in Fig. 16, and includes information such as the identifier of the table to be accessed, the transaction number, and whether RO or RW. The follower nodes 1-2 and 1-3 refer to the handle information included in the consensus formation contents received from the leader node 1-1, and request the state management unit 60 to generate a handle with the same contents.

[0167] The transmission queue 87 stores an external transmission message (processing result) by the task 21 driven by the event. When there is an external transmission request from the task 21 and the consensus formation is not yet complete, the task execution control unit 50 holds the transmission message in the transmission queue until the consensus formation is complete.

[0168] <Transaction start processing of leader node> 24 is a flow chart showing an example of a transaction start process performed in the leader node 1-1. This process is executed when the task execution control unit 50 starts a transaction.

[0169] First, in step S111, the task execution control unit 50 outputs a request to start an RO transaction without specifying a transaction number (XNO 202). Since the transaction number (XNO 202) is not specified, the value of XNO 202 of Committed in the database management table 200 is used as the transaction number.

[0170] The state management unit 60 generates a handle in response to a request to start an RO transaction, and the task execution control unit 50 obtains the handle from the state management unit 60 .

[0171] Next, the task execution control unit 50 refers to the task management table 220, refers to the row of the task ID 221 to be driven, and generates the contents of the update target table 223 as a list (S112).

[0172] In steps S113 to S115, the task execution control unit 50 repeatedly executes the process of step S114 for the table 110 in the list generated in step S112. In step S114, the task execution control unit 50 outputs a request to the state management unit 60 to start a RW transaction for the table T currently being processed, and obtains a handle from the state management unit 60's response.

[0173] Next, the task execution control unit 50 judges whether the task 21 to be driven this time is driven by another task (S116). If the task 21 to be driven is the result of a drive event from another task 21, the process proceeds to step S117, and if not, the process proceeds to step S122.

[0174] The above determination can be made by the task execution control unit 50 referring to the event management queue 80, and if the target task ID 83 of the task to be driven is included in the request source task ID 82, it can determine that the other task 21 is to be driven.

[0175] When task A (21-A) drives another task B (21-B), in order to guarantee the order of state updates and references, it is necessary for the driving task 21-A to provide the updated contents to the driven task B.

[0176] For example, if the RO transaction number XNO in step S111 above is 100, and the transaction number XNO of table 110 updated by driving task A is 103, then another task B cannot reference the contents updated by driving task A using the handle obtained in step S111 (it can only reference older contents).

[0177] Therefore, the task execution control unit 50 designates the table in question as the table 110 to be accessed, and requests the start of an RO transaction with transaction number XNO=103. In this case, the RO transaction with XNO=103 for the table in question cannot be started until the driving task A ends (commits) the RW transaction with XNO=103. Therefore, the processing is resumed after the transaction with transaction number XNO=103 is completed (committed).

[0178] In step S117, the task execution control unit 50 refers to the task management table 220, obtains the value of the update target table 223 from the row of the task ID 221 of the driving source task A, and generates a list of accessible tables.

[0179] Next, in steps S118 to S121, the task execution control unit 50 repeatedly executes the processes of steps S119 to S120 for the table 110 to be updated in the list generated in step S117. This process is for enabling the driven task B to refer to the state updated by the driving task A in a read-only manner.

[0180] In step S119, the task execution control unit 50 determines whether the transaction number (XNO202) of the table 110 to be updated in the list is greater than the transaction number of the table 110 corresponding to the handle acquired in step S111 and whether the table 110 has been committed. If the table 110 has been committed, the process proceeds to step S120. If not, the process proceeds to step S121.

[0181] Even when task A drives task B, which drives task C, thereby driving another task 21 in a chain reaction, task C needs to be able to refer to the update results of task A. Therefore, not only the RW transactions of the driving task A, but also the RO transactions for a specific table 110 need to be handed over to the driving task B.

[0182] However, if the transaction number (XNO) of the table 110 currently being processed becomes smaller than the transaction number (XNO) of the RO transaction that targets all tables 110 for which the handle was obtained in step S111, the desired state can be referenced using the handle obtained in step S111, so there is no need to request the start of a new transaction.

[0183] In step S120, the task execution control unit 50 outputs an RO transaction start request for the currently processed table T to the state management unit 60, and obtains a handle for table T from the state management unit 60.

[0184] If the commit has not been completed for the transaction to be updated, the start of the transaction will fail, but it will be ignored. In this case, it is considered that the intention is not to have the update contents of the state referenced by the driven task B, so there is no problem from the perspective of application logic even if the driven task B is made to reference the state before the update.

[0185] The handle information acquired by the task execution control unit 50 in step S120 is set as the handle 86 of the event management queue 80 corresponding to the event (S122). The handle 86 is used (1) to convey transaction information to the follower nodes 1-2 and 1-3 as the content of consensus formation, or (2) for the task execution control unit 50 to implicitly end the transaction when the task 21 completes the processing of the event and enters event waiting again.

[0186] <Transaction start processing of follower node> 25 is a flowchart showing an example of a transaction start process performed by the follower nodes 1-2 and 1-3. This process is started after the follower nodes 1-2 and 1-3 reach an agreement in response to a consensus building request from the leader node 1-1.

[0187] In step S131, the task execution control unit 50 acquires a handle from the consensus formation information from the leader node 1-1. The consensus formation information includes the handle information acquired by the leader node 1-1. Thereafter, by referring to this handle, it is determined which table 110 and which transaction number (XNO 202) to start a transaction with.

[0188] In step S132, the task execution control unit 50 specifies the same transaction number (XNO202) as that of the leader node 1-1 from the handle information acquired in step S131 above, outputs a request to start an RO transaction to refer to all tables 110 to the state management unit 60, and acquires the handle generated by the state management unit 60.

[0189] If the transaction number (XNO 202) of a row in database management table 200 where transaction status 201 is "Committed" is smaller than the transaction number (XNO) for which the handle was obtained, the transaction cannot be started. In this case, the process waits until the ongoing transaction is completed and Committed becomes XNO before resuming processing.

[0190] The table 110 to be accessed includes not only the table 110 that can be updated by the driven task 21, but also the table 110 updated by the driving task A when another task B is driven from task A. The driven task 21 is a RW transaction, and when referencing the table 110 updated by the driving task A, it is an RO transaction. As with step S132, there are cases where a new transaction cannot be started until the ongoing RW transaction ends (commits), so a suitable wait is performed before resuming.

[0191] In step S133, the task execution control unit 50 refers to the task management table 220, refers to the row of the task ID 221 to be driven, and generates the contents of the update target table 223 as a list.

[0192] In steps S134 to S136, the task execution control unit 50 repeatedly executes the process of step S135 for the tables 110 in the generated list. In step S135, the task execution control unit 50 outputs a request to the state management unit 60 to start an RO transaction and a RW transaction for the table 110 currently being processed, and obtains a handle from the state management unit 60 response.

[0193] In step S137, all the handles acquired in step S135 are set as the handles 86 of the event management queue 80 of the task 21.

[0194] Through the processes in FIGS. 24 and 25, transactions are started in the leader node 1-1 and the follower nodes 1-2 and 1-3, respectively.

[0195] <If consensus fails> Consensus formation between nodes 1 may fail. The cause of the failure of consensus formation may be a communication error or hardware failure that causes communication messages (consensus formation information) to be lost or delayed among only some of the nodes 1, making it impossible to form a consensus within a specified time (timeout).

[0196] In well-known or publicly known examples, consensus is completed when a majority of nodes 1, including the node itself, agree. At this time, the majority of nodes 1 who reach an agreement are considered to have succeeded in reaching an agreement, while nodes 1 who do not agree are considered to have failed.

[0197] When consensus formation fails, the node 1 that failed to reach consensus is excluded, and the remaining nodes 1 re-elect a leader. Note that the method of re-electing a leader is well known or is included in a known consensus formation protocol, so details are omitted here.

[0198] The excluded node 1 performs recovery processing as appropriate. For example, in the case of the former leader node 1-1, the processing of the task 21 that was being executed in advance is interrupted, the RW transaction is discarded without being committed, and the transmission data is also discarded. Then, the task execution control unit 50 starts the transaction again based on the agreement result, and restarts the task 21.

[0199] <External Communication Unit 40> FIG. 26 illustrates an example of the transmission process.

[0200] The external communication unit 40 of each node 1 obtains the processing order, the message 302 for external transmission associated with the agreed-upon event 810 in the event information, and the destination information, and transmits the message to the destination (proxy 3). In this case, the external communication unit 40 assigns a connection ID and a message ID to the header 310 of the message 300.

[0201] The connection ID is a unique ID assigned by the proxy 3 when a connection is established between the client application 2 and the proxy 3. The message ID is an ID (serial number) uniquely assigned to the message 300 transmitted by the external communication unit 40 through the connection.

[0202] Focusing on a certain connection, it is expected that each node 1 will send the same message 300 with the same message ID to the proxy 3. The proxy 3 checks the connection ID and message ID to confirm that it has received the same message multiple times, selects one message based on a predetermined policy, and transfers the message 300 with the header 310 removed to the client application 2. The policy may be, for example, first come first served or majority vote.

[0203] With the above configuration, even if some of the nodes 1 stop due to a failure before sending to the outside, it is possible to obtain the processing result message 300 from other nodes 1 with which an agreement has been formed, and the client can continue processing smoothly without timeouts and retries. EXAMPLES

[0204] In the first embodiment, if there is a task 21 that takes a long time to process, it may take a long time from the start to the end (commit) of the RW transaction. Even if several RW transactions for other tables occur before this process is completed, the RO transaction for the entire state (all tables) cannot refer to those update contents. This is because the RO transaction for the entire state does not refer to data updated after the transaction number that has been committed in consecutive numbers, so as not to refer to uncommitted data.

[0205] Therefore, in the second embodiment, the state management unit 60 leaves the transaction number (XNO202) of the RW transaction undetermined until the end of the commit. When the task execution control unit 50 of the leader node 1-1 commits the RW transaction, a consensus is reached between the nodes 1 as to which handles to commit and in what order. The task execution control unit 50 executes the actual commit only when a consensus is reached.

[0206] As a result of the above, each node 1 always updates the committed transaction numbers in the order of commit, so even if there is a task 21 that continues to hold a RW transaction for a long time, RO transactions are possible for the entire state of the latest version.

[0207] The difference from the first embodiment is that part of the processing of the state management unit 60 and part of the processing of the task execution control unit 50 are changed, and the other configurations are the same as those of the first embodiment.

[0208] Fig. 27 is a flowchart showing an example of a request to start a RW transaction. This process is the same as that shown in Fig. 15 in the first embodiment except that steps S32 and S33 are replaced with steps S141 and S142.

[0209] The state management unit 60 refers to the table management table 210 of the specified table 110 and determines whether the value of InProgress 213 is an invalid value, that is, whether processing has started and the transaction number is undetermined (S31). If the value of InProgress 213 is an invalid value, the process proceeds to step S141, and if it is not an invalid value (transaction is being processed), the process proceeds to step S34.

[0210] In step S141, the state management unit 60 sets a value meaning "undecided" (for example, -2) to the value of XNO 202 of the record whose transaction status 201 in the database management table 200 is LAST. The state management unit 60 sets an "undecided" value to Inprogress 213 in the table management table 210 of the specified table 110.

[0211] In step S142, a handle 510 that references the table 110 is generated and responded to the task execution control unit 50. As shown in FIG. 28, the handle 510 has an undefined value of "-2" set in the transaction number XNO.

[0212] On the other hand, in step S34, if the value of InProgress 213 in the table management table 210 for the specified table 110 is not an invalid value, a transaction is currently being processed, so the state management unit 60 notifies the task execution control unit 50 of an error and terminates the processing.

[0213] By the above process, if the specified table 110 is not currently undergoing transaction processing, the state management unit 60 sets an undefined value to the handle for accessing the table 110 and responds to the task execution control unit 50 .

[0214] Then, when the RW transaction ends, the state management unit 60 adds 1 to the value of Committed in the database management table 200 and updates the XNO 202 of the transaction from an "undefined value" to the value of "Committed+1". Accordingly, for records in the table 110 to be updated that have a transaction number XNO of "undefined", the transaction number XNO is updated to the new value of "Committed+1".

[0215] On the other hand, the process of the task execution control unit 50 is changed as follows.

[0216] When the task execution control unit 50 substitutes for the end (commit) of a RW transaction, it adds a process of forming a consensus among the nodes 1 regarding the order of the handles to be committed.

[0217] Then, after the consensus on the order of commit is completed, the task execution control unit 50 executes the commit in that order. Neither the leader node 1-1 nor the follower nodes 1-2 and 1-3 commit until the consensus is completed. As a result, the task execution control unit 50 also suspends the external transmission of the processing results.

[0218] While this consensus is being processed, task 21 is allowed to proceed without blocking (since failure to reach consensus does not affect task execution).

[0219] As described above, in the second embodiment, since the transaction numbers of node 1 are always updated in the order of commit, even if there is a long-running transaction, RO transactions are possible for the entire state of the latest version.

[0220] <Conclusion> As described above, the multi-processing system of the above embodiment can be configured as follows.

[0221] (1) A multi-processing system comprising a plurality of nodes (1), each including a processor 11, a memory 12, and a communication device (communication interface 13), the plurality of nodes (1) being connected via a network (5), each of the plurality of nodes (1) executing one or more tasks (21) that provide a unique output in response to an input, the plurality of nodes (1) each having a task execution control unit (50) that receives an event and initiates a transaction for executing the task (21) corresponding to the event, and a state management unit (60) that manages the version of data (data in a table 110) in a database (100) accessed by the task, the task execution control unit (50) forming a consensus among the nodes (1) on the version of data referenced by the task (21) in the database (100) or the version of data updated by the task (21) prior to starting processing of the task (21).

[0222] With the above configuration, by forming an agreement between the nodes (1) constituting the multiplexed system as to which version of data to reference and which version to update the data as before starting processing of task 21, it is no longer necessary to make the order of reference to or updates to data consistent between the nodes (1) every time access is made, as in the conventional example, and it is possible to reduce the delay required for synchronization.

[0223] (2) The multi-processing system according to (1),

[0224] The multi-processing system is characterized in that the task execution control unit (50) collectively forms an agreement among the nodes (1) regarding data access in processing one of the events.

[0225] With the above configuration, by forming an agreement between the nodes 1 collectively regarding the data to be accessed, it is no longer necessary to make the order of references or updates to the data consistent between the nodes (1) each time access is made, as in the conventional example, and the delay required for synchronization can be reduced.

[0226] (3) A multi-processing system as described in (2) above, characterized in that, when the event is a timer event, the task execution control unit (50) forms an agreement among the nodes (1) collectively regarding the data to be accessed before the timer is activated.

[0227] With the above configuration, in the case of a timer event, the task execution control unit 50 can reduce the delay required for synchronization between nodes 1 by forming an agreement in advance on the version of the data to be accessed before the timer starts.

[0228] (4) A multi-processing system as described in (1) above, wherein the node (1) includes a leader node (1-1) and follower nodes (1-2, 1-3), and the task execution control unit (50) of the leader node (1-1), upon receiving the event, requests the follower nodes (1-2, 1-3) to reach an agreement on the data accessed by the task (21) corresponding to the event, and starts the task (21) before the agreement is reached.

[0229] With the above configuration, the leader node 1-1 starts executing task 21 before consensus is reached, and the follower nodes 1-2 and 1-3 start executing task 21 after consensus is reached, thereby enabling at least the leader node 1-1 to avoid delays due to waiting for consensus to be reached.

[0230] (5) A multi-processing system as described in (4) above, characterized in that, if the reaching of the agreement fails, the task execution control unit (50) of the leader node (1-1) interrupts the transaction that has started processing and discards the processing result of the transaction.

[0231] With the above configuration, each node 1 always updates the committed transaction numbers in the order of commit, so that even if there is a task 21 that continues to hold RW transactions for a long time, RO transactions are possible for the entire data (state) of the latest version.

[0232] (6) A multi-processing system as described in (1) above, characterized in that the state management unit (60) assigns a transaction number (XNO) to the transaction as a number for identifying the transaction, and manages the version of data in the database (100) processed by the transaction using the transaction number (XNO).

[0233] With the above configuration, the database 100 is managed by the MVCC method, and the version of the data to be accessed is collectively agreed upon in advance between the nodes 1, thereby reducing the load required for synchronization.

[0234] (7) A multi-processing system as described in (6) above, characterized in that, when the transaction is a read / write transaction, the state management unit (60) sets the transaction number of the read / write transaction to an undetermined value until the commit of the read / write transaction is completed, and when the task execution control unit (50) commits the read / write transaction, an agreement is formed between the nodes (1) regarding the order of data to be accessed in the read / write transaction.

[0235] With the above configuration, each node 1 always updates the committed transaction numbers in the order of commit, so that even if there is a task 21 that continues to hold a RW transaction for a long time, RO transactions are possible for the entire state of the latest version.

[0236] The present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment is described in detail to easily explain the present invention, and is not necessarily limited to those having all the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, the addition, deletion, or replacement of other configurations can be applied to a part of the configuration of each embodiment, either alone or in combination.

[0237] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations and functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0238] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0239] 1-1~1-3 nodes 2 Client Applications 3. Proxy 4 Client computer 5. Network 11 Processors 12. Memory 20 Server Applications 21-A~21-C Tasks 30 Multiplexing Processing Unit 40 External Communications Department 50 Task execution control unit 60 State Management Department 70 Inter-node communication section 80 Event Management Queue 90 Shared Memory 100 Databases 110-1~110-3 Tables 200 Database Management Tables 210-1~210-3 Table Management Table

Claims

1. A multi-processing system having a plurality of nodes, each including a processor, a memory, and a communication device, said plurality of nodes being connected via a network, said plurality of nodes each executing one or more tasks that provide a unique output in response to an input, The node: a state management unit for managing the version of data in a database accessed by the task; a task execution control unit that receives an event and initiates a transaction for the task corresponding to the event to access the database, The task execution control unit Before at least some of the plurality of nodes start processing the task, a consensus is formed among the nodes regarding a version of data that the task refers to in a transaction in the database or a version of data that the task updates in a transaction, with respect to access to data in processing one of the events; A multi-processing system characterized in that, if the event is a timer event, a consensus is formed among the nodes collectively regarding the data to be accessed before the timer starts to operate.

2. A multi-processing system having a plurality of nodes, each including a processor, a memory, and a communication device, said plurality of nodes being connected via a network, said plurality of nodes each executing one or more tasks that provide a unique output in response to an input, The node: a state management unit for managing the version of data in a database accessed by the task; a task execution control unit that receives an event and initiates a transaction for the task corresponding to the event to access the database, The task execution control unit Before at least some of the plurality of nodes start processing the task, a consensus is formed among the nodes regarding a version of data that the task refers to in a transaction in the database or a version of data that the task updates in a transaction, with respect to access to data in processing one of the events; The state management unit is A multi-processing system comprising: a transaction number for identifying said transaction; and a version of data in said database processed by said transaction being managed by said transaction number.

3. A multi-processing system having a plurality of nodes, each including a processor, a memory, and a communication device, said plurality of nodes being connected via a network, said plurality of nodes each executing one or more tasks that provide a unique output in response to an input, The node: a state management unit for managing the version of data in a database accessed by the task; a task execution control unit that receives an event and initiates a transaction for the task corresponding to the event to access the database, The task execution control unit Before at least some of the plurality of nodes start processing the task, a consensus is formed among the nodes on the version of data that the task refers to in a transaction in the database or the version of data that the task updates in a transaction; The state management unit is assigning a transaction number to the transaction as a number for identifying the transaction, and managing the version of the data in the database processed by the transaction using the transaction number; If the transaction is a read / write transaction, the transaction number of the read / write transaction is left undetermined until the commit of the read / write transaction is completed; The task execution control unit, a plurality of nodes for accessing data in a read / write transaction; a plurality of nodes for accessing data in a read / write transaction;

4. A method for controlling a multi-processing system having a plurality of nodes, each including a processor, a memory, and a communication device, said plurality of nodes being connected via a network, said plurality of nodes each executing one or more tasks that provide a unique output in response to an input, comprising: a state management step in which the node manages the version of data in the database accessed by the task; a task execution control step in which the node receives an event and initiates a transaction for the task corresponding to the event to access the database; The task execution control step includes: Before at least some of the plurality of nodes start processing the task, a consensus is formed among the nodes regarding a version of data that the task refers to in a transaction in the database or a version of data that the task updates in a transaction, with respect to access to data in processing one of the events; A method for controlling a multiprocessing system, characterized in that, if the event is a timer event, a consensus is formed among the nodes collectively regarding data to be accessed before a timer is activated.

5. A method for controlling a multi-processing system having a plurality of nodes, each including a processor, a memory, and a communication device, the plurality of nodes being connected via a network, the plurality of nodes each executing one or more tasks that provide a unique output in response to an input, comprising: a state management step in which the node manages the version of data in the database accessed by the task; a task execution control step in which the node receives an event and initiates a transaction for the task corresponding to the event to access the database; The task execution control step includes: Before at least some of the plurality of nodes start processing the task, a consensus is formed among the nodes regarding a version of data that the task refers to in a transaction in the database or a version of data that the task updates in a transaction, with respect to access to data in processing one of the events; The state management step includes:

13. A method for controlling a multi-processing system, comprising the steps of: assigning a transaction number to each transaction as a number for identifying the transaction; and managing a version of data in the database processed by the transaction using the transaction number.

6. A method for controlling a multi-processing system having a plurality of nodes, each including a processor, a memory, and a communication device, the plurality of nodes being connected via a network, the plurality of nodes each executing one or more tasks that provide a unique output in response to an input, comprising: a state management step in which the node manages the version of data in the database accessed by the task; a task execution control step in which the node receives an event and initiates a transaction for the task corresponding to the event to access the database; The task execution control step includes: Before at least some of the plurality of nodes start processing the task, an agreement is formed among the nodes on a version of data that the task refers to in a transaction in the database or a version of data that the task updates in a transaction; The state management step includes: assigning a transaction number to the transaction as a number for identifying the transaction, and managing the version of the data in the database processed by the transaction using the transaction number; If the transaction is a read / write transaction, the transaction number of the read / write transaction is left undetermined until the commit of the read / write transaction is completed; The task execution control step includes: A method for controlling a multiprocessing system, comprising forming an agreement between said nodes regarding an order of data to be accessed in said read / write transaction when said read / write transaction is committed.

Images